I've been reading web application vulnerability reports from tools and services for 6-7 years and found that 99% of these reports are geared towards security engineers or system administrators. Many of the reports I see focus on
- The type of flaw and what it its impact is
- The URL affected
- Links to references and additional reading
Security engineers typically aren't the ones fixing application level security defects and often file them into a development bug tracking system for development to address. A lot of the text is 'fluff' that is useful to security folks or those wanting a detailed explanation but overkill for those in development/qa. The fluff aside none of them seem to provide manual reproduction instructions (beyond reproducing the issue in their penetration testing tool) which doesn't allow for QA to regression test an issue in the
future. In my own experiences I've found myself needing to 'translate' these tool/services reports into language that makes sense to a developer/QA engineer (including writing manual reproduction instructions).
Many vendors are shifting to security testing throughout the development life cycle yet still aren't creating reports for the consumers within development. Development and QA are most interested in
- Knowing what application is affected.
- Knowing what parameters are affected.
- Knowing the flaw's impact on the safety of users, the site, and the application.
- Knowing how to test/reproduce the issue.
- Knowing the manual steps for reproduction.
- Knowing the vulnerable expected result.
- Knowing the non vulnerable expected result.
- Knowing how to fix the flaw.
I'm asking all vendors (product and services) reading this post to
- Consider your audience in your vulnerability summaries.
- Add *good* reproduction instructions
- Manual reproduction instructions so we can verifying things without needing to use your tool or service.
- Assume that the people consuming these reports have zero knowledge of tools such as Paros or burp proxy. Walk them through each step to reproduce the issue.Create reports even my mother could reproduce.
This site gets its fair share of product and service vendors readers so I'm considering this an open request to all of you. While this post can be seen as targeted towards a small audience I'm sure others have had the same experiences and frustrations and I welcome others to post what their experiences are.
Comments are open!
