Observations on Smoke Tests – Part 3

AppSec Program Management

While attending one of our technology partner’s security training courses, the instructor presented on their product’s various features and capabilities. Some of the discussion centered around application and vulnerability management. As a consultant who mainly focuses on security testing, these features seemed rather useless to me. The importance of application vulnerability management was not revealed until I gained career experience with larger, global enterprise clients. Some had very immature AppSec programs; for example, some were not completely aware of the number of their applications, which of them had been tested, or even how secure they were. Referring to Figure 1 in the second blog post of this series, having this level of program visibility and awareness provides risk and security managers with an overview of what they need to know in one shot (e.g., number of applications tested, the issues needed to be resolved, severity trends, risk exposure over time, etc.). 

Building out and integrating this sort of insight into your application security program is a big topic. It often requires thoughtful preparation of a risk management strategy and careful design of program metrics. Here is a quick tip: Never underestimate the complexity of application risk management. As your business grows, the sheer volume of potential vulnerabilities from security tools and processes integrated into your SDLC pipeline can become overwhelming. By leveraging the right expertise and technology, you can plan and define an effective vulnerability management strategy that balances the right amount of risk management with the resources and budget you have to work with. Fellow AppSec consultant, Shawn Asmus, recently wrote about key elements of an effective AppSec program, which you can read about here. 

Conclusion

Most of the security tools we use help us get the work done faster, including the application scanners we leverage for smoke testing. However, they alone will never deliver the same level of quality or assurance provided through comprehensive security testing. As mentioned in my other posts, there are a lot of issues that cannot be detected by automated tools, as well as the issue of false positives. That’s why full web application security assessments will always be necessary. 

There are numerous security tools in the market today, each with their pros and cons. Choosing the most suitable ones for your environment that satisfy your budget and technical needs, resource requirements, etc. can be challenging. Consulting with outside expertise and knowledgeable specialists can be very beneficial. 

That may sound cliché, but I’ve found that this simple advice hasn’t been recognized by many in the industry. In fact, the most frequent question people ask me when they first find out I am a security consultant is “what tools do you use?” This pertains not only to non-technical individuals but some developers and IT professionals. I usually explain that we are not tool users, and security testing is not just about running some tools. Sure, my response may include common tool names, and that might sound disappointing to some. But the true value is in our services which are constantly developing and improving.