There are lots of security tools available, so how do you know which one to pick?
If your security team is not including the application teams in the decision, you run a big risk of failure. The security team does get the ability to form relationships with vendors. We see them at conferences. We know people that work there. Because our focus is on security, we know the tools that exist in our space and we have an idea of which ones may be better than others. Of course, this is often due to what we hear from others, rather than our own experience with those tools.
Picking a tool in a vacuum is less than ideal. Sure, the tool may have five stars and has great detection rates. However, if the tool doesn’t fit into the development process and causes more overhead and friction, it will end up failing. A tool with 100% accuracy is still useless if it is not actually being used.
I have seen over and over where a security group acquires a tool, especially static analysis, and a few months later they realize it is just sitting on the shelf. Like all of us, we are excited when we get a new toy. We use it, learn it. Then, the newness wears off. Without the proper processes in place, this can happen with security tools as well. We don’t want this to happen to you.
The first step in determining which tools make sense is to understand the development process now and where it is going. Lets talk about static analysis for a moment. If your development process doesn’t make use of continuous integration and doesn’t plan to then that is not a high priority feature of the tool you receive. Have you considered what IDE the developers use?
I once had a situation where static analysis was about to be rolled out. The security team worked to pick a vendor and get the ball rolling. It was later in the process that the development team was brought on board with the conversation. They asked questions that were less of a priority to the security team, but more of a priority to them. For example, The tool provided a plugin for their IDE. However, when digging deeper, the plugin ran a few versions behind. So what had appeared to be a good setup, now looked a little less efficient since the plugin may not work. This isn’t a deal breaker, but depending on how you were planning on this solution working, it adds friction to the process. The more friction, the more pain.
The next step is understanding who will ultimately be using this tool? For those that listen to the podcast on a regular basis and follow my blogs, you know I am a huge fan of the application teams having direct access to these tools. In my opinion, static analysis is a developers tool. It is there to evaluate the source code and identify policy violations that only the developer can resolve. Giving them access and control over that function and embedding it into their process reduces that friction. Don’t confuse this with the idea that security is not getting the results. Security still needs to have insight into what is going on with security flaws within the application. But to be efficient, the results of these tools need to fit into the flow of how development works. Not be one-off reports under different processes.
This is no different, in my opinion, than dynamic analysis or interactive analysis. These are tools that should be used by the QA group. The group that is responsible for testing and most likely already has automated testing capabilities. They are trained in identifying bugs, reporting, and tracking them. Building these types of tools into their process just makes sense. How many listeners have their own stories of the security team exporting a thousand page report out of that dynamic scanner and sending it to the application team as a pdf? I have been there. Even I won’t read through that report.
The moral of the story here is that if we don’t understand how development works, what type of tools they already use, and what they can handle, we will probably pick a solution that will be ineffective, or at the very least, cause us a lot more work. The goal is not to just shove tools onto the development teams and say do this. As application security representatives, our goal is to help build better applications. That doesn’t mean that our job is to run all the tools and hound the application teams to fix items. By understanding the environments and processes we can pick tools that will fit much better, allowing us to focus our time on building out other solutions or processes for the organization. We are still relied upon to provide the expert advice and guidance to the issues that are identified.
If you are considering implementing these types of tools, take a moment to sit down with all involved parties to get everyone on board. A well laid out plan will go much further than a shotgun approach.