Perhaps you are in the middle of a security rollout, or have new security initiatives in place. How do you know your project is successful? How do you identify problem spots, the areas which need fine-tuning or modifications? Simply put, what metrics do you have in place to help you understand the project's effectiveness?

Measurements aren't supposed to be easy. Threats change on a regular basis, and priorities can shift seemingly overnight. While it's critical to collect something, the kind of measurements taken is even more important. It's perfectly possible to say the email gateway blocked several thousand pieces of malicious content in a given week, or that 25 endpoints had their operating systems patched last month. But unless that means something for the CEO and the board of directors, they can't do anything with that fact. What exactly is being measured? The CFO has no idea whether that security investment was worth the amount spent. What success or failure can we take away from that number? Should the initiative continue?

That's the challenge for IT and security professionals: to develop security metrics and reporting that effectively communicates the successes, failures, and potential risks of a security program to a business audience.

The above example is a perfect example of what happens when the organization takes a tool-centric view of security rather than a risk-centric one. CISOs are measuring tactical things and collecting event-driven data, such as the amount of sensitive data blocked from being transferred out of the network, or the number of malware blocked by the firewall and antivirus. Senior executives and boards talk in the language of risk—they make their decisions based on the risk status. It's not possible to convert those event-driven numbers into a risk-based metric.

While we are seeing more organizations take a risk-based approach to securing the business, it is still limited. An effective CISO knows the importance of translating security concerns into risk when communicating with the board, but many of them are still struggling with collecting metrics that would be useful to the board and senior executives. In a study by Wisegate last June nearly half of the respondents said they didn't have a way to measure their top security risks. Security teams can’t easily measure if their top risks are increasing or declining, or if their efforts are having an effect on the risk, Bill Burns, former executive in residence at Scale Venture Partners who worked with Wisegate on the report, said at the time.

Many security products have built-in dashboards based on the specific threat they address, but they don't provide a way to aggregate and map the data to a holistic business impact. "This is like flying a plane with the three most important cockpit indicators taped over while you try to navigate over complex terrain and weather conditions," the report found.

Security professionals are aware that their security metrics and reporting processes are immature, and are looking for best practices and information on how they can collect the information they need, Burns said at the time.

That sentiment was alive and well at RSA Conference this year. There were several sessions addressing how to communicate with the board and discussing what kind of metrics CISOs need to have in hand. This is an area everyone is looking at their peers to find out what others are doing. There aren't straightforward answers, but perhaps we can make some headway towards figuring out what works and what doesn't work. Because not coming up with a workable, practical solution means we have no way to show whether our security efforts are actually working or beneficial.