There is no shortage of data out there. Virtually everything with a power source is logging events and churning out data almost constantly—including all of your security tools. That data—your security metrics—can uncover valuable truths about your security posture if used and analyzed properly, but it can also be very misleading or completely useless.

Aaron Levenstein is credited with this little tidbit of wisdom: “Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital.” The bottom line is that your security metrics can help you uncover issues with your security posture and make more effective decisions about how to improve it, but only if you’re considering the right security metrics for the right reasons.

Here are five ways that security metrics can actually do more harm than good for your organization’s security:

1. Collecting too much data
You can’t just collect security data for the sake of collecting it. You can quickly amass gigabytes, terabytes or more of security metrics and then you face the challenge of parsing and sifting through it all to try and ferret out the one or two valuable takeaways from it all.

2. Gathering useless data
Part of the solution for not gathering too much data is to make sure you’re only collecting data that has some relevant value. Some will argue that all data has value—it’s all in what questions you want to answer and how they’re asked. If the goal is to limit the volume of security metrics data, though, you have to use some discretion about which security metrics matter and which data you want to gather.

3. Lacking the skills and/or tools to effectively analyze data
Collecting the security metrics data is just the beginning. A massive database of log data doesn’t provide any value until or unless you have both the right tools and skills to filter through it and figure out what it means.

4. Failing to act on security metrics analysis
Assuming you’ve addressed the first three items you’ve collected the right amount of the right data and done a thorough analysis of it to gain some insight into your security posture and any issues you might have. If you don’t act on that analysis and do something to improve your security posture in some way, then what was the point? The entire security metrics process is a complete waste of time if you don’t do anything with the results.

5.  Checking a compliance box
This point lies somewhere between lacking the skills to effectively analyze the security metrics and lacking the will to address the issues that are uncovered. If your only purpose in collecting and analyzing security data is to create some sort of pretty vanity metrics that look good on a report and let you mark a box on a compliance checklist the security metrics aren’t helping you.

Most of these points actually just illustrates how security metrics can be abused—not so much how they might do more harm than good. The reason these five security metrics failures ultimately lead to potential harm is that they create a false sense of security and give executive management the impression that someone is monitoring the security posture and addressing any identified issues when that isn’t really happening at all.