Setting up and maintaining a security awareness program is almost identical to doing a research study. You start with a question or behavior, you create your study/manipulation, and you measure the results. Sometimes the results lead to the end of the experiment, many other times it leads to small or large manipulations and further experimentation. The same is true when setting up a security awareness program. You evaluate your culture and priorities to determine what behaviors you want to change. You create a program (training, messaging, etc.) that is designed to enact that change and then you measure to see if you got your desired results. While this may seem like a ‘no duh,’ to date I have not seen one program that doesn’t view this last step - metrics - as icing on the cake. Essentially it is viewed as nice but not necessary. This could not be farther from the truth. Not having metrics in place is like setting up an experiment and not collecting the results. The subjects/participants are going through your manipulation and you hope it worked, but will you ever know for sure?
Metrics must be used when setting up a security awareness architecture for three very important reasons:
- It allows you to succeed
- It enables sustained behavior
- It gets you money
Measuring Your Success
Security is complicated and humans are much more so. We are a very complex organism full of systems all interacting to create behavior. Some people are motivated by one thing, some another and no ONE thing is going to change the behavior of EVERYONE in the world. With this in mind it is no surprise that no security awareness effort is immediately effective. Some are epic failures and that happens. It doesn’t mean you should pack up shop and call security awareness a failure. Instead you need to re-think, adapt and find out what DOES work. This is why metrics are key. After you have created your experiment and implemented it on your users you NEED metrics to know if your program/content/event was effective. Did it change behavior? Even more, does that behavior persist over time?
Sustaining Secure Behavior
Let's say you are one of the lucky ones that just gets humans and you make a security awareness program that is stellar. Human issues are at an all time low. You have a content plan set up and your training is well received. Are you done? Have you overachieved yourself out of a job? Of course not. Threats are constantly evolving so even your current security awareness program will become outdated (i.e. any training focused on the Nigerian phishing scam is pretty out of date). Rather than waiting until something isn't effective and you’re back to solving ALL the problems, having the proper metrics allows you to see when behavior is significantly changing. You can identify problems before they are big, make slight modifications, and get the program back up to successful performance. Without this you just end up putting out big fires and spending a lot of money in the process.
Getting Money
What makes the world go round? Money. How else are you going to pay for the staff, the training, your learning management system, vendors, and every other thing required to keep security awareness at a successful status? While some - very few - may come from organizations in which the executive teams fully understand the importance of addressing human behavior - and therefore gives you a sufficient budget - we aren’t all put into that convenience. Several of the clients I work with have to consistently show that their budget was well spent. How do you show this? Results. You show that you reduced behavior X, or increased behavior Y, all leading to an overall financial and/or security benefit to the organization. Without metrics you are essentially going to the senior team and saying ‘The program worked. Trust me.’ By identifying the key behaviors that you and your management team see as an issue to security, addressing those and measuring them with metrics you are able to say “Hey guys you told me this was a problem. We did this, and we saw this drastic change. This has lead to an overall benefit of X to the organization…now keep giving me money." On the other hand you can also use metrics as a means by which to justify getting a higher budget. If you are measuring user behavior and show that something is presenting a major risk to the organization it’s much easier to justify budget if you can show a small experiment reduced a small group of users behavior.
So, as you can see, metrics are not the whipped cream topping to a security awareness program, they are the foundation. They provide baseline behavior, indicators of success, indicators of failure, and insight into future problems. Used correctly you can create a program that can adapt to any challenge that comes its way.