This post is by Rich Mogull, analyst and CEO of Securosis, an independent security research firm.
I am inherently lazy. If I can come up with some new, automated way to solve a problem and save some time, I'll spend many hours more than it would take to knock it out manually on the off chance of some future time savings.
But I understand I'm a bit unusual that way (and it is sometimes to my detriment). Most of the time it's easier to plow through a well-honed process than devote the time to come up with code, tools, macros, or whatever to shave off a few minutes or seconds.
That's the challenge of automation — identifying those tasks worth the initial investment, without dragging yourself down black holes that suck up your time, or leave bad results.
I think that's why automation hasn't really taken off in security. At least, not in a big way. For much of our history we lacked the fundamental tools for automation. Most products lacked automation capabilities, and fewer products still even enabled external connections to support cross-product workflows.
Plus, more often than not, anytime a vendor promised automation, it generally failed to deliver. I mean how many of you are really cross-correlating SIEM alerts beyond the basics?
This all started changing a few years ago. Now I'm not saying security automation, and the tools for it, are completely new. I've worked with some organizations that have used automation for a while, but it's never anything that's taken off on a wide scale, and the barrier to entry is high. It's only been relatively recently that the three biggest drivers for change all lined up. Customer demand, the growth of cloud computing, and the transition to REST-based APIs.
APIs are the glue that allow products and services to talk with each other. In cloud computing, where everything is network accessible, they are the means of managing your infrastructure and applications. For a long time we mostly used SOAP APIs. SOAP stands for "Simple Object Access Protocol", but I prefer the acronym "Seriously Over-engineered Actually Painful". SOAP is incredibly flexible, but gets very complex, very quickly, and isn't overly accessible to anyone who isn't a dedicated programmer.
Most cloud services, and many products, now use (or are moving to) REST-based APIs. REST (Representational State Transfer) is designed for web services. It's simpler, just as flexible, and easier to work with. It's the kind of thing a good UNIX admin can build into a good script, without having to learn C++.
As an analyst, I see products every day that open up REST APIs. And when you get creative, they are incredibly powerful. Here's an example I programmed almost two years ago.
Let's take a common security (okay, compliance) task. Identifying any unmanaged servers in your data center. Nearly every security pro I talk with that is anywhere near a management position has to do it at some point. The usual process is:
- Scan the heck out of your network and identify all running systems.
- Scan it again, hoping you find all the blind spots.
- Pull a report from your CMDB or whatever else you use to track servers.
- Compare the results to find things that don't match.
I'm told it can take weeks or months. Here is the same thing with automation. In my example, your data center is Amazon Web Services, and your configuration management system is Chef. It's Ruby code, minus all the authentication bits. "Ridley" is the Ruby module to talk to a Chef server.
# Get a list of all instances in AWS, organized by their internal DNS name
instancelist = AWS.memoize { ec2.instances.map(&:private_dns_name) }
# Identify all the servers in Chef and map them by name, which in our
# environment is the same DNS name
nodes = ridley.node.all
nodenames = nodes.map { |node| node.name }
# Compare the two lists by name, and tell us if they are managed or not
instancelist.each do |thisinstance|
managed = nodenames.include?(thisinstance)
puts " #{thisinstance} #{managed} "
end
Run it, and you instantly know the real-time state of all your servers in Amazon. (BTW- there is a small security logic flaw in here I had to put in due to a bug in the Ridley SDK, bonus points if you catch it in the comments).
This runs in a few seconds for hundreds of servers. I, for one, welcome our new automated future. Where we don't just do things faster, we do them better.