Analyst Experience: SOC Analysts Finally Escape The Shackles Of Bad UX

Posted on in Podcasts

Security analysts must fight both attackers and the products and services they rely on to drive their detection and response capabilities because of a lack of usability of security tools and processes. This podcast explains an emerging cybersecurity technology trend: analyst experience (AX). Join us to understand how AX can enable your team to find more effective security tools, build better workflows, and help security analysts make faster, more accurate decisions.

Podcast Transcript

You're listening to the RSA Conference podcast, where the world talks security.

Kacy Zurkus:
Hello listeners, and welcome to this edition of our RSAC 365 podcast series. Thanks so much for tuning in. I'm your host, Kacy Zurkus, content strategist with the RSA Conference team. Today, we have the pleasure of having two Forrester analysts join us to talk about the emerging analyst experience trend.
Before we get started, I want to remind our listeners that here at RSAC, we host podcasts twice a month, and I encourage you to subscribe, rate, and review us on your preferred podcast app so that you can be notified when new tracks are posted.
Now, I'd like to ask Allie and Jeff to take a moment to introduce themselves before we dive into today's topic. Allie, let's start with you.

Allie Mellen:
Thank you so much, Kacy. It's so great to be here with you. My name is Allie Mellen. I'm a senior analyst on the security and risk team at Forrester, and I cover security operations, people, process, technology in the SOCs. That includes EDR, XDR, SIEM, SOAR, security analytics as well as ransomware and minor attacks. I have been in security for a while now and I got my start on the hacker side of things before running my own consultancy, working for a security vendor for a little while on the security team, and then ending up at Forrester. It's great to be here with you.

Jeff Pollard:
Jeff Pollard here, vice president and principal analyst on a security and risk team here at Forrester. I cover lots of different topics, some of them directly related to what Allie covers. We often collaborate pretty extensively. My topics include security services, things like managed detection and response, so MDR. Topics related to the chief information security officer, things like budgets, business cases, metrics, dashboards, et cetera. A whole lot of things around detection and response in general as mentioned. And then finally, I also cover securing emerging technology as well as emerging technologies that are coming to cybersecurity as well.
Prior to joining Forrester, spent a little over a decade or so on the vendor side of the house, various MSSPs and things like that, and then joined Forrester back in kind of the mid-2000s and I've been here since.

Kacy Zurkus:
Awesome. Well, we are super-excited to have you both with us today. Allie, I wanted to start with maybe asking you to lay the groundwork for this conversation so that we can make sure everyone understands exactly what we're talking about when we say analyst experience. Can you maybe talk about this emerging trend and explain what it is, how it's different from the user experience, and maybe even why CSOs and security professionals should care about it?

Allie Mellen:
Great question. One of the CSOs that I used to work with in a past life would always tell me that security analysts have the worst job in the world. I'm pretty sure he was only half-joking and honestly, he was kind of right. They do, in many ways, have a very difficult and very challenging job. In fact, according to a survey Forrester does every year of thousands of security decision-makers, we found that the vast majority at 42% cited investigation as what takes the most time in their organization, instant alert and response process. The tools are not enabling one of the most important parts of incident response, understanding the full scope of an attack and enabling response to that.
But we know and we've found that it doesn't actually have to be that way and there's an opportunity happening now for this to change. We coined the term, security analyst experience earlier this year in part due to Jeff's and my research on EDR, XDR, SIEM, MDR and MFSCs.
So, what is analyst experience? It is security analyst perception of their interactions with a particular security product, service or process across various work streams. It's about identifying and enhancing the analyst workflow, whether that's through technology or process or other means. That's why it's important that security leaders actually leverage analyst experience as well. They can use this as a tool for retention. We know that it's quite difficult to retain talent in the security organization, or in some cases, even to get it in the first place. And analyst experience can be leveraged to help entice new talent. If a CSO is paying attention to the experience that the analysts in their SOC have, that the experience of the people on their security team have, and is prioritizing that, it makes it clear that they actually care about the role succeeding and moving forward.
And that comes not just as you mentioned from the technology and from the user experience with the technology, but also how we think about the actual processes that are built in the SOCs. Are they built to enable the security analyst or are they more cumbersome than helpful? It also comes to things like career paths. What are the security analysts perceptions of the way that the business is enabling their future success? All of these pieces come into play with analyst experience and we truly think that it is the next generation of detection and response, particularly in the SOC, is looking for ways that security vendors and CSOs are able to make the security analyst role better through analyst experience.

Kacy Zurkus:
That was a lot. Jeff, I'm hoping you can maybe help us break that down a little bit. No, it was a lot of detail and I think that was a great detailed answer, but I want to dig in a little bit to what you said, and Jeff, I'm hoping that maybe you can expand on Allie talked about the challenges with the tools. Can you talk to us maybe a little bit about how this lack of usability of security tools and processes creates issues? What do we mean when we say that security analysts have to fight the products and services they rely on to drive their detection and response capabilities?

Jeff Pollard:
I think first and foremost, that that's probably the least bold call that Allie and I have ever made together is that security practitioners have to fight products and services. Security usability kind of doesn't exist, right? The security industry is very late to of understanding user experience, even employee experience in a lot of cases. And so you wind up with the situation where a lot of the tools that security practitioners are forced to use are sometimes jokingly called expert systems. In fact, one of my stints for the security vendor, that's what we called it as kind of a joke, where this is an expert system. What that meant was you have to be an expert to use it, but you don't just have to be an expert on the domain of the technology, the sort of area that it sits in. You also have to be an expert on the technology itself. And so that doesn't scale because you wind up having to train two different skillsets for every single role at a minimum and maybe more than that.
To give you some examples of this. Allie and I conducted a global security operations sentiment survey where we reached out to security practitioners and folks on the user side, folks on the vendor side, and talked to them about the tools and technologies they use. 79% of the respondents agreed or strongly agreed that context switching between interfaces diminished their ability to perform their job optimally. So, the fact that they had to have 19 browser windows open, pivoting between tools, that they had a thick client and a thin client and then a browser and then something else, really diminished their ability to understand what they were doing and how they were doing it. They would lose sort of the decision-making because they had to constantly shuffle between tools. 47% of the respondents agreed or strongly agreed that the security tech they used was built by people that understand the task they routinely perform. So, under 50% felt like the product managers, the developers, the security vendors out there were building technologies for them that enhanced what they did on a day-in, day-out basis.
To me, those responses in the survey that we conducted along with the research really get to the heart of what we mean by fighting the products and services they use. These things, they force analysts to context switch between multiple interfaces. They are not built by people that understand the mission or the role of the security operations center analyst. And as a result, what winds up happening is that the tools are not facilitating the work streams. They're not facilitating the processes. We're trying to find adversaries, but to even get there, there's so much friction before we can do it, that's brought about by the fact that you have to master an area of expertise, then you have to become an expert on technology or multiple technologies to even get to that point. That's kind of the core essence of that.

Kacy Zurkus:
I can't imagine having to navigate 19 browsers. It's hard enough for me to write my weekly news blog and copy the headline onto a Word doc with two screens. But Allie, I'm hoping that maybe you can share with our listeners what you see as some of the tangible benefits of analyst experience.

Allie Mellen:
Sure. Yeah, I'm happy to. There are a couple of things that are important here. In all of the research that I do on the SOC and that Jeff and I do on the SOC, I think about it through the lens of quality. Efficiency and optimization ultimately mean nothing in the SOC if quality and resilience are not maintained. And so I try to look at everything through that lens, including analyst experience.
Analyst experience is actually something that has the potential to be a great enabler of quality, quality outcomes, quality processes, quality workflows, and ultimately the goal of analyst experience is to provide faster and more complete incident response, which is a huge benefit to security teams who are already strapped for time, better analyst experience and employee wellness, which is another key thing that we often hear cited as one of the top challenges.
I did some research earlier this year into the security analyst role and what that needs to look like and what that should look like. In the interviews that I did with security analysts, overwhelmingly, they said, "This is not a 9:00 to 5:00 job. This is a job that you need to work outside your own hours and if you want to get to your next level in your career, you need to, in addition to working extra hours, work on your own side projects too and learn about the other areas of security that you might want to move into, like threat hunting, like penetration testing, et cetera." And so making sure that you have employee wellness prioritized through analyst experience is really key here.
And then the last one is better reaction time for response. If you have quality workflows and you have quality data upfront that the analyst needs to have, that they don't have to go manually collecting, then you're able to give them better reaction times for response and then also more complete response. It's all about quality and improving quality for the SOC and for the security team.

Kacy Zurkus:
I've heard you mentioned a few times, both of you, this building better workflows. Jeff, I'm hoping you can maybe explain how will finding more effective security tools help teams to build better workflows and enable security analysts to make more accurate decisions?

Jeff Pollard:
To answer that, I want to start with the problems first a little bit because one of the things that Allie and I have noticed as we've been out there talking about analyst experience over the last six months or so, as we've had engagements about it, as we've had tons of client conversations, we've encountered a set of really, really common obstacles, that may be phrased differently, but are pretty concentrated in terms of the big issues that security analysts encounter. And so we've heard this as UX pros have started to think in terms of AX, we've heard this as service delivery personnel are starting to deal with it. Some of the examples of the problems they deal with in their current sort of work streams as they want to make those better and more accurate decisions are things that are even surprising a little bit.
One of my favorites is as we've talked to practitioners about this, one of the things that they've shared over and over again is that the next steps of what to do when they deal with a new threat are not really clear from a technology perspective or a process perspective. What that means is that if they've seen it before, they know what to do. If they've seen that category or that specific threat, they totally understand what happens next. But when they see something new, even if it looks like something old, it's kind of a reinvention every time. They have to make all new decisions, they have to perform all new investigations. That's not necessarily bad, but it is bad when it's not scalable and when it's happening for every single thing that's new.
Some of the other examples of some of the problems that sort of inhibit that decision-making or act as obstacles to it are things like a lack of automation. They're not able to gather artifacts they might need in the investigative process automatically because they don't have the tools, technologies or integration to perform that work.
Or one of my other favorites is that we routinely hear that investigations sort of succeed or fail based on the search proficiency of the analyst in the technology that they're using. Their sort of expertise in a query language is what dictates the thoroughness of their investigation or whether they found all of the things that they needed to find.
When we think about analyst experience, the core of it is really solving a lot of those problems so that we are not forcing analysts when they see a new threat, that they have to work it as if it's an entirely new novel and never before seen issue. The investigative process that AX facilitates should account for that. Yes, you're going to learn new things. Maybe this is that ultra-cool, sort of really rad attack that you're going to write a blog about, but it's probably not. There's a saying in medicine, "When you hear hooves, think horses, not zebras." The same is true in the SOC. When you see bad, think commodity, not nation state or whatever it might be. And that's a terrible version of that saying, we have to figure out how to make that better, but that's what we're really trying to help with.
And so when you think about those faster, more accurate decisions, if what's holding me up from making that decision is how great I am at searching in SIEM platform A or B or whatever it is, well, again, that's a technical proficiency I have to obtain, that I need when it's most urgent, when an event is happening.
So, what we're trying to do with analyst experience is really remove the need for that kind of capability, remove the need to cultivate all of those skills, especially around technology, and think about it more holistically so that way you can be confident in the decisions you're making and you can make them as fast as possible or as fast as reasonable.

Kacy Zurkus:
This has all been so enlightening. I so appreciate both of you bringing this to the RSA Conference community. I would love it if each of you could share your perspective on this final question that I have for you. Jeff, let's start with you. What is the future of analyst experience?

Jeff Pollard:
I think the real future of analyst experience is, one, uplifting the products and services that we rely on to provide security to our organizations, to our customers, to our clients. Whatever role you happen to be in, it's about making those better. The reason why we want to make those better is not just because of better security, that's absolutely true, but we want to make those better so that way we can make the employee experience, the culture, the aspect of working in security as colleagues better for everyone. So that way, this industry is more inclusive, more welcoming, less toxic, less gatekeeping. We want to make products and services better so that way we can make everything better for the people that do this every day, so that way they can make things better for everyone that they work with. That's really what I think the future of analyst experience is, is transforming the technology that we work with so that way it has a real human impact on the people that use it, so that way they can be better at everything they're doing.

Allie Mellen:
I totally agree with everything that Jeff just said. I think that that is the most critical piece here is giving security analysts and security teams more time to focus on things that are more important, things that are more strategic. A vehicle to do that, one of many, is analyst experience and is making investigation response easier for security teams.
For the short-term, I think that the biggest change that we are going to see and we should see is on investigation and is on ease of investigation and abilities around that.
In the longer-term, it's more focused on bringing all of these pieces together and enabling faster response and then leading to something like faster mitigation. That is a much longer-term initiative and is going to require a lot, seem a lot more data-driven and we've historically been able to do in internal SOCs, but it's something that is top of mind as we do the research that we're working on. And as we think about where analyst experience has to go in order for it to be a benefit to every kind of SOC as opposed to just the biggest enterprises in the world.

Kacy Zurkus:
Do you also see that this could potentially have an impact on the future workforce in mitigating the reality of burnout?

Allie Mellen:
Absolutely. I think that that is one key piece here is bringing more people into this field, making it easier for people to learn how to excel in this field, and giving them the time and the space to explore different areas. One of the reasons that we have so much burnout in this industry is because just coming back to what we started with, the security analyst role is really hard and it is nonstop and there is a ton of pressure because you are working on something that has huge implications for the business and for the brand. And so if we can simplify that a little bit, provide more value to the analyst upfront so that they can be in this role, but also learning about security, learning about the organization, having the space to think more strategically and to think about their future, we will definitely see better outcomes when it comes to keeping people in this industry and bringing more people into it as well.

Jeff Pollard:
This is one of the areas where we're super-lucky from a Forrester perspective, that Forrester has a pretty significant body of research on employee experience, for example. One of the things that we found in our future of work research and in the employee experience research that our colleagues do is that improving the tools and technologies that your employees use makes life better for them at work. It leads to higher retention rates, it leads to more advancement opportunities for them, it leads to more employee referrals. While some of those studies are outside of security, some of that research is well outside of security, I can certainly say that if you are able to improve some sort of archaic kind of 911 system or whatever it might be for the folks that have to interact with that and you can increase retention, increase satisfaction, then if we can make a security analytics interface or an MDR, EDR interface better, we're going to do the same too.
So, that's one of the areas that Allie and I were able to pull from is the fact that Forrester has spent a ton of time on customer experience, employee experience, and areas like that. And so we're able to take those lessons and apply those to security and kind of break that knowledge silo and bring something that may not have been applied to this industry yet, but use it as something that will improve it.

Kacy Zurkus:
Wonderful. I love it. It sounds very optimistic. Jeff and Allie, thank you so much for joining us. Listeners, thank you for tuning in.
To find products and solutions related to the analyst experience, we invite you to visit Here you'll find an entire ecosystem of cybersecurity vendors and service providers who can assist with your specific needs. Please keep the conversation going on your social channels using the #RSAC, and be sure to visit for new content posted year-round.

Allie Mellen

Analyst, Security and Risk, Forrester Research

Jeff Pollard

Vice President & Principal Analyst, Forrester

Professional Development & Personnel Management

behavioral analytics big data analytics consumerization practitioner perspectives professional development & workforce security analytics security operations

Share With Your Community