Podcast Transcript

Introduction:
You're listening to the RSA Conference podcast, where the world talks security.


Kacy Zurkus:
Hello listeners, and welcome to this installment of our RSAC 365 podcast series with today's topic, Shift Left, Scanning and Security Compliance from Day Zero. This month's theme is cloud security in cloud sec ops and today we're joined by Joe McCrea and Mariam Triki of SAP. Our guests, will be talking about the security challenges inherent in migration to public cloud and how to detect threats in a cloud landscape. So before I turn it over to my guests to have them introduce themselves, I want to let you know that here at our RSAC we host podcasts twice a month, and I encourage you to subscribe rate and review us on your preferred podcast app, so you can be notified of a new track we've posted. And now I'd like to ask Mariam and Joe to take a moment to introduce themselves before we dive into today's topic. Mariam, let's start with you.


Mariam Triki:
Thank you so much, Kacy, It's lovely to be here today with you. I guess I'll talk a little bit about how I ended up being a DevSecOps in SAP. I started my studies doing just math and physics, but then IT peaked my interest and I switched to a computer science engineering degree and fell in love with security. Unfortunately, at the time in university, not enough people wanted it, so they ended up shutting the division that was started in security, but I managed to find an internship in SAP security labs in France. And that's how I launched my career in security, I did the six months there and then got hired as a DevOps in Belfast. But I kind of missed security again and creeped my way into the DevSecOps team in the same place, in the same company and I'm still there.


Joe McCrea:
So my story started the same with university and did a degree in computer science and then tried to move into security, just by securing a few internships and stuff like that. Eventually then I started full time, I was in a purely software engineering role, but naturally there was a hole where security was. And because I was interested I just filled in where I could and then just moved over to SAP then, there was a position that was open that felt like a dream move, so I was really happy to go for it. I'm really passionate about open source and security and fortunate enough to join a team that both uses and contributes to open source and is based in security, so that's how I ended up at SAP.


Kacy Zurkus:
So it's great to have you both with us today. I'm so excited that you found your way into security and found your way to RSA Conference so that we can have this conversation with you. I know that you both bring experience as members of the DevSecOps and multi-cloud teams. So I'd love it if you could start us off by maybe highlighting the importance of continuous scanning from misconfigured resources on the cloud, Mariam, why don't you jump in here?


Mariam Triki:
Yeah, of course. I'm happy to do that. I'd say it's really important not just because it's my job, but the cloud in general is new and migrations are happening now more and more for large companies, that hold a lot of sensitive data and a lot of critical day to day business, but it is also not something a lot of people have experience with. And even experienced people, experienced cloud engineers can misconfigure resources and that leads to huge gaps. You move your app from being deployed on prem in an environment that you are really familiar with to this new shared responsibility model, where the lines can get a little bit hazy.


Mariam Triki:
Where does your responsibility start? Where does your cloud provider responsibility start? How much of that documentation about the product you're going to use? Did you miss the small print of telling you that even though we promise a very secure product on the cloud, you have to make sure to set it up in this way, not to use it for these purposes, and that's when the confusion happens and that's where the errors happens.


Mariam Triki:
Misconfigurations are usually not something done... 99% it's not something done on purpose and it's just exploring a new world and you are testing it, but you don't know the full limits of your product, and you don't know the full extent of this new setup and you can leave a big threat surface if you do not do that. And especially when you have already things dealing with your app for a certain amount of years, that now has to suddenly move that app to the cloud.


Mariam Triki:
You can't expect them to be expert in one day, they know their product really well, it's just getting to explore the cloud and you have to have set of eyes looking at these configurations to make sure you didn't miss anything. People who are more familiar with the cloud offerings and that sort of configurations you can miss and the fine print you can overlook.


Joe McCrea:
I don't want to repeat what Mariam just said but it's like those.... pretty much sums it up very well, there's such a fine line between the responsibilities in cloud that becomes very easy for somebody to make a mistake and for that to become misconfiguration, it could be abuse by somebody. But I think at the end of the day security is down to a numbers game. In an ideal world, you would reduce the chance of a security incident to 0%, but people are human, they're bound to make mistakes.


Joe McCrea:
So I think why it's so important maybe to start scanning everything because at least at that point you catch something as it's being created, rather than waiting until something unfortunate happens before you actually take a look at your actual posture and realize that things could have been done better.


Kacy Zurkus:
So Joe then where do you start? Maybe you could walk our listeners through how to lay the groundwork, for those that are just starting out on their journey toward addressing these imminent threats. How do they go about translating security compliance policies to code, in order to achieve what you might call this compliance as code, and are there any open source tools that can help toward that end?


Joe McCrea:
I think that's a great question. When we first looked at it, it was kind of more of a, "Let's get some visibility," which obviously you don't have at the very start. And it's near enough impossible to defend from something that you can't see or identify. As for tooling wise, I think that there's number of tools out there that have different benefits. And the approach we took was to use Chef InSpec, it's an open source kind of compliance checking tool. But some of the things that we really found benefits of it were that it's extremely flexible. It allowed us to build our own tool that kind of utilized Chef InSpec to do what we want it to do.


Joe McCrea:
So it allowed us to actually go out and use that tool to verify the security compliance of accounts. And I know there's plenty of tools out there to use, but I think that's after we had looked around a bit it was the one that we settled on. So I think when you're starting out, it's important to start small and just try and figure out how to scan the security compliance with a single account. And then from there we took that approach where we were like, okay, we'll try and verify one account and then we just need to figure out then how to architect the solution that allowed us do that on the scale that we have to deal with on a daily basis. So scanning was one issue that we discovered where it's like we built something, at least that was the scanner offering part that we're done with. But one of the other challenges we had then was SAP has a 100,000 employees and there's thousands of cloud accounts in it with millions of resources.


Joe McCrea:
It turns out it's quite a challenge to figure out which of those 100,000 employees you actually need to send a report to. It's like trying to find a needle in the hay stack based on who it is, we have a few tools that were built by other teams that we were able utilize that matched up the owners, or people who would actually care about the security of that account. So we were in quite a fortunate situation where we could use our tool as well as utilizing the data from other tools within SAP, to actually let us build the kind of pipeline where it goes straight through from scanning your account right through to the person to notify. Now I would say that's not a problem that everyone's going to have, but I think that was one of the challenges that we had definitely experienced.


Kacy Zurkus:
That's great, Joe, thank you so much. Lots of good insight there. My next question was actually going to be about some of the challenges that you faced. So Mariam, maybe you could give us a little bit of a combined answer on laying the groundwork, but also how that groundwork posed some challenges that you faced given that SAP is such a large global enterprise.


Mariam Triki:
As Joe talked about the challenges of once you find those security threats or those misconfiguration, how do you get about to reaching the responsible people, that would also be something you discover when you are laying your first steps is you need processes. Security is not just figuring out the right tool, it's more processes, it's more of educating your employees and the people who are doing these deployments and using these products. So that would be the first steps, even before choosing an open source tool or deciding you just can or not, you have to put the processes and what happens when you are going to find this misconfiguration, how do you fix it and implement the fixes?


Mariam Triki:
It also goes to once you start developing these tools, how are you going to keep them up? How are you going to keep up with the new misconfigurations that comes up, that's also one of the two challenges we face is you can have a resource or you can build your application to be secure on the cloud, but the cloud is not stable. You don't roll out the updates anymore, you're using a product, someone else's rolling updates and continuously working on improving it, but that could leave you behind and leave you with outdated versions and that could lead to also some security threat. So I'd say keeping up with security and keeping up with all the changes in the cloud world is also a challenge.


Speaker 1:
And nothing in security is without its challenges. And I would imagine that this whole concept of shift left presents different challenges. So can you talk a bit about what it actually means to shift left, and how this enables you to detect security compliance violations in product infrastructure? And what has allowed you to provide security compliance status throughout the product life cycle from development all the way through to production. Mariam, why don't we start with you this time?


Mariam Triki:
Well, shift left was basically trying to tell or to say security should be a first thought. Anyone who has worked on the development of a new product in even a small company knows you start by creating a proof of concept, so you can get to your shareholders, tell them that it actually can work and this is potentially a revenue generating product, so can we get some hand on it? And then from that, the wheels start rolling and it gets into this rhythm where you kind of let go of security or you don't have time. Because also you start with really a small group of people and you don't have the experience, nor the time to put security as your main focus and make sure your product is secure and scalable. But then you end up at a certain point with a product that you rolled out and you have clients on it, using it.


Mariam Triki:
And then you try to look backward and do a threat study, or try to figure out what could potentially be a security threat to you, what can be now enhanced and that's really, really hard. To implement security backward it costs even more resources, sometimes it's not even feasible anymore at that stage. And by saying shift left, we say even at a POC level, even when you're creating proof of concept for an idea that is just starting, you should be thinking about security. The first deployment, if you're creating a scalable product, even if you creating one pod of a Kubernetes cluster, or you're creating one virtual machine to test, security should be a concern for you at that point. You should be able to spot what you're not doing right, or what is not being up to standard at that point.


Mariam Triki:
Otherwise, if you complete the life cycle of this product, it keeps growing and you look back and try to make it better at that point, that's either going to require building up from scratch, which can be done in 99% of the time, or it means you just have to live with it and keep patching the best you can. And that's what really we're talking about by bringing security further ahead into the life cycle of a product, by what we've done with our product that would bring security to the beginning of the lifecycle of a product is we made our scanning tools mainly from containers, they can easily be integrated in a CI/CD pipeline. So once you start building your application and pushing code to your ethers, enhancing your application and working on new features, one step from your pipeline could be this tool within SAP.


Mariam Triki:
So if you add it, it's going to scan whatever you deployed for this test and let as you go from the first try of deploying your application, what could be going wrong, what could be looked at and what could be enhanced, what definitely should be fixed and changed. In that way you are working on having a secure deployment and a secure cloud environment from day one, from the first deployment and the first ever resources created.


Joe McCrea:
Even as Mariam touched on that, the idea that you could actually integrate this in the CI/CD it really, it kind of shows what we mean by shift left. I think even the tool that we built was so flexible that, as well as implemented in CI/CD, we were also able as a component of your CI or CD, we were actually able to also integrate this into scanning the SAP organization. So we can reach out to say AWS or Azure, GCP, or even any cloud from the day you create a resource in that account, that account then immediately starts to get scanned by scale, by a separate set of tools, all of them, the one that you would implement in your CI.


Joe McCrea:
So it means from the first day, we can kind of reach out to you and say, "Hey, this POC that you're developing-" As Mariam it's a true story that many POCs end up making it to production without much of a input in security or even for this, it's always at the end of the stack. So I think it allows us to come along on the first days of the POC and say, "By the way, you might want to rethink this certain element of it because it's going to cause you headache in the future."


Kacy Zurkus:
Focusing on this idea of what folks should be thinking about. I'd love for you to maybe just discuss a little bit about what we should be thinking about when developing scanning goals. If the goal is to keep up with emerging threats and the influx of new cloud offerings, it's sort of like, how do you frame your expectations for preparing for the unknown?


Mariam Triki:
Personally, I would think developing your own scan tool, shouldn't be for everyone, reinventing the wheel isn't always the way to go about it. As we talked about open source for scanning, there is a lot of great tools. There are ones you have to pay for and others that are open source, like we were using Chef, there is Puppet, there is Ansible. The main thing to think about when you decide to scan is what is the easiest way to do it? And that's going to depend on how much resources you have, how many cloud hyperscalers are you going to use, are using offerings from different hyperscalers, or is it one main, and that would change drastically if your going to scan or not. Is it a really small deployment? Are you using the graphical interface with this? Or do you have another tool doing the deployment such as Terraform and you're writing it as code.


Mariam Triki:
So that will also play a huge part in deciding how are you going to scan your infrastructure. But the main idea is you have to scan it, you cannot assume that you got it right. You cannot assume that just because when we designed this deployment voice secure, it means it went on a clear cloud and is secure. And in small environments I would always say these rules, if you did end up using any of the open source tools, Chef or other. If you know what kind of offerings you're using from your hyperscalers, you can always focus on just implementing the rules for those set of products you're using. If as in the case for SAP we have huge teams, basically, we don't know anyone who's using these offerings from the cloud. And everyone within SAP has the liberty to choose whatever suits their product to deploy it, so we had to scan for everything.


Mariam Triki:
We had to think about anything that could be a threat and scan for it, and look for any potential threat that could make the job a bit more way harder, because looking for threats on four different hyperscalers, everyone has their own properties, everyone has their own offerings, everyone gives you different metadata about your resources, so that would be challenging. But as long as you keep in mind, what are your main sources, pain points and tackle those that will eventually lead to you also discovering other points you can look after and you can anticipate bit by bit.


Mariam Triki:
The fact that the cloud will always change, the fact that the offerings will always change, that you cannot control. And it's actually something really, really good. If you are large like SAP, you need a team of DevSecOps so you can upload the technology and always looking out for other pain points and future potential threats to close.


Joe McCrea:
Yeah, absolutely. Like, I think the more seamless you can integrate security into your existence for the tools and processes, the better off you'll end up. We were in a pretty fortunate situation where we developed something that was a puzzle piece to finish off the puzzle, and I think that's how you have to deal with this because cloud offerings are constantly changing, there's constantly new threats. I think the more seamless you can integrate security into your actual setup, the easier it'll make it for you, so you don't have to continually rethink it. It's the kind of thing that ideally you want something that's constantly on the search for you, rather than you having to constantly update it and constantly be looking what's coming on.


Joe McCrea:
But again, there's also an element of that. I think as Mariam said once you apply it once and it's secure, that doesn't necessarily mean it'll always be secure. So it's something that you constantly have to revisit and just have in the back of your head that, you have to keep your eye on the news, it's impossible to tell what's coming up next.


Kacy Zurkus:
One thing I wanted to touch on is the tool that you built and I know it's a bit unique. So if you could maybe Joe maybe let's start with you and talk us through your approach of integrating the DevSecOps engineers in the effort to build this tool. Why did you decide to go that route?


Joe McCrea:
It was a lot of certain markets and personal experience within the team, that kind of proved to us that there was no product out there that exactly fit all of our requirements. So ideally we wanted something that would work at scale, that would scan at scale, something that would integrate with our existing tools and processes, and then something that allowed us to take that data and actually add on our own metadata around it, such as who are the proper security contacts for this is, is it a production system, is it a development system? So we kind of didn't find a product that really allowed us all three of those and as I say, get that final piece to the puzzle.


Joe McCrea:
So I think DevSecOps engineers, we were all kind of coming from a development and security background, so it kind of meant that we had the knowledge of what needed to be scanned, as well as the ability to actually create the tool to scan it. So sort of catering for all those allowed us to build security at its core. And it allowed us to constantly have people in the room that could input on the security background, and that could input into the best ways to develop this to kind of future proof it. I think Mariam has something to add there.


Mariam Triki:
I agree. Totally agree with what you said. If I'd add one thing is the fact that it's not in every case that you need to go this route, that we decided to build our own tool. It's just the setup as I said to you was very particular. In most companies, you'd only be using one hyperscaler, everyone will be on the same tool for CI/CD, say everyone is going to go use Jenkins, or any kind of other open source tool or not even open source. You have policy, everyone has to have their code as infrastructure as a code, then we are going to scan that code, which would make life really easy, but that's not the case for SAP. SAP, it tries to give teams as much freedom as possible without compromising our main concern and one of SAP's main selling points within security. Every team gets to choose whatever tools they have.


Mariam Triki:
Every team gets to choose whatever hyperscaler they go to. And we have so many of them and they're deploying so many resources. So looking at the market there isn't something really that would scale up to that point, to take it into consideration all this wide spread and to be able to keep up with the size. For smaller other users I'd try to take a look at other tools first, see if they actually fit your purpose, but if it doesn't then building your own tool is always fun. And we had really a lot of fun doing this new approach where you get to build something for particularly niche configuration and the cloud, add security threats and do it for the main hyperscalers all throughout.


Kacy Zurkus:
I so appreciate that you are sharing your story of breaking down silos and really integrating efforts across teams in order to overcome your business specific security challenges related to cloud migration. And Mariam that's something that I've heard from you quite a bit today in this conversation is that like, know what we did isn't necessarily going to work for everyone. And that it is very focused on the issues that you were trying to solve in your business, that there is no silver bullet, that this is the way that it has to be done and you'll never have a security issue again. Yours is a journey that many organizations are either embarking on or just about to embark on or really continuing on, so your story is so incredibly helpful. Before we wrap up I'd love to hear from either of you, if you have any parting words for our listeners?


Mariam Triki:
Anyone who preaches there's a security tool that will solve all your problems is scamming you, there is no such thing as 100% secure. There is more like an interesting and a good security manner, a good security processes. It's always worth it to invest in security, it doesn't matter how good your customer experience is, it doesn't matter how scalable your tool is, if it's not secure, it is going to bound to fail. Also keep in mind to integrate this in early stages, backward security is not even close to be as efficient as security that was integrated from the get go. Every tool has its own specificity. Every company has its own environment. And IT world is one of the few places where there is always something that is going to fit you. And if there isn't, well, that's what code is for, you build it.


Joe McCrea:
As Mariam said there's no perfect tool and I think everyone's in a completely different situation a lot of the time. I think Mariam sums that up perfectly there.


Kacy Zurkus:
Wonderful. Joe, Mariam thank you so much for being here with us today. Listeners, thank you so much for tuning in. To find products and solutions related to cloud security, we invite you to visit rsaconference.com/marketplace. Here, you'll find an entire ecosystem of cybersecurity vendors and service providers, who can assist you with your specific needs. Please keep the conversation going on your social channels using the hashtag RSAC. And be sure to visit RSA conference.com for new content posted year round. Thank you all so much.