Podcast Transcript

You're listening to the RSA Conference podcast where the world talks security.

Hello, listeners. Welcome to this edition of our RSAC podcast series. Thank you for tuning in. I'm Tatyana Sanchez. And I'm Kacy Zurkus. And we are your RSAC podcast host. Kacy, what are we going to discuss today?

Well, Tatyana, no surprise that AI is front of mind for everyone. And governance around AI, simply put, is just different from traditional governance. Moreover, it needs to be treated differently.

And still many organizations continue to approach it as this sort of documentation or compliance exercise. And we know that that's just not working.

AI governance is more dynamic and it needs to move beyond just a checklist mindset. So that's why I'm excited. I know you are as well to be joined today by Varun Raj who's going to talk to us about how to reframe AI governance through a system design lens.

So he's gonna offer just a few tips, how organizations can treat governance as a runtime property, how separating AI control planes can create a more secure system, and how teams can shift from a model risk mindset to a systemic risk. So are you ready to dive in, Tatyana?

Yes. But before we get started, we wanna remind our listeners that here at RSAC, we host podcast twice a month, and we encourage you to subscribe, rate, and review us on your preferred podcast app so that you can be notified when new tracks are posted.

And now we would like to ask our guest to take a quick moment to introduce himself before we dive in. Varun?

Thank you, Tatiana, and thank you, Casey. It's great to be here. I am a cloud and engineering executive working on large scale platforms with cloud infrastructure, data systems, and machine learning come together to power real production environment. Much of my work focuses on how organizations move generative AI from experimentation into reliable systems that operate safely at scale.

What we are learning as an industry is that AI introduces a different category of operational risk than traditional software. With most software behavior is deterministic, you can review the code and test it. With AI systems, emerges from the interaction between models, data, and surrounding platform. So the real governance challenges is no longer simply is the model performing well.

The more important question is, is the system governing the model behaving safely while I run it in the production? That shift from evaluating models to governing AI systems is where many organizations are focusing right now.

Thank you, Varun. And you mentioned how, you know, we need to change our mindsets and look at AI and the behavior. And security practitioners are seeing that AI risk is behavioral, which is why governance has to be dynamic rather than static.

An organization shouldn't just treat governance as a policy artifact, but rather as a runtime property. So, Ravrun, can you explain what runtime property is, and why does it matter that the system is governed while it runs?

Absolutely. And and that's a great question because it gets it gets to the heart of why traditional governance approaches don't fully work for AI systems.

Historically, governance has often been treated as a static activity, policies, documentation, approval processes before the deployment. But AI systems are dynamic. They interact with new prompts, new datasets, new users, and sometimes even other AI services. That means risk isn't just introduced during development. It can emerge during operations.

So governance has to exist at run time.

It needs to be embedded into the system itself through things like policy enforcement, access control around data retrieval, guardrails around model interaction, and full traceability of what the system is doing.

One useful way to think about this is how cloud platform approach reliability. We don't assume reliability just because something passed testing. We build systems that continuously monitor and contain failure conditions. AI governance needs a similar shift.

What this leads to is what I often describe as a run time governance model for AI system, where policy enforcement, access control, and observability operate continuously as part of the system architecture.

Governance should not only approve AI systems before deployment, it should continuously govern how those systems behave while they operate. When governance becomes part of runtime architecture, organizations gain much greater visibility and control over AI behavior.

So I wanna talk about why it is important to have an AI control plane that is separate from the data plane and the application. So, Varun, can you talk to our listeners about not only how, but where they should be enforcing least privilege access?

Absolutely. And the answer to that question is architecture.

And that's why I say this is where the architecture becomes extremely, extremely important.

In many earlier option deployments, governance controls are scattered throughout individual applications. Prompts filters here, safety checks there, access rules embedded inside different services. The challenge is that governance becomes fragmented and inconsistent.

A much more robust approach is to separate the control plane from the data plane, which is a model cloud architects are already very familiar with. The data plane is where the model runs and where interface happens. The control plane is where governance decisions are made, identity verification, policy evaluation, authorization, and enforcement of system wide rules. When you centralize governance in a control plane, you can apply least privilege principle continuously and consistently across entire AI system. For example, the control plane can determine which dataset a model is allowed to retrieve, which tools an AI agent can invoke, which prompts are permitted, and how outputs are allowed to propagate downstream. This architecture ensures that the model operate within clearly defined boundaries.

In other words, the model is no longer treated as a autonomous black box. It becomes a governed component within the controlled system architecture.

And, Varun, earlier you talked about, you know, shifting a mindset from a model risk thinking to a systemic risk thinking. So can you explain to our listeners what's the difference between the two, and how can organizations mitigate the risk of a good or even a great model being embedded into a not so mature system?

What could possibly go wrong there?

Absolutely. I would say this is one of the most important mindset shifts organization need to make as AI becomes embedded into real operational systems.

Most governance frameworks today focus on model risk, things like bias, fairness, the model evaluation. Those are important areas, but they only address part of the bigger picture. In production environments, many risks actually emerge from how the model interacts with the surrounding system. A model might perform extremely well in testing, but if that model is connected to sensitive enterprise data, automated workflows, external APIs, or agentic systems, even smaller can propagate very quickly. What organizations are really dealing with in those situation is systematic risk. It's no more an operational risk.

A helpful analogy is aviation.

Aircraft safety doesn't depend on engine alone. It depends on the entire system, the navigation, the instrumentation, the monitoring, and the operational controls. AI system requires the same kind of thinking. If well trained model placed inside an immature system can still produce serious failures, that's why governance has to move beyond evaluating models and towards engineering resilient AI systems where safety observability and policy enforcement are built directly into the architecture. And that's ultimately what allows organizations to adopt generative AI responsibility at scale.

What about regulators and CISOs? You know, they want evidence, not only assurances. So when we talk about, you know, shifting this mindset for the business, how do we move toward a verifiable and auditable AI model where security, legal, and engineering align around what you talk about these architectural controls?

Absolutely. And the answer to that question is runtime governance model for AI systems. This architecture will allow the organizations to handle the systematic risk, which are becoming more and more prominent beyond the model risk. Architecting the systems that enforces governance at run time will help CSOs to ensure that they don't have the risk that regulators are concerned about, allowing organizations to run and operate AI systems in secure and forward looking manner.

Varun, thank you so much for being here today. Listeners, thank you for tuning in. Please keep the conversation going on our RSAC membership platform by visiting 1rsac.com/membership, and be sure to check 1rsac.com for new content posted year round. Until next time.