• Tool Misuse and Exploitation: Agents may use legitimate tools but with destructive parameters, similar to the vulnerabilities seen in the Amazon Q exploit.

2. Supply Chain and Code Execution

This cluster involves vulnerabilities within the agent's ecosystem, such as:

• Poisoned Model Context Protocol (MCP) servers or plugins.
• Compromised agent skills at runtime.
• Memory and context poisoning, where an agent's "history" is manipulated to influence future actions.

3. Autonomy, Trust, and Rogue Behavior

• Spoofed messagescan misdirect an agent’s logic.
• Agents may exceed their intended boundaries or even manipulate humans to bypass safety oversight.
• The ultimate risk is the rogue agent, which acts entirely outside of its intended programming.

Each agent acts as a non-human identity. Without strong credential controls and behavioral monitoring, agents can overstep privileges or be impersonated.

Implementing Agents

In an RSAC 2025 podcast, Ken Huang, CEO of DistributedApp.AI and Chris Hughes President and Co-Founder of Aquia discussed their book,Securing AI Agents – Foundations, Frameworks, and Real-World Deployment. Huang noted that while adoption is high, 19% of organizations still feel uncomfortable deploying AI agents. To build the necessary confidence, Huang recommends three foundational steps:

• Define a Job Description:Clearly outline the agent's purpose, the specific data and tools it can access, its level of autonomy, and, crucially, its boundaries.
• Establish a Governance Committee:An AI review board is non-negotiable for establishing ownership and ensuring no security gaps fall through the cracks.
• Design with a Human-in-the-Loop:Build architectures where humans remain central to the agent's lifecycle.

But how can an organization practically build an architecture to safely secure and deploy these entities? 

Dorothy Li, CVP of Microsoft Security Copilot and Ecosystem and Marketplace at Microsoft, and Ryan Munsh, Principal Product Manager at Microsoft, identified five keys to unlocking the potential of AI agents based on direct customer concerns:

1. Empower Users Where They Are, Not Where We Wish They Were

If organizations are going to adopt AI agents, the technology needs to blend in. Agents shouldn’t force employees to change how they work; they should work alongside them.

This is crucial because 45% of people report that "context switching" significantly hinders their productivity. To address this, we must prioritize embedded augmentation by designing agents that live within an organization's current tools and processes. By meeting users where they already are, agents eliminate the cognitive drain associated with jumping between tasks, ultimately making established operations faster, more intuitive, and less intrusive.

2. Eliminate the Drudgery of Security and IT Work

Our day-to-day work is often buried under repetitive tasks, such as triaging phishing alerts where the vast majority provide no real signal.

This burden is underscored by the fact that SOC teams face an average of 4,484 alerts daily, resulting in 67% of these notifications being ignored and 83% being dismissed as false positives. To solve this, great agents should act as a "silent force" that automatically sorts through the noise to generate reports that would normally take humans hours to complete.

This approach is not about replacing human expertise, butrather protecting it by reclaiming valuable time from the grind. As agents successfully reduce the noise, security teams are empowered to focus on what they do best: thinking critically, investigating deeply, and acting decisively.

3. Build Trust Through Transparency

In high-stakes fields like cybersecurity, autonomy without transparency is a deal-breaker because, much like human relationships, agents must earn their trust. Currently, a significant "trust gap" exists, with 60% of security professionals trusting findings verified by humans over AI-generated results, largely because users often cannot see the underlying reasoning. To bridge this gap, we must build agents that show their work, cite their sources, and explicitly acknowledge when they are uncertain. By incorporating a Human-in-the-Loop model that allows users to audit, verify, and adjust the decision tree, we can transform "black box" AI into a transparent and reliable partner.

4. Shift from Reactive to Proactive

Once trust is established, agents can transition from reactive assistants to proactive defenders, which is critical given that 85% of vulnerabilities remain unpatched after 30 days, leaving a massive window for attackers. To solve this, true agents must do more than wait for a command; they should observe silent patterns and anomalies in real-time. This proactive insight serves as a game-changer by allowing agents to work alongside security teams to stay ahead of threats. By scanning, prioritizing, and automating the "low-hanging fruit" of remediation, these agents can effectively close security gaps before an exploit ever occurs.

5. Lead Innovation with Empathy, Not Hype

We must put humans at the center of the design, ensuring we never lose sight of the people who are front and center fighting bad actors every day. This human-centric approach is supported by the fact that 63% of security professionals believe AI has the potential to enhance security, but only as a tool to augment, rather than replace, human roles. To achieve this, we must lead with empathy rather than buzzwords, creating agents that make users feel supported and respected instead of confused or left behind. This process starts by involving real practitioners in the design and feedback loops, ensuring the implementation of clear boundaries and safety mechanisms that protect the ongoing collaboration between humans and machines.

To learn more about AI agents and how to govern them, we invite you to visit our RSAC library.