Library Header Image Library Header Image

Governance in AI: Cyber & Technology Risk


Posted on by Alexander Miranda , Subramanya Shankar

Key Takeaways
  • AI governance is a cyber risk discipline, not a policy exercise. AI introduces attack surfaces across the full lifecycle — data pipelines, model artifacts, inference APIs, retrieval-augmented generation (RAG), and agent orchestration — that traditional IT controls cannot mitigate alone.
  • Regulation increasingly embeds security requirements. The EU AI Act mandates robustness testing and incident reporting for high-risk systems, and sector regulators such as the NYDFS now expect AI risk to be managed inside existing cybersecurity frameworks.
  • Established frameworks make governance operational. Pairing ISO/IEC 42001 and the NIST AI RMF with threat-informed resources such as MITRE ATLAS and the OWASP Top 10 for LLM Applications turns governance intent into adversarial testing, secure MLOps, and supply chain controls.

AI governance encompasses the policies, frameworks, and controls required to ensure that AI systems are developed, deployed, and operated safely, ethically, and in compliance with evolving global regulations. As binding regimes such as the EU AI Act, US federal and state initiatives, and Asia-Pacific regulatory frameworks emerge, organizations face the dual challenge of regulatory compliance and the management of novel technical risks unique to AI.

Unlike traditional IT systems, AI introduces new attack surfaces across the full lifecycle: training data pipelines, model artifacts, inference APIs, retrieval-augmented generation (RAG), and agent orchestration layers. Threats such as data poisoning, adversarial evasion, model extraction, prompt injection, privacy leakage, and agent hijacking demand that AI governance be treated as an integrated cyber and technology risk discipline, rather than a purely ethical or policy exercise.

Embedding governance early in the AI lifecycle improves regulatory readiness, strengthens stakeholder trust, reduces long-term security risk, and enables organizations to scale AI responsibly as both threats and expectations evolve.

Principles of AI Governance Frameworks

AI governance frameworks are grounded in shared principles, each with direct cyber and technology risk implications. Human oversight requires enforceable controls such as kill switches, audit trails, and override mechanisms resilient to manipulation. Transparency depends on explainability tooling and logging pipelines whose integrity can be assured. Accountability demands clear ownership across training data, model development, deployment, and downstream consumption.

Safety and resilience extend beyond reliability to include adversarial robustness testing, red teaming, and incident recovery capabilities. Fairness and nondiscrimination introduce risks from distribution shifts and adversarial inputs designed to trigger biased outcomes. Privacy requires controls to prevent memorization and leakage of sensitive data through inference. Proportionality ties governance depth to system impact, while supply chain integrity mandates verifiable provenance for data, code, and models.

AI Cyber & Technology Risk Taxonomy

AI-specific threats can be grouped into five major risk domains. This layered view is consistent with analysis published An RSAC 2025 blog, which argues that — much as with cloud-native applications — LLM services comprise distinct architectural layers (training infrastructure, APIs, plugins, orchestrators, and user-facing applications), and that security teams should map threats to the layer where they arise: model theft is most relevant at the inference API, while data poisoning begins upstream in the training data pipeline. MITRE ATLAS catalogs many of these adversarial behaviors as discrete, testable technique IDs (for example, AML.T0051, LLM Prompt Injection), giving the taxonomy below a threat-informed anchor , and the OWASP Top 10 for LLM Applications translates the same threat classes into developer-facing priorities.

Five Major Risk Domains:

Training and data pipeline risks include data poisoning, label manipulation, unverified data provenance, privacy leakage through memorization, and second-order risks from synthetic data.

Model development and Machine Learning Operations (MLOps) risks arise from ML supply chains and CI/CD pipelines, including compromised dependencies, tampered model registries, poisoned training automation, insecure notebooks, and inadequate model versioning.

Inference and deployment risks encompass prompt injection, jailbreaking, adversarial evasion, model inversion, extraction attacks, membership inference, API abuse, and hallucination exploitation — particularly dangerous in agentic systems. These risks are not theoretical: a technical write-up in An RSAC 2026 blog documented a prompt injection attack against Apple’s on-device Apple Intelligence model that bypassed the model’s internal guardrails and both input and output filters simultaneously, achieving an average attack success rate of roughly 76 percent across randomized test prompts .

Agentic and multi-agent system risks introduce new failure modes such as memory poisoning, tool abuse, multi-agent trust boundary violations, and excessive autonomy leading to irreversible actions. Infrastructure and operational risks amplify conventional cloud threats through the high value of model weights and data, including GPU leakage, misconfigured storage, vulnerable model servers, sensitive logging, and inherited risk from third-party AI providers.

AI Legal and Regulatory Frameworks

Global AI regulation increasingly embeds security and robustness requirements. The EU AI Act (Regulation (EU) 2024/1689) requires high-risk AI systems to achieve appropriate levels of accuracy, robustness, and cybersecurity — including resilience against attempts to manipulate system behavior — and imposes serious-incident reporting obligations. The penalties give these obligations teeth: as the Cloud Security Alliance summarizes, non-compliance with high-risk system requirements can draw fines of up to €15 million or 3%of worldwide annual turnover, and prohibited practices up to €35 million or 7%.

US federal policy has shifted. Executive Order 14179 (January 2025) revoked Executive Order 14110 and reoriented federal policy toward AI leadership and reduced regulatory barriers [10]. Rescinding an executive order, however, does not unwind the technical guidance produced under it: NIST’s AI Risk Management Framework and Generative AI Profile remain the de facto baseline for enterprise AI risk management, and state-level laws continue to impose safety, fairness, and accountability obligations across an increasingly fragmented landscape.

Sector-specific guidance extends cybersecurity expectations to AI usage. The NYDFS Industry Letter of October 16, 2024 maps AI-related risks — deepfake-enabled social engineering, AI-enhanced attacks, exposure of nonpublic information, and third-party supply chain vulnerabilities — onto the existing 23 NYCRR Part 500 cybersecurity framework rather than creating new requirements, signaling how regulators expect AI risk to be absorbed into established programs [12]. In banking more broadly, longstanding model risk management expectations under Federal Reserve SR 11-7 and the OCC’s Comptroller’s Handbook — governance, validation, and independent review of models — provide a supervisory template that examiners are now extending to AI and machine learning systems.

Internationally, frameworks such as China’s AI regulations, the OECD AI Principles, and the G7 Hiroshima Code of Conduct reinforce human-centric, secure, and transparent AI development; DLA Piper’s global AI regulation tracker documents how quickly these regimes are proliferating and diverging across jurisdictions. Regulatory non-compliance frequently correlates with gaps in technical security controls.

AI Governance Frameworks

Several major frameworks shape enterprise AI governance. As EC-Council’s plain-English comparison observes, the leading regimes differ mainly in force and focus — the EU AI Act is mandatory law with extraterritorial reach, ISO/IEC 42001 is a voluntary but certifiable management-system standard, and the NIST AI RMF is a voluntary operational playbook — yet all three converge on risk assessment, human oversight, and documentation, so building one program well advances the others.

ISO/IEC 42001 establishes a certifiable AI Management System, defining leadership accountability, lifecycle controls, risk planning, performance monitoring, and continual improvement. Its Annex A controls translate governance intent into policies, lifecycle security gates, data integrity requirements, and third-party oversight.

ISO/IEC 23894 provides structured AI risk management aligned to ISO 31000, explicitly recognizing adversarial machine learning, data poisoning, emergent behavior, and supply chain risk.

MITRE ATLAS functions as the definitive adversarial threat knowledge base for AI, mapping tactics and techniques across the AI attack lifecycle. It enables threat-informed governance, red teaming, and detection engineering, with recent expansions addressing agentic AI risks.

The NIST AI Risk Management Framework (AI RMF) provides a full-lifecycle structure across four functions — GOVERN, MAP, MEASURE, and MANAGE — integrating policy, threat modeling, testing, and control deployment. Its Generative AI Profile (NIST AI 600-1), released in July 2024, addresses LLM-specific risks such as hallucination, prompt injection, and intellectual property leakage.

The OWASP Top 10 for LLM Applications translates governance into developer-focused security practices, prioritizing prompt injection, data poisoning, excessive agency, output handling, and supply chain vulnerabilities.

AI Supply Chain Risk

The AI supply chain — foundation model providers, open-source models, frameworks, datasets, vector databases, plugins, and fine-tuning services — represents one of the most significant and least governed attack surfaces. The NYDFS reached a similar conclusion for financial institutions, flagging that every link in the AI supply chain can introduce exploitable security risk to an organization’s nonpublic information [12]. Effective governance requires provenance verification, behavioral testing, supplier security assessments, and specialized scanning tools capable of detecting malicious model artifacts and adversarial weaknesses.

Modern AI governance must be threat-informed, evidence-based, and continuously updated. AI systems introduce fundamentally new attack vectors that cannot be mitigated through traditional IT controls alone. Organizations that integrate governance frameworks with adversarial testing, secure MLOps, and supply chain controls will be best positioned to scale AI safely, meet regulatory expectations, and sustain trust as the AI threat landscape rapidly evolves

Contributors
Alexander Miranda

Consultant, Optiv Security

Subramanya Shankar

Principal Consultant, Optiv Security India Pvt Ltd

Blogs posted to the RSAConference.com website are intended for educational purposes only and do not replace independent professional judgment. Statements of fact and opinions expressed are those of the blog author individually and, unless expressly stated to the contrary, are not the opinion or position of RSAC™ Conference, or any other co-sponsors. RSAC Conference does not endorse or approve, and assumes no responsibility for, the content, accuracy or completeness of the information presented in this blog.


Share With Your Community

Related Blogs