Large language models (LLMs) are everywhere. They transcribe and translate conversations, power chatbots and digital assistants, analyze consumer sentiment based on customer service calls, moderate online traffic to filter out offensive content, and more. According to the Grand View Research Large Language Model Market Size, Share and Trends Analysis Report the market for LLMs was estimated at $4.35 billion in 2023 and is projected to grow at a compound annual growth rate (CAGR) of 35.9% from 2024 to 2030.
As with any new technology, especially one that is growing at this rate of velocity, LLMs need guardrails and a framework to achieve transparency, accountability, and enforce privacy and security standards. Otherwise, we will see more cases like the recent lawsuit where Air Canada was found liable for incorrect information its chatbot gave a customer buying a ticket online.
While most companies implement guardrails for their IT systems, LLMs pose a different set of challenges from a human resource and technical standpoint. Artificial intelligence (AI) engineers don’t yet fully understand security, and security teams are only in the early stages of the AI learning curve.
On the technology front, solutions that block and remediate threats in traditional IT systems are not equipped to perform the same tasks in AI systems. For example, language models are not deterministic; and will generate a different response for every prompt, making it difficult to apply controls in production. Additionally, models have to adapt to different languages, but many solutions are mainly focused on English leaving a big gap to detect attacks, data leakage, or content moderation in other languages.
How to Start Securing LLMs
To address these challenges, organizations need to start by determining who is responsible and accountable for managing its LLM governance strategy and bringing together key stakeholders from the AI engineering and security teams to agree on the prerequisites for an AI governance framework.
Next, conduct an assessment of whether existing tools or policies are transferable and applicable to AI systems and conduct a gap analysis.
This should be followed by implementing full monitoring of all LLM deployments across first-party, third-party, and consumer LLMs using purpose-built scanners. These should be capable of identifying data leakage, adversarial prompt attacks, and integrity breaches of the LLM itself such as the deviation of the LLM from its intended purpose through offensive content, recommendations, etc.
This monitoring should also be used to establish a baseline of behavior which can then factor in the non-deterministic nature of LLMs. This baseline can be used to identify and fix security gaps before enforcing guardrails.
It’s best to start implementing guardrails and controls on employee applications before even considering the deployment of LLMs into production for customer-facing systems. Unintended consequences from governance controls on internal apps are easier to fix and less damaging than repercussions from overly-restrictive guardrails on customer apps that block too much and don't allow users to accomplish anything.
Applying guardrails in the real-world means dealing with issues of cost and friction. Latency can slow down the user experience, so selecting and knowing what guardrails are needed for which applications is critical. Certain LLM use-cases may not need all guardrails while all LLM use-cases absolutely require prompt injection detection. Overly restrictive guardrails can also put hurdles in the way of developers and users, so balancing guardrails against the risk profile of the assets being protected can give those guardrails a real-world check.
LLMs that perform low risk functions using trusted sources of information that have been pre-vetted against poisoning, injection attacks, etc., may not need strict guardrails. On the other hand, LLMs that access sensitive databases, operational systems, and other critical assets should be governed by zero-trust principles.
This includes enforcing least-privilege access and limiting what actions they can perform, just as much as human users. Only allowing access to required data, assets, and the ability to execute prescribed tasks helps limit the blast radius in the event of a security incident. Just as least-privilege keeps a bad actor with a compromised credential from escalating an attack, a compromised LLM can only do so much damage if it has limited access.
LLMs are here to stay. They drive business value for organizations by improving operational efficiency, reducing costs and improving user experience. But the pace of adoption of AI tools is leaving security gaps that must be addressed. By first building a bridge between AI and security, finding blind spots, and establishing baselines. Before applying risk-based controls, an organization can establish a workable strategy for LLM security governance that can work in the real world.