Securing LLM and RAG Applications
AI apps add new attack surface: prompt injection, RAG data leakage, unsafe tool execution, and secret exposure. A practical security checklist for LLM apps.
AI applications inherit every traditional web and API risk — and then add a new attack surface that standard penetration tests do not cover. If your product uses an LLM, retrieval-augmented generation (RAG), tools/function-calling, or autonomous agents, you need to test for prompt injection, data leakage, unsafe tool execution, and secret exposure.
The top AI application security risks
| Risk | What it looks like |
|---|---|
| Prompt injection | User input overrides system instructions to change behavior or leak data |
| Indirect prompt injection | Malicious instructions hidden in retrieved documents or web pages |
| RAG data leakage | The model surfaces another tenant’s or restricted source data |
| Unsafe tool execution | The model invokes actions (email, payments, code) without proper guardrails |
| Excessive agency | An agent has more permissions or autonomy than the task requires |
| Secret / key exposure | API keys or system prompts leaked via output or logs |
What is indirect prompt injection?
Indirect prompt injection is the AI risk teams most often miss. Instead of attacking the chat box directly, an attacker plants instructions in content the model will later read — a document in your RAG knowledge base, a support ticket, or a web page an agent browses. When the model retrieves that content, it may follow the attacker’s instructions. Treat all retrieved content as untrusted data, never as instructions.
A practical AI security checklist
- Separate system instructions from untrusted content; use strict templating
- Apply least privilege to tools and allow-list permitted actions
- Add human-in-the-loop confirmation for high-impact actions
- Enforce tenant isolation and source authorization in RAG retrieval
- Filter and validate model output before it is rendered or executed
- Keep API keys and system prompts out of responses, logs, and traces
- Log AI interactions for auditability and abuse detection
- Run adversarial / red-team testing for injection and jailbreaks
These map closely to the OWASP Top 10 for LLM Applications, which is a useful reference for scoping AI security work.
Do traditional pen tests cover this?
Only partially. A standard web/API test covers the application around the model, but the AI-specific layer needs dedicated testing. AssuranceOps’ AI App Security Assurance combines prompt-injection and RAG leakage testing, tool-execution boundary review, and secret/PII exposure checks with traditional web/app/API testing.
Ready to test your own systems? Request a security assessment or explore Security Assurance packages.
Frequently asked questions
- What are the biggest security risks in LLM and RAG applications?
- The most common are prompt injection (including indirect injection via retrieved documents), sensitive data leakage from RAG sources, unsafe tool or function execution, excessive agency, model output handling flaws, and exposure of API keys and secrets. Many are summarized in the OWASP Top 10 for LLM Applications.
- What is indirect prompt injection?
- Indirect prompt injection is when malicious instructions are embedded in content the model later retrieves — for example a document in a RAG knowledge base or a web page an agent reads — causing the model to follow attacker instructions instead of the system prompt. Treat all retrieved content as untrusted data, not instructions.
- Do traditional penetration tests cover AI risks?
- Not fully. Standard web/API pen tests cover the surrounding application, but AI-specific risks like prompt injection, RAG leakage, and tool-execution boundaries require dedicated AI application security testing in addition to traditional testing.
Prove your systems are ready.
Human-validated security assurance with an audit-ready evidence pack.
Request an assessmentRelated reading
- Penetration Test vs Vulnerability Scan: What’s the Difference?
Scans are automated and cheap; pen tests are human-validated and prove real risk. When to use each — and what auditors and customers actually expect.
- How Much Does a Penetration Test Cost?
What a pen test actually costs in 2026, the factors that move the price, and how to scope an assessment so you don’t overpay or under-test.
- SOC 2 Evidence Collection Checklist
What evidence SOC 2 auditors actually ask for, organized by control area, with concrete examples of artifacts that pass review.