RAG Pipeline Security Best Practices: A Checklist

Retrieval-augmented generation pipelines fail differently than the applications security teams are used to reviewing. Getting rag pipeline security best practices right means treating the retriever as an untrusted input source, not a trusted extension of your database layer, because the model can’t tell the difference between an instruction from your system prompt and an instruction hidden inside a retrieved chunk. That distinction drives most of what follows.

A RAG stack has three places where security controls actually matter: ingestion (what gets embedded and stored), retrieval (what gets pulled back and handed to the model), and generation (what the model does with what it was handed). Each has a distinct failure mode, and a control that works at one layer usually does nothing at the others.

Why RAG breaks the normal threat model

Traditional access control assumes the query layer enforces permissions. RAG breaks that assumption the moment documents get converted into vectors — a document pulled from Confluence, SharePoint, or an internal wiki loses its original ACLs on the way into a vector store unless something explicitly re-attaches them. If your ingestion pipeline embeds a folder of mixed-permission documents into one collection without carrying metadata for who’s allowed to see what, every user who can query that collection can potentially retrieve chunks from documents they were never authorized to read.

OWASP’s Top 10 for LLM Applications formalizes this as LLM08:2025 Vector and Embedding Weaknesses, alongside LLM02:2025 Sensitive Information Disclosure for the data-exposure angle and LLM01:2025 Prompt Injection for what happens when retrieved content itself carries adversarial instructions. Treat all three as one connected risk surface in a RAG system, because a single unfiltered retrieval call can trigger all three at once — pulling unauthorized data, leaking it into a response, and doing so because a poisoned document told the model to.

MITRE’s ATLAS knowledge base — the adversary-tactics catalog modeled on ATT&CK but scoped to AI systems — added RAG poisoning as a distinct technique in its 2025 expansion, describing it as an adversary planting content in a knowledge base so that malicious instructions get retrieved and executed as if they were legitimate context. That’s indirect prompt injection with a persistence layer: unlike a one-off malicious prompt, a poisoned document sits in the index and fires every time a relevant query touches it, including after model or prompt updates.

Ingestion-layer controls

Poisoning starts here, so this is where filtering has the best cost-to-effectiveness ratio.

• Source integrity. Only ingest from sources you control the write path to, or that go through a review step before indexing. A public wiki, ticket queue, or inbox that feeds your RAG store directly is an open injection channel — anyone who can write to that source can write to your model’s context window.
• Content sanitization before embedding. Strip or neutralize markup, HTML comments, and instruction-shaped text (“ignore previous instructions,” system-prompt-style phrasing) from documents before they’re chunked and embedded. This won’t catch everything, but it removes the cheap attacks.
• Sensitive-data handling at write time, not read time. AWS’s reference architecture for securing sensitive data in Bedrock-based RAG applications lays out two patterns worth knowing: redact PII before it ever reaches the vector store (so it structurally cannot leak at query time), or preserve it but gate retrieval behind role-based metadata filtering. Redaction is the stronger default when the data doesn’t need to survive in the store at all; RBAC filtering is the right call when different users legitimately need different views of the same corpus.
• Provenance metadata. Tag every chunk with its source document, ingestion timestamp, and — critically — the access-control attributes of the original document. If that metadata doesn’t survive chunking, you’ve already lost the ability to enforce permissions at retrieval time.

Retrieval-layer controls

This is where most teams under-invest, because it’s easy to assume “the vector search already scoped the results.”

Enforce permissions at query time, not just at the application’s outer edge. Microsoft’s guidance on multitenant RAG architectures recommends putting an API layer in front of the vector store that acts as a gatekeeper — every retrieval request is filtered against the requesting user’s authorization before results are assembled into a prompt, and the underlying store is never queried directly by application code. The three isolation models it lays out are store-per-tenant (strongest isolation, higher operational cost), a shared multitenant store with a tenant discriminator on every query, and a shared store for genuinely public data. Pick per data sensitivity, not uniformly across your whole corpus — a support-ticket RAG index and a public-docs RAG index don’t need the same isolation model.

Two more retrieval-layer practices worth calling out:

• Log every retrieval, not just every generation. If a response turns out to be wrong or leaked something, you need to know which chunks were retrieved and why, not just what the model said. Audit trails at the retrieval layer are what make incident response possible.
• Rate-limit and monitor query patterns. Repeated near-identical queries probing slightly different phrasings is a common signature of someone trying to extract the full contents of a restricted index through the model’s answers rather than direct database access.

Generation-layer controls

Even with clean ingestion and scoped retrieval, treat everything the retriever hands back as untrusted input to the model, the same way you’d treat user input in a web app.

• Separate instructions from retrieved content structurally, not just with a delimiter in the prompt string — use whatever your model provider’s API offers for role separation (system vs. tool/context vs. user), since plain-text delimiters are themselves spoofable by content that includes fake delimiter tokens.
• Validate and constrain output. If the RAG system feeds downstream tools or takes actions, apply output validation and schema enforcement so a manipulated response can’t smuggle a tool call or command through unchecked. OWASP’s LLM06:2025 Excessive Agency and LLM05:2025 Improper Output Handling categories both apply directly here.
• Watch for system prompt leakage (OWASP’s LLM07:2025) as a signal, not just a risk in isolation — if an attacker can get your system prompt back, they can craft retrieval-poisoning payloads that are precisely tuned to bypass whatever instructions you gave the model.

Governance and the bigger picture

None of this is a one-time hardening pass. NIST’s Generative AI Profile (NIST AI 600-1) frames GenAI risk management as a lifecycle activity spanning design, deployment, and ongoing monitoring — which maps cleanly onto RAG, where your corpus keeps changing after launch and every re-ingestion is a fresh opportunity for poisoned or over-permissive content to enter the index. Build periodic re-audits of what’s in your vector store and who can retrieve it into the same cadence as your dependency and access-control reviews, not as a separate AI-specific process nobody owns.

To turn this checklist into a repeatable assessment with trust boundaries, severity ratings and retest triggers, work through the AI security review checklist; the wider control set for the other nine OWASP categories is in the OWASP LLM Top 10 mitigation guide. If you want to measure a retrieval-augmented agent rather than review it, note that most public safety datasets do not touch indirect injection at all — LLM security benchmarks compared covers the two agent benchmarks that do.

For teams building out broader LLM application defenses beyond the retrieval layer — prompt injection testing, jailbreak detection, and guardrail tooling — see the runtime-filter comparisons on guardml.io and the offensive-testing writeups on aisec.blog, which cover the adversarial side of the same threats discussed here.