Multi-Tenant RAG: How to Keep One Customer's Documents Out of Another's Answer

Start with the data boundary

A document assistant can produce a plausible answer and still expose another customer's data. The failure may be in the passage it retrieved, the filename it cites, or a log that records the passage. Protecting the answer alone is too late.

Treat the signed-in user and their current permissions as inputs to retrieval. The server must resolve the permitted collection before the search starts. A tenant ID supplied by the browser is a request parameter, not proof of membership.

Map a request to an authorized scope

Authenticate the request, check the user's membership, then look up the store or collection that belongs to that scope. If access has been revoked, fail closed. Do not let a model select an arbitrary store merely because the store name appeared in a document.

The example is application pseudocode. It shows the order of checks, not a complete authentication or database implementation.

// Application pseudocode: implementations are deliberately omitted.
async function selectCollection(session: Session, requestedTenant: string) {
  const membership = await db.memberships.findActive(session.userId, requestedTenant);
  if (!membership) throw new Error("Access denied");
  const collection = await db.collections.forTenant(membership.tenantId);
  if (!collection) throw new Error("No authorized collection");
  return collection;
}

Choose your isolation level

Separate stores can make ownership easier to reason about. A shared store with metadata filters can be useful for other workloads, but it demands careful, tested filter construction and defense in depth. Neither choice removes the need to authorize each request.

For OpenAI vector stores, file attributes can be used in retrieval filters. Build the filter on the server from verified identity. Do not rely on the prompt to tell the model which documents it should ignore.

• OpenAI retrieval: file attributes and filters ↗

Protect citations and operational data too

Apply the same access policy when a user opens a cited document or a source preview. A source link is another read operation. If a user loses access between answer generation and opening the link, the document endpoint must check again.

Keep logs useful without storing raw passages by default. Record the request ID, permitted scope, result status, and document IDs only when your policy permits it. Limit who can inspect traces.

Prove isolation with two synthetic tenants

Create two accounts with deliberately different policy answers. Ask the same question as each user, then attempt cross-tenant access through search, source links, filters, and cached responses. A denied request must not reveal the other tenant's filenames or whether a private document exists.

• Test a user who belongs to no tenant and a user whose membership was revoked.
• Repeat the test after changing a document's permissions or deleting it.
• Review cache keys and background jobs for the same tenant boundary.

Plan for a permission change

Decide how quickly revocation must take effect, then test that behavior across ingestion, indexes, caches, source downloads, and audit logs. A deleted source that remains retrievable is an incident, even if the answer looks correct.

Keep your access checks in application services you can test independently. Retrieval configuration should reinforce that boundary, not be its only implementation.

• OpenAI file search and filtering ↗

Walk through a policy question

Imagine a support assistant serving two companies. Both have an expense policy, but one permits reimbursement for a particular expense and the other does not. A user from company A asks about the rule. Your server authenticates that user, resolves their active membership, and selects only company A's permitted collection. The retrieval request then searches within that scope. The answer should cite only sources the user can open.

Now repeat the request with company B's collection ID in the browser payload. The server should ignore or reject the unauthorized ID after checking membership. Test this as an integration scenario: a unit test of a filter function alone cannot prove that every route, job, cache, and source endpoint uses the same boundary.

Include an expired or revoked membership in the test. The expected result is an access denial or another deliberately designed response with no private passage, filename, citation, or document-existence clue from the other company.

Review ingestion, caching, and incident response

Retrieval authorization begins before the question is asked. Ingestion should attach ownership and version information from a trusted workflow, not from a document's own text. Check file status before allowing search, and define what should happen if processing fails or a document must be replaced.

Caches need the same boundaries. A cached answer or passage must be scoped to the permissions that produced it, and a source preview must be reauthorized when it is opened. Design a way to invalidate cached material after a permissions change. Time-to-revocation is a product requirement that should be measured in your deployment.

If a cross-tenant result appears, stop the affected path, preserve a restricted incident record, identify the source of the exposure, and test the correction against both tenants before restoring access. Decide the people and process for that response before the first customer uses the assistant.