Design retrieval for customer support knowledge

Real workflow example

A support assistant has access to product docs, billing policies, release notes, and customer account records. When everything is dumped into one search index, the assistant sometimes answers with outdated policy text, shows internal notes to the wrong user, or cites a general article when an account-specific exception applies.

The better design separates source types. Public help articles answer general questions. Internal policies guide what support can promise. Account records provide the user's actual state. Each source has freshness, audience, product area, and permission metadata.

Implementation approach

Start by listing the knowledge classes that support agents already use. A good split is public docs, internal support playbooks, policy/legal text, product changelogs, and customer records. Each class should have a different permission rule and freshness expectation.

Retrieval should return the smallest useful evidence set. Do not ask the model to read twenty chunks when three high-quality sources are enough. Include titles, source IDs, dates, audience labels, and excerpts so the answer can explain where it came from.

The UI should display evidence beside the answer. This lets support staff verify the response, teaches users why the answer is trustworthy, and creates a debugging path when the answer is wrong.

Code or config snippet

type SupportSource = {
  id: string;
  title: string;
  sourceType: "public_doc" | "internal_policy" | "account_record";
  audience: "public" | "support" | "admin";
  productArea: string;
  updatedAt: string;
  excerpt: string;
};

function canUseSource(source: SupportSource, role: "user" | "support" | "admin") {
  if (source.audience === "public") return true;
  if (source.audience === "support") return role === "support" || role === "admin";
  return role === "admin";
}

Mistakes to avoid

• Do not mix private account records with public docs in one permissionless index.
• Do not retrieve stale policy text without showing the update date.
• Do not force the model to choose between conflicting sources without priority rules.
• Do not show an answer when no supporting source was retrieved.
• Do not expose internal excerpts in the customer-facing UI.

Ready checklist

• Source classes are separated by audience and permission level.
• Each chunk has title, source ID, product area, freshness, and audience metadata.
• Retrieval filters by user, organization, role, and intent before ranking.
• The answer includes source references or evidence summaries.
• Low-confidence and source-missing answers route to review.
• Support staff can report bad or stale sources.

Practical tip

Give policies priority over blog posts and changelogs when the question involves refunds, terms, security, or compliance. Retrieval quality depends as much on source priority as embedding similarity.