Enterprise Knowledge Base Multi-Agents on AWS Bedrock

Enterprise knowledge systems become trustworthy when ingestion, retrieval, agents, governance, and cost visibility are designed as one operating system.

A multi-tenant AWS Bedrock architecture for ingesting large PDFs, structured records, semi-structured JSON, and operational knowledge into governed multi-agent workflows with semantic retrieval, evaluation, observability, and FinOps reporting.

At a glance

• Status: Enterprise reference architecture
• Architecture focus: Agents
• Primary outcome: Governed multi-tenant knowledge intelligence
• Stack: AWS Bedrock (Claude 3.5 Sonnet/Haiku), AppSync GraphQL, OpenSearch Vector Engine, Lambda, SQS, Step Functions, CloudWatch, Firehose, S3, Glue, Athena, QuickSight

The challenge

Enterprise teams needed a knowledge base that could reason over multiple source types: large PDF libraries, structured business data, semi-structured JSON, and changing operational records. The system also had to support multi-tenant access by organization, route tasks across specialist agents, control cost, and satisfy governance expectations for GDPR, HIPAA, and PII handling.

The difficult part was not only retrieval. The platform needed a full production control plane: ingestion jobs, semantic cache, model routing, supervisor-worker agents, evaluation tests, observability, security policy, tenant isolation, and cost reporting.

The architecture decision

Use AWS Bedrock as the model and agent foundation, but keep orchestration, retrieval, data governance, and cost telemetry explicit. AppSync GraphQL exposes controlled application access. Lambda and jobs run ingestion and task workflows. OpenSearch stores vectorized knowledge. Bedrock agents follow a supervisor-worker pattern. Firehose, S3, Glue, Athena, and QuickSight turn agent task events into a low-cost FinOps and governance dashboard.

System architecture

The architecture separates ingestion, retrieval, agent orchestration, and governance telemetry so each concern can scale and be audited independently.

Open the standalone architecture and workflow diagrams.

Architecture layers

1. Multi-source ingestion - Large PDFs, structured records, semi-structured JSON, and enterprise source exports enter through controlled batch and event-driven pipelines.
2. Knowledge processing - Jobs and Lambda functions extract text, normalize metadata, classify sensitivity, chunk content, enrich records, and prepare embeddings.
3. OpenSearch vector DB - Vector and keyword indexes support hybrid search with tenant, source, security, freshness, and document-level metadata.
4. AppSync GraphQL API - Applications access knowledge and agent workflows through typed GraphQL operations with tenant-aware authorization.
5. Bedrock multi-agents - A supervisor agent decomposes tasks and delegates to worker agents for retrieval, summarization, reasoning, compliance checks, and action preparation.
6. Governance and FinOps - Agent task events flow through Firehose and Lambda into date-partitioned S3 for Glue, Athena, and QuickSight reporting.

How the system works

1. Ingest - Pipelines load PDFs, structured data, and JSON into a normalized knowledge model with tenant, source, policy, and sensitivity metadata.
2. Retrieve - Queries use semantic cache first, then hybrid OpenSearch retrieval when fresh evidence is needed.
3. Route - LLM routing chooses the right model and agent path based on task complexity, cost budget, latency target, and compliance requirements.
4. Govern - Task events, model choices, retrieval evidence, policy decisions, token usage, latency, and cost are recorded for evaluation and reporting.

Agent responsibilities

The platform used a supervisor-worker pattern so each agent had a narrow responsibility and a measurable output contract.

Supervisor agent

The supervisor agent owned task interpretation, planning, delegation, reflection, and final response assembly. It did not directly query every system. Instead, it selected worker agents, passed scoped instructions, tracked budgets, compared worker outputs, and decided whether the response was complete enough to return.

The supervisor tracked:

• User intent, tenant, role, and allowed action scope
• Required evidence sources and freshness expectations
• Which worker agents were needed
• Token, latency, and cost budgets
• Reflection checks before final answer
• Escalation or graceful failure conditions

Knowledge retrieval agent

The retrieval agent handled semantic and hybrid search. It checked the semantic cache first, then queried OpenSearch when fresh evidence was required. It returned citations, source metadata, confidence, freshness, and tenant policy context rather than only text snippets.

Ingestion agents

Ingestion was also agent-assisted. Dedicated ingestion agents classified source type, inspected document structure, proposed chunking strategies, detected PII, validated metadata, and decided whether a file should move forward, be quarantined, or require human review.

For large PDFs, the ingestion agent checked document size, page structure, OCR quality, section boundaries, tables, and duplicate content. For structured data, it validated schema, keys, types, and tenant fields. For semi-structured JSON, it inferred shape, normalized nested fields, and preserved source lineage.

SQL agent

The SQL agent was responsible for safe analytical queries over approved relational sources. It never received broad database access. The supervisor provided a scoped question, the SQL agent inspected the current schema, generated a read-only query, executed through a controlled tool, reviewed errors, and returned a result with query rationale.

Finance and billing agent

For customer and user-support assistants, this agent retrieved invoices, subscriptions, usage, credits, refunds, and payment status through approved internal tools. It returned facts and next-step options, not unrestricted account changes.

Policy and compliance agent

This agent checked product policy, tenant rules, GDPR/HIPAA/PII constraints, retention policy, and escalation requirements. It could veto an answer, request redaction, or require a human handoff.

Real-time data agent

This agent queried current operational data through controlled APIs or GraphQL resolvers. It was used when the answer depended on live state rather than indexed knowledge.

MCP tool layer

Several MCP tools wrapped internal AWS-hosted applications. Tools exposed bounded operations such as checking account state, retrieving ticket history, querying usage, opening a case, validating entitlement, or reading operational metrics. Each tool had typed inputs, IAM-scoped execution, tenant validation, normalized errors, and audit events.

Ingestion pipeline as an agentic workflow

The ingestion pipeline was not a blind ETL process. It behaved like a governed workflow where agents helped decide how content should become safe, searchable knowledge.

1. Source profiler - Detects whether the input is PDF, table, JSON, export, or API payload and extracts source lineage.
2. PII classifier - Flags sensitive fields, regulated content, PHI, personal data, and tenant-specific restrictions.
3. Chunk planner - Chooses chunking and metadata strategy based on document type, table structure, section hierarchy, and retrieval use case.
4. Index validator - Confirms embeddings, metadata, tenant keys, freshness fields, and OpenSearch index readiness before publishing.

Failures did not automatically poison the knowledge base. Bad files moved to quarantine with reason codes such as OCR failure, missing `org_id`, unsupported schema drift, PII policy conflict, or low extraction confidence.

Reflection loop and failure auto-heal

The supervisor agent used a bounded reflection loop before final output. Reflection was not open-ended self-critique; it was a checklist against the task contract.

The verification step checked:

• Did every claim have retrieved evidence or tool output?
• Did the response stay inside tenant and role permissions?
• Did any worker disagree with another worker?
• Did the SQL query run against the current schema?
• Did model routing stay inside cost and latency budget?
• Did policy or compliance require redaction or escalation?

Auto-heal actions were deliberately limited:

• Retry retrieval with adjusted filters when evidence was weak
• Refresh schema before regenerating a SQL query
• Fall back from semantic cache to live retrieval when freshness was uncertain
• Re-route to a smaller or stronger model based on task difficulty
• Ask a worker agent for a narrower answer instead of expanding the whole prompt
• Escalate to human review when policy, PII, or confidence thresholds failed

The loop had a maximum number of repair attempts. If the task still failed, the assistant returned a transparent partial answer, asked for clarification, or escalated rather than hallucinating.

SQL agent ReAct behavior under schema change

The SQL agent followed a ReAct-style loop: reason about the question, inspect the schema, act through a read-only SQL tool, observe the result or error, and revise only within the allowed query contract.

When a database changed suddenly, such as a renamed column, new table, removed join key, or changed data type, the SQL agent did not keep retrying the same query. It reacted through a controlled recovery path:

1. Observe error - Capture database error, failed column, failed relation, or type mismatch.
2. Refresh schema - Call the schema-inspection tool for current tables, columns, constraints, and sample-safe metadata.
3. Compare intent to schema - Map the user question to available fields and identify whether the original intent is still answerable.
4. Rewrite safely - Generate a new read-only query using the refreshed schema and tenant filter.
5. Validate result - Check row count, null rate, aggregation shape, tenant boundary, and expected units.
6. Escalate if needed - If the schema no longer supports the question, report that the data contract changed instead of fabricating an answer.

Every SQL query carried `org_id`, role scope, row-level policy constraints, and a maximum cost/time budget. The SQL agent could explain the query path, but it could not bypass policy or run write operations.

Autonomous SRE agents: self-healing production operations

The same bounded-agent pattern extends past the customer-facing workers into observability. Production exceptions do not wait for an on-call engineer to open a dashboard — CloudWatch Subscription Filters stream exceptions through SQS to a dedicated observability agent that investigates the failure itself and drops a pre-debugged remediation ticket into Slack before a human is paged.

The observability agent is built on the same MCP tool-agent pattern used for finance and real-time data access — typed, IAM-scoped tools for infrastructure diagnostics, with every action still passing through the platform's audit stream. On each event it:

• Parses the stack trace and exception metadata carried by the SQS event
• Runs diagnostic checks against infrastructure APIs to isolate the failing component
• Takes a bounded, reversible protective action when the failure risks data integrity — such as pausing a pipeline — rather than only observing
• Posts a prescriptive, pre-debugged remediation ticket to the team's Slack channel with the root cause, the action already taken, and the exact fix required

This removes the manual triage phase entirely. An engineer opens the ticket already knowing the root cause and the fix, not just that a job is failing — which is most of where the mean-time-to-resolution improvement comes from. It is not faster paging; it is skipping the investigation step altogether.

A production incident: schema drift in the ingestion pipeline

A SQL ingestion pipeline breaking on schema drift is a representative example. A source table renamed a column the pipeline depended on, and instead of a generic failure alert reaching an engineer, the observability agent produced:

• Alert: Pipeline ingestion paused
• Root cause: Schema drift detected in the `users` table — the `billing_address` column was renamed to `invoice_address`
• Automated action taken: Paused the active ingestion batch through Step Functions to prevent partial or corrupted writes
• Remediation required: Update the schema model in `/models/user.py`, line 42, to match the renamed column

The protective action and the diagnosis happened before the batch could write a single corrupted row, and the ticket that reached the team already named the file and line to fix.

The concierge model: one supervisor, four specialist workers

The customer and user-support assistant is where this platform's agent design is easiest to see end to end. A single large prompt trying to handle authentication, billing, troubleshooting, and retention at once becomes slow, expensive, and hard to evaluate — every response pays for capability it did not need. The concierge model splits that responsibility instead of enlarging the prompt.

1. Concierge supervisor (Claude 3.5 Sonnet) — Reads the incoming request, classifies intent, and delegates to one or more of the four workers below. It owns conversation state, resolves conflicts between worker findings, and assembles the final response. Running the stronger, more expensive model only here — not on every worker call — is what keeps the system both capable and affordable; the routing logic behind that choice is covered in the optimization section below.
2. Account and auth worker (Claude 3.5 Haiku) — Verifies identity, checks session and permission state, and handles password, MFA, and access-scope questions through typed identity tools.
3. Billing and finance worker (Claude 3.5 Haiku) — This is the finance and billing agent described earlier, reused here: it reads invoices, subscriptions, usage, credits, and refund status, and proposes — never silently executes — billing changes.
4. Tech support and RAG worker (Claude 3.5 Haiku) — This is the knowledge retrieval agent described earlier, reused here: it answers product and troubleshooting questions from the OpenSearch knowledge base, with citations and confidence attached.
5. Escalation and retention worker (Claude 3.5 Haiku) — Applies the policy and compliance agent's escalation rules to detect churn risk, repeated frustration, or policy-sensitive requests, and prepares a full-context handoff to a human instead of letting the model attempt a save on its own.

Each worker only sees the tools and data relevant to its own domain — the account worker cannot read billing data, and the billing worker cannot alter permissions. The concierge supervisor is the only component that sees the full conversation, which is also what keeps the audit trail for a multi-domain request coherent instead of scattered across independent tool calls.

Architecture decisions and trade-offs

ADR-01: Static knowledge boundary, dynamic agent reasoning

The knowledge base owns ingestion, metadata, access rules, and retrieval quality. Agents reason over approved evidence rather than directly improvising over raw enterprise files.

Trade-off: Ingestion and metadata quality require upfront discipline, but retrieval becomes governable, testable, and tenant-aware.

ADR-02: Supervisor-worker agent pattern

A supervisor agent plans the task and delegates bounded work to specialist workers: retrieval, summarization, compliance review, calculation, and final response preparation.

Trade-off: Multi-agent orchestration adds trace complexity, but it prevents one broad prompt from hiding planning, evidence selection, and compliance behavior.

ADR-03: Semantic cache before model escalation

Repeated questions and known answer patterns hit a semantic cache before expensive retrieval and generation paths are used.

Trade-off: Cache invalidation must respect source freshness and tenant boundaries, but it reduces latency and cost for repeated enterprise queries.

ADR-04: FinOps as part of the agent platform

Every agent task emits cost, latency, model, tenant, and outcome events. Firehose streams events into S3 with date partitions; Glue updates the Data Catalog; Athena powers QuickSight dashboards.

Trade-off: Telemetry design becomes part of the architecture, but cost attribution and usage governance are available without expensive operational tooling.

Cost and latency optimization

The concierge model above only stays affordable and fast if the surrounding platform is tuned for it. Five techniques do most of the work.

1. Semantic caching at the gateway — Repeated or near-duplicate questions are matched against a semantic cache before any model call is made, not just before retrieval. This is ADR-03's cache-before-escalation rule applied one layer earlier, at the request boundary, so a cache hit avoids the concierge supervisor call entirely.
2. Hierarchical routing instead of a single model — The concierge supervisor runs on Claude 3.5 Sonnet; every worker runs on Claude 3.5 Haiku. Most requests are narrow and well-scoped once triaged, so a small, fast model handles them correctly, and the larger model is reserved for triage, ambiguity, and final response assembly rather than every step of every task.
3. Minimize tool definition bloat — Each worker receives only the tool schemas relevant to its own domain, not the platform's full tool catalog. A smaller tool manifest means fewer tokens spent on every call and less room for the model to select the wrong tool.
4. Enable context prompt caching — System instructions, tool schemas, and other stable context are cached at the prompt level, so repeated calls to the same worker do not re-pay the input-token cost of static content on every turn.
5. Enforce aggressive token budgeting and early exits — Every worker call carries a token and step budget, and the loop exits as soon as the task contract is satisfied instead of continuing to generate. This is the same discipline behind the reflection loop's bounded repair-attempt cap, applied to routine calls rather than only failure recovery.

Governance, security, and compliance

• Tenant isolation: Every record, embedding, cache entry, query, and task event carries `org_id`.
• Authorization: AppSync resolvers, IAM policy, and row-level controls enforce tenant and role boundaries.
• Encryption: KMS protects S3, OpenSearch, logs, and sensitive intermediate artifacts.
• PII controls: Ingestion classifies sensitive fields and document sections before indexing or retrieval.
• Compliance posture: GDPR, HIPAA, and PII concerns are handled through retention policy, access logging, encryption, and least-privilege IAM.
• Auditability: Agent decisions preserve prompt context, retrieved evidence, model route, policy checks, latency, cost, and final outcome.

Evaluation and observability

The system evaluates retrieval, generation, routing, policy, and cost separately. Test suites cover source coverage, citation quality, permission boundaries, hallucination risk, latency budgets, cache hit behavior, and model-routing decisions.

Operational traces capture supervisor plans, worker steps, retrieval calls, semantic cache decisions, OpenSearch query metadata, Bedrock model usage, policy denials, and user-visible outcomes.

FinOps dashboard

The FinOps path is deliberately low-cost:

1. Agent task events are emitted from workers and supervisor orchestration.
2. Lambda enriches events with tenant, route, model, cost, latency, and outcome metadata.
3. Firehose writes compressed, date-partitioned data to S3.
4. AWS Glue crawlers or jobs update the Data Catalog.
5. Athena SQL queries produce tenant, model, workload, and time-window cost views.
6. QuickSight dashboards expose usage, cost, latency, cache efficiency, and anomaly trends.

Production qualities

• Reliability: Job retries, idempotent ingestion, dead-letter handling, and task-level traceability.
• Scalability: Independent scaling of ingestion, indexing, API access, agents, and analytics.
• Cost control: Semantic cache, hierarchical Sonnet/Haiku routing, prompt caching, token budgeting, date-partitioned S3, Athena SQL, and lightweight QuickSight dashboards.
• Security: IAM least privilege, KMS encryption, tenant-aware policies, row-level controls, and protected PII handling.
• Governance: Evaluation tests, trace logs, compliance metadata, and auditable agent task events.
• Operational resilience: Autonomous SRE agents triage production exceptions and post pre-debugged remediation tickets before an engineer is paged.

Outcomes

• Unified knowledge access across PDFs, structured data, and semi-structured JSON
• Multi-agent reasoning with explicit supervisor-worker boundaries
• Tenant-aware retrieval and governance using `org_id`
• Cost-effective FinOps reporting through S3, Glue, Athena, and QuickSight
• Evaluation and observability designed into the agent platform
• Lower mean-time-to-resolution from self-triaging SRE agents that remove the manual investigation step

Technology stack

• AWS Bedrock
• Claude 3.5 Sonnet and Claude 3.5 Haiku
• AppSync GraphQL
• OpenSearch Vector Engine
• AWS Lambda
• AWS Glue
• Amazon Athena
• Amazon QuickSight
• Amazon S3
• Amazon Data Firehose
• Amazon SQS and Step Functions
• CloudWatch Subscription Filters
• IAM and KMS

Related writing

• Production agent systems need bounds, not bigger prompts
• Evaluating an enterprise AI assistant beyond answer accuracy
• Scale data pathways before code complexity
• Make each agent do less — cost and incident response both improve