AIIntegrationGDPRBusiness softwareMethod
AI agents for business on AWS Bedrock: your documents, your tools, real guardrails
9 min read
Anyone can wire a chatbot into a website in an afternoon. The demo goes well, then a colleague asks the first real question: "What is our process for this particular customer?" The model knows nothing about your processes, your customers or your tools. It answers anyway, confidently, and wrongly.
An AI agent that is genuinely useful inside a company has to do three things a consumer assistant does not: know your documents, act in your tools, and never show anyone what they are not allowed to see. This article explains how we build that on Amazon Bedrock, drawing on a platform we designed and put into production: an AI coaching product in which every training scenario is grounded in the documents of the organisation using it.
It is written for the people who have to sign off such a project: a CTO, an IT director, an operations lead. You will find the architecture, the access control model that most tutorials skip, the operating rules for agents, and what we learned the hard way.
What Bedrock gives you, and what it does not
Amazon Bedrock is AWS's managed service for foundation models. From one AWS account you can call models from several vendors, attach knowledge bases, and use features such as Bedrock Guardrails, which filter content and can mask personal data in prompts and answers. AWS also offers Bedrock AgentCore for running agents in production.
What Bedrock does not give you is your business logic. It does not know which department may read which procedure, which actions an agent is allowed to take, or what a wrong answer costs you. Those rules are yours to design, and they decide whether the project passes a security review.
Knowing your documents: retrieval-augmented generation
A language model only knows what it learned before its release. To make it answer from your documents, you do not retrain it: you hand it the relevant passages with every question. That is retrieval-augmented generation, or RAG. For a deeper look at the pattern itself, see our guide to RAG for business documents.
The pipeline we run on AWS looks like this:
- Documents are stored in S3, inside the company's own AWS account.
- A Bedrock knowledge base syncs them. It splits each document into passages and turns each passage into a vector, a numerical fingerprint of its meaning, using Amazon's Titan Text Embeddings model.
- Vectors go into OpenSearch, AWS's managed search engine, together with the text and the metadata of each passage.
- For every question, the application turns the question into a vector, finds the closest passages in OpenSearch, and passes them to the model that writes the answer.
None of this leaves the company's AWS account. AWS's data protection documentation states that model providers do not have access to Bedrock logs or to customer prompts and completions. For an IT department, that sentence is often the difference between a project that starts and a project stuck in the security committee.
Showing only what each user is allowed to see
This is the part demos leave out, and the part that needs the most care.
On our platform, several organisations share the same application. A knowledge base can belong to one organisation, be shared with others, or be attached only to specific scenarios. Every indexed passage therefore carries its organisation, its access groups, its source (an uploaded file, a Confluence page, a Jira ticket) and business topics.
Bedrock offers an all-in-one retrieval call, which is convenient for a prototype. We chose to have the application build the OpenSearch query itself, because that is where access rights are enforced: the organisation filter, the groups, the sources, and full AND, OR and NOT combinations for complex cases. The model only sees what the search returns. No clever phrasing of a question gives it a passage the filter has excluded.
| Approach | Where access is enforced | Risk |
|---|---|---|
| Instructions in the prompt | Inside the model | Prompt injection can bypass it |
| Separate index per customer | Infrastructure | Safe, but costly and rigid when sharing |
| Metadata filter built by the app | Search query, before LLM | Safe if the filter is tested like code |
| Content guardrails only | Model input and output | Masks data types, not document rights |
That filter is the system's main line of defence. It has to be reviewed and tested like security code on every change, because one badly combined condition is enough for one organisation to read another's documents. That is exactly the kind of defect a two-step review, by a second model and then by a developer, is designed to catch.
Acting in your tools: from assistant to agent
An assistant answers. An agent acts: it reads a ticket, books a meeting, queries a database, starts a job. Moving from one to the other changes the nature of the project.
The standard that has emerged for connecting an agent to tools is MCP, the Model Context Protocol. On our platform, MCP servers are declared from the admin area, with their authentication, including OAuth when the tool requires it. Every tool call the model decides to make is recorded with the message that triggered it, so we always know what the agent did and why.
We also contribute to Hermes Agent, an open source agent that runs continuously on a server. Both experiences lead to the same operating rules:
- Least privilege. An agent that needs to read tickets does not need permission to delete them.
- Human approval for sensitive actions. Sending, paying, deleting or changing production data goes through a person.
- A log of every action. Without a trace, an incident cannot be explained, and therefore cannot be fixed.
- A lifespan designed from day one. An agent that runs for weeks must survive restarts and deployments without losing its tasks.
If you process personal data, these rules also make the GDPR conversation much shorter: you can show what the agent reads, what it does and where the data lives. Our GDPR checklist for business applications covers the rest.
Keeping costs visible and the model replaceable
An agent consumes tokens, the billing unit of language models, on every question and every tool call. Our platform records the tokens used by each message along with the price of each model, so the cost of a use case is read in the application rather than discovered on the monthly invoice.
The architecture also keeps the choice of vendor open. Document retrieval sits behind a common interface, with one implementation on AWS and another with a second provider. Changing models, or combining several, does not mean rebuilding the application. Bedrock itself gives access to models from several vendors from the same account, which makes side-by-side comparison on your own questions straightforward.
What we learned in production
The console is not enough to check an index. In our case, the list of indexed documents shown in the Bedrock console proved unreliable, with erratic pagination and documents missing from the list. We check the passages actually indexed directly in OpenSearch.
Model access rights before screens. Own knowledge base, shared knowledge base, base attached to a scenario: each case has its own table and its own uniqueness constraint. Adding them later would have meant a rebuild.
Answer quality depends mostly on the documents. A more powerful model does not make up for an outdated procedure or three contradictory versions of the same document. Sorting the sources is the first step of a RAG project, not the last.
Evaluate on real questions. Collect the questions people actually ask, with the answers your experts expect, and replay them after every change of model, prompt or filter. A demo question that works proves very little.
Before you start: a short checklist
- Which documents, owned by whom, and how current are they?
- Who may read what, and does that rule already exist somewhere you can reuse, such as your directory groups?
- Which actions should the agent take, and which need a human to approve?
- Which AWS region, and does it match your data residency requirements?
- Who reads the logs and the cost report each month?
Where to start
An AI agent is not always the right answer. Sometimes good document search or a well designed business application solves the problem more simply; our article on build vs buy helps frame that choice, and our guide to what a custom AI agent costs covers the budget side.
If an agent is the right answer, we work the way we do for any software: a human-led scoping session, then a free visual prototype presented on a video call within 5 business days, then development at a fixed price or on a subscription, never billed by the hour, and in your own AWS account if you prefer. The result is an agent that knows your documents, acts in your tools, and shows each person only what they are entitled to see.