It started with everyday back-office support. When ops hit a problem, the first reflex was to post in the engineering channel — what happened to this order, what does this rule do, can someone pull this data — and every question interrupted an engineer. The strange part was that the tools already existed. The knowledge assistant had been live for six months, but documents went stale and retrieval wasn't accurate enough for the way people actually phrase things; after a few misses, nobody trusted it. Read-only database access had been granted long ago, so people ran raw queries through Claude Code and produced revenue numbers that were wrong in a very convincing way.
As the PM, my read was that only one piece was missing: AI couldn't see the code, and didn't know the correct way to query. So instead of writing yet another query handbook, I locked "how this company should actually query" into 12 read-only tools, so every query carries the correct method with it — ops ask their own AI and get the answers they used to queue an engineer for.
AI isn't short on the ability to write SQL. It's short on this company's common sense. What this project did was turn that common sense into infrastructure.
CHALLENGE 01
"What's revenue this month" sounds like one line of SQL. In our system it's a minefield: the amount has to come from the detail view rather than the order total, it has to be grouped by product line, the order date has to be the creation time rather than the recurring-charge date, and charity donations have to be excluded. Miss any one of those and the number is wrong — convincingly wrong, because it still returns something plausible. This actually happened: ops produced a revenue figure that didn't match the admin dashboard, an engineer was pulled in to reconcile it, and half a day disappeared.
Restricting access doesn't fix this, and neither does a handbook — the mines aren't in the permission layer, they're in the business logic. My answer was to lock that common sense into the tools: the revenue calculation went into the MCP, so everyone asking the same question gets the same number.
CHALLENGE 02
Before building anything I did a round of user research the way I would in UX — not an inventory of features, but of how each group actually gets through the day:
| User | Had | Missing | So… |
|---|---|---|---|
| Ops | SQL read access, Claude Code, knowledge assistant | Correct query method, code access | Raw queries gone wrong, or a queue for an engineer |
| Customer service | Knowledge assistant | Fresh documents | Back to asking people when search misses |
| Engineers | Everything | Time | Interrupted several times a day |
Not one person was missing access or tools. What was missing was a trustworthy path wiring the existing pieces together. This was never a "build a new feature" problem — it was a wiring problem, and that framing decided every design choice that followed.
CHALLENGE 03
This was the longest argument of the early project. The intuitive route was to add capabilities to the knowledge assistant — the users were already there, the interface existed, it looked cheapest. As PM I blocked that convenient route and chose to build new, for three reasons:
One: a live service is not a lab. The knowledge assistant is a production service that customer service uses on the phone every day. Bolting brand-new capabilities like database queries and code reading onto it means gambling its stability on every iteration. That call was later validated — the one deployment we made to it in order to share a capability caused two brief outages.
Two: pick the wrong interface and the whole product is wrong. The knowledge assistant lives in Teams, which suits one question, one answer. But the real working interface for the target users was already Claude Code — they didn't want somewhere else to type, they wanted the AI already in their hands to get stronger. MCP is an interface built for AI: once it's connected, ops' Claude, engineering's Claude and any future AI can all use it, without being locked to a single chat window.
Three: capabilities can be borrowed, not relocated. Rather than copying the assistant's retrieval and write logic, I had it expose two small internal APIs, and the new tool borrows them for knowledge-base search and document editing — so the company's document write logic only ever exists once, instead of two copies drifting apart.
Extend or build new — the test isn't cost, it's who carries the risk.
APPROACH
Build a read-only MCP that locks business logic into tools. The value isn't "providing database access" — that already existed. The value is that every query automatically applies the correct method: with the revenue calculation locked inside the tool, everyone asking the same question gets the same number.
The plan was one core with two outlets: the same set of tools, with Phase 1 letting ops self-serve inside Claude Code and build the habit, and Phase 2 feeding engineering's automated triage — an error comes in, the bot runs the same tools and produces a handover document. Get the core solid once, and both sides benefit.
FEATURES
Post the order number in the engineering channel and wait
query_sql + query_mongo
I insisted on including Mongo: the real invoice issuance result, whether the customer ever reached the completion page, and the checkout snapshot of a lost order. None of that truth lives in SQL; connecting SQL alone would leave a whole class of questions permanently unanswerable.
Dig through docs (possibly stale) or ask a senior colleague (possibly misremembering)
search_code
It greps the remote default branches of six repos directly. Code is the only documentation that never goes stale.
Ask someone to resolve the mismatch between the docs and the system
explain_business_rule
It greps the code and semantically searches the documents at the same time, returning both side by side for the AI to compare. The code is deterministic, and it holds up the credibility of the whole answer.
Find a stale document and schedule a separate update
propose_kb_update / apply_kb_update
Generate an edit draft at the moment of discovery, then publish after a human confirms. It turns keeping knowledge fresh from someone's to-do item into a side effect of looking something up.
Work out the investigation path again for every similar issue
diagnostic_playbook
It distils the fixed investigation path for each class of problem into a readable SOP.
Ops writes the requirement, then engineering fills the gaps question by question
draft_requirement_guide
It helps ops find the integration points, the template and the completeness gaps while they're still writing the requirement.
DECISIONS — ADR
Every one of these had a more convenient alternative, and I wrote down the trade-off exactly as it stood at the time. They all went into an append-only decision record (ADR), so whoever inherits this can see why it was built this way.
Decision one: validate on existing infrastructure, then let the data decide whether to scale
The engineers proposed putting the MCP directly on the main site as a /mcp route, which was the fastest option. I rejected it because search_code clones six repos and runs git grep, potentially competing with production traffic. The Windows test plan did not support Python and already hosted nine sites, so I deployed the tool on the existing Linux B1 test plan at zero additional cost.
I knew this was not the most stable long-term architecture, but the immediate goal was to validate demand at the lowest cost. We would first observe adoption and response times; if resources fell short or colleagues clearly depended on the tool, the data could justify an upgrade or dedicated service and support more company-wide AI tools.
Decision two: the permission model
All data open, no role tiers. The boss had already granted ops read access, and if the MCP were stricter they'd simply go back to raw queries — which is exactly the problem I was solving. Restricting access doesn't make the numbers correct.
Decision three: what the queries hit
query_sql always hits the read-only replica and never touches the production database. This is blast-radius thinking: the worst case is the replica slows down and customers checking out never notice. I don't want to know what the worst case looks like against production.
Decision four: the write boundary
Permanently read-only against production code, databases and cloud resources. The only write entry point is the knowledge base, and nothing lands without human confirmation. The worst a read tool can be tricked into is a wrong answer; a write tool going wrong is a different category of problem.
The point of an architecture decision isn't what you picked — it's writing down what you didn't pick, and why. Each of these four has since shut down one round of "why didn't we just…".
AI COLLABORATION
Almost all the code was written by AI — I'm not going to dodge that. But everything this tool knows, I taught it. My job isn't typing; it's feeding the AI the real business context behind the code and the data, so what it produces grows on the right logic the first time. Three concrete examples:
Teaching it how revenue is calculated. Knowledge like "the order date is the creation time, not the recurring-charge date" is something AI can't guess and column comments won't tell you — it came from half a day of chasing a reconciliation that didn't add up. It used to live in my head and in chat logs, needing to be re-taught every time a new conversation started. Now it's locked in the tool layer, and everyone, and every AI, gets it automatically.
Calibrating it with real examples. draft_requirement_guide doesn't have the AI invent a template. I fed it real cases from our requirement backlog — four well-written ones, three that got sent back, four unusually complex ones — and distilled the four things you can't omit: scope, states, edge cases, definitions. Every rejected requirement had died on one of those four. This tool moves that interrogation forward to the moment ops writes the requirement.
Accepting its uncertainty and covering it with architecture. Knowledge-base retrieval has an honest limitation: query expansion is done by an LLM, so asking the same question twice shifts the ranking. My call was not to chase perfect retrieval, but to use the composition of explain_business_rule so that deterministic code holds up the retrieval that wobbles.
I don't train models, I train tool boundaries. Models get swapped and conversations end; the business logic written into the tool layer stays.
GOVERNANCE
Knowledge that changes lives in markdown files; only stable capabilities stay in code. diagnostic_playbook came out of that idea — the fixed investigation path for each class of problem is its own .md, and the tool only lists them and returns the full text. Adding a new investigation path means dropping in a file, with no code change and no redeploy.
This is part of a company-wide AI governance model: know-how from individual conversations first settles into version-controlled, reviewed skills (managed in GitHub, changed through PRs), and the ones that are high-risk, high-frequency and must give everyone the same answer — like the revenue calculation — sink further down into the MCP tool layer. The lower it goes, the more stable it is, the harder it is to get wrong, and the harder it is to change. Put it at the right layer and you almost never need to ship a release for it.
SECURITY
This tool lets more than a dozen people touch company-wide data directly. Each person gets their own token with no shared credentials (reusing the existing PAT system from the company back office, so revoking one person doesn't affect anyone else); credentials never land on anyone's laptop (centrally hosted, with connection strings existing only on the server); and an audit log records every call — who, which tool, what they queried, how long it took, how much came back.
Even the authentication mechanism has to be verified. After wiring up PAT I found a trap: even if you paste the old shared key, the request still returns 200, which looks like PAT is working. My test in the end was to watch whether the "cached identities" count on the health endpoint went up by one — only requests that genuinely go through PAT authentication enter that cache.
"The request succeeded" and "it took the path you think it took" are two different things.
CHALLENGE 04
Once the tool was built I thought the hard part was over. Wrong — the hard part was installing it on the laptops of more than a dozen people who had never opened a terminal.
Version one: a handbook plus "let AI walk you through it". It sounds reasonable, but most colleagues didn't even know what MCP was, and AI handed a handbook will very dutifully say "please open the terminal" and "you need to install git first" — each of which is gibberish to a non-technical colleague. It ended with IT installing it machine by machine. The mistake was in my assumption: I thought handbook plus AI equalled having someone guide you, when in practice the AI just translated the handbook into a series of instructions the user couldn't carry out.
To someone who doesn't use a terminal, "please open the terminal" and no instructions at all are the same thing. The barrier isn't the user's problem, it's the designer's.
Version two: an installer, no thinking required. I moved everything the AI would have told the user to do themselves into a single install script: environment detection is no longer guesswork, the script works out how far this machine has got and fills in whatever is missing, and anyone who gets stuck halfway can rerun it and pick up from where it stopped. Alongside it, a skill for helping colleagues install company tools, whose behaviour is to do the work and explain each step in plain language — rather than setting the user an exam. On the day it was released company-wide, colleagues who had never had IT sit with them installed it themselves and started querying straight away.
Adoption is designed, not announced.
RESULTS
Live on 8/18, released company-wide on 9/1. As of 9/1: 1,702 tool calls, 14 colleagues querying for real, 17 completed installs, with a single-day peak of 404. query_sql was used most (740 calls, 3.4s average), followed by search_code (342 calls, 7.3s average).
More interesting than the numbers is who's using it: the second and third heaviest users aren't engineers. They now look up members, orders and product settings themselves, instead of posting in the engineering channel and waiting.
LESSONS
Going live taught me the most. For the two days before launch SQL was actually broken (once the Azure gateway cut the global connection cache it returned "Not connected" forever and never self-healed); the container had no git, so search_code failed outright for one to two minutes after every restart; two settings threw errors when unconfigured with error messages that gave no hint at all; and the backup schedule failed six nights running before I noticed.
The worst came on release day, when four "the system reports success but it's actually broken" problems surfaced — one of them a wrong parameter in the server sync command, which meant every update to the query methods had never once taken effect, while the system reported success from beginning to end.
"No error message" does not mean "working". The failures that deceive you are never the red ones — they're the green ones.
So now, whenever I finish something, I ask the AI one more question: alright, now assume it didn't actually take effect — how would you prove it really ran?
NEXT
Pipe the audit log into Log Analytics (it currently only writes to the container and is cleared after three days); work backwards from the query history into tool design — the questions people actually type are the source material for the next batch of playbooks; and engineering's automated triage (Phase 2), where an error comes in, the same tools complete the investigation and produce a handover document, and an engineer takes it from there, with no changes executed automatically.
STACK
Python 3.13, FastMCP (Streamable HTTP), Azure App Service (Linux B1, reusing the existing plan at zero additional cost), an Azure SQL read-only replica, Cosmos DB for MongoDB, Bitbucket (six repos mirror-cloned with scheduled fetches), knowledge base reused through the internal APIs of the Teams knowledge assistant, PAT authentication and audit logging, and architecture decision records (ADR). Internal tool; source not public.