← Back to home WHY ENUMERATE THE INFINITE?

If the scenarios can never be fully enumerated, why enumerate at all?

AI Tool · Jul – Sep 2026 · Rule design, MCP development, testing

A season ending early, weather, quality below shipping standard — fresh-grocery e-commerce has a cancellation scenario nobody else has: cancelling a whole batch of orders at once and issuing store credit as compensation. Parent-and-child order structures multiplied by three coupon types and discount caps mean calculating each one by hand takes days, and a wrong number is a customer complaint. As PM, I first scoped 12 refund rules the traditional way — then overturned my own plan: no back-office button, no scenarios hard-coded into a pile of if-else. Instead, an AI tool that stops and asks a human whenever it isn't sure about an order, so the rules grow after launch. The real output of this project was breaking the conventional development assumption that every rule has to be settled before development and carved into code. And this one — from rule design to code — I wrote myself.

0 Unit tests, all passing
0 MCP tools I wrote myself
12 + 7 Refund rules + must-stop scenarios
Days → minutes Processing time per batch
CANCEL & REFUND COPILOT RULES AT RUNTIME, NOT UPFRONT 12 REFUND RULES 6 HARD GUARDS 7 STOP CONDITIONS UNKNOWN CASE → AI STOPS → ASK A HUMAN → ANSWER BECOMES A RULE TEACH THE SYSTEM TO ADMIT WHAT IT DOESN'T KNOW
MASS CANCELLATIONS WEATHER · SEASON END · QUALITY CANCEL A BATCH OF ORDERS × SINGLE vs SPLIT ORDERS × COUPON TYPES × CAP RULES DAYS OF MANUAL MATH EVERY REFUND HAND-CALCULATED, HAND-NOTED, HAND-CREDITED — SLOW AND OCCASIONALLY WRONG

CHALLENGE 01

Why does one batch of cancellations take days to work out?

The trouble is in calculating the refund amount. Orders come in two structures — single, or parent-and-child (split); coupons come in three kinds — percentage, fixed-amount, and single-item discount codes; percentage coupons also carry a discount cap that varies by member tier — over the cap you refund the original price, under it you refund the discounted price. Multiply the three dimensions and every order needs someone to look up its structure, sum the linked orders, check the cap, compute the amount, issue the invoice, write a note, and convert to store credit. A batch takes days, and a miscalculation is a complaint.

CHALLENGE 02

Why are 12 rules never enough?

The requirement arrived in a perfectly orthodox shape: ops wanted a "batch convert to store credit" button in the back office. I started scoping it the orthodox way too — expand every scenario into rules, and landed on 12 refund rules (single × parent × child, each multiplied by no coupon, percentage coupon over/under cap, fixed-amount coupon, discount code). Then things started going wrong:

  • The requirement doc's own example was wrong — checked against its own rules, one example over-refunded.
  • I was wrong twice myself — two gaps I'd flagged turned out, on checking real data, to be my misjudgement, and I withdrew them publicly.
  • Every round of discussion grew new question marks: if the whole set is cancelled, is the coupon returned? How is a coupon split across product lines? What if the parent order is zero and the money sits on the child? Three rounds in, the scenario list kept getting longer and confidence kept getting lower.

Enumeration is a race you're destined to slowly lose. Every meeting adds rules and subtracts confidence — and even if you finish the list, the real world will hand you an order you've never seen the day after launch.

Even with AI helping me scope, the nature of the road hadn't changed: a PM and an AI enumerating together is still enumerating.

THE ROAD I TRIED FIRST ENUMERATE EVERYTHING R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 ? ? ? ? ? ? 12 RULES WRITTEN… AND THE QUESTION MARKS KEPT COMING THE SPEC'S OWN EXAMPLE OVER-REFUNDED I RETRACTED TWO OF MY OWN FINDINGS EVERY REVIEW ROUND FOUND NEW HOLES ENUMERATION IS A GAME YOU LOSE SLOWLY — EACH MEETING ADDS RULES AND SUBTRACTS CONFIDENCE

TURN

The turn: let the scenarios walk in on their own

On the third round I stopped and flipped the problem: rather than spend weeks understanding every special case, let the user confirm with the AI what to do when they actually meet that order.

Traditional development has a shape: scenarios are all settled in the requirements phase and become a pile of if-else in the code — and every new scenario afterwards means changing code, testing, and redeploying. For this project I threw that shape away entirely. So the final decision: no back-office button. Ops drive the AI through conversation, and the AI calls tools to execute. The flow:

  1. Ops give the AI the order numbers to cancel (up to 60 per batch).
  2. The AI looks up each order's structure and discounts, computes the amount by the rules, and returns a preview table: processable / must-stop / out of scope.
  3. Ops confirm once, and the AI runs four steps on each order in sequence (cancel → issue store credit → issue invoice → write the customer-service note) without asking again mid-way.
  4. When the batch finishes it reports a table, four cells per order, listing the ones that need a human.

The key is "must-stop": when an order isn't covered by the rules, the AI doesn't guess, doesn't calculate, doesn't execute — it stops and asks ops to check with customer service first. If customer service's answer is a general rule, the AI opens a PR to write it into the rules, which I review — and the next time that kind of order shows up, it no longer asks.

Requirement confirmation is no longer a "phase" — it becomes a loop. The system doesn't need to be omniscient on launch day. It only needs to be honest, and then get smarter with use.

THE FLIP LET EDGE CASES WALK IN BY THEMSELVES ORDER ARRIVES MATCHES A RULE? YES EXECUTE 4 STEPS REPORT NO AI STOPS ASK A HUMAN GENERAL? → PR HUMAN-REVIEWED → BECOMES A RULE → NEVER ASKS AGAIN REQUIREMENTS STOP BEING A PHASE. THEY BECOME A LOOP.

DESIGN

Hard things in the tool layer, soft things in the rule layer

"Let AI execute refunds" sounds dangerous, so I don't put the money's safety in the rules — I put it in the architecture:

  • Six hard guards in the tool layer that the AI can't get around: only the specified product lines, must be a real paid order, exclude charity and test accounts, no duplicate store credit on the same order, no re-cancelling a cancelled order. These are SQL-level checks — however the AI gets talked into it, they don't pass.
  • The rule layer lives in a skill, changed through GitHub PRs that I review: 12 refund rules, 7 must-stop scenarios, the report format. Changing a rule needs no code change and no deployment — but every change is seen by a person.
  • Every write leaves an audit trail: all four actions are written into the back office's order edit history under the operator's account; access is controlled by an allowlist, and adding or revoking someone takes effect with the next SQL statement, again with no deployment.

The safety of the money is an architecture problem. Judging scenarios is a content problem. Mix the two and you build a system that's neither safe nor flexible.

HARD VS SOFT WHERE EACH KIND OF RULE LIVES TOOL LAYER — AI CANNOT BYPASS 6 HARD GUARDS: RIGHT PRODUCT LINE · PAID & REAL · NOT CHARITY · NOT TEST ACCOUNTS · NO DOUBLE CREDIT · NOT ALREADY CANCELLED SKILL LAYER — HUMAN-REVIEWED, EASY TO EVOLVE 12 REFUND RULES + 7 STOP CONDITIONS + REPORT FORMAT CHANGED BY PULL REQUEST, REVIEWED BY ME — NO REDEPLOY MONEY-SAFETY IS ARCHITECTURE; JUDGMENT IS CONTENT

BUILD

This one I wrote myself

The developer field on this project has my name in it — five tools (look up, cancel, issue store credit, issue invoice, write note), step-by-step guardrails, the permission table, audit writes — the whole thing from rule design to code, with AI as my development partner.

Sign-off wasn't "it runs": 724 unit tests all passing (12 rules × every amount combination, each check written as a test), two end-to-end rounds on the test server across 10 orders with real structures, and the back-office edit history verified line by line. Along the way I also fixed a small bug in existing code — store-credit records weren't writing the creating user.

I BUILT THIS ONE MYSELF PM AS THE DEVELOPER 5 MCP TOOLS 724 UNIT TESTS PASS 2 ROUNDS END-TO-END +1 LEGACY BUG FIXED EVERY WRITE LANDS IN THE ADMIN EDIT LOG WITH THE OPERATOR'S ACCOUNT — FULLY AUDITABLE

IMPACT

The way requirements are asked has changed

The first layer of impact is the numbers: days of manual calculation became minutes of batch execution, and the amounts stopped being wrong.

But I care more about the second layer: ops saw a new shape of requirement with their own eyes. Before, asking engineering for a feature meant the PM had to spell out every scenario in the requirements phase, with weeks of back-and-forth. Now they know there's another way to ask — "build me an AI tool" — where the requirements phase only defines the boundaries (what must never happen, what to do when unsure), and special cases are left to be confirmed in use.

What changed isn't one feature. It's the cost structure of every internal requirement this company raises from here on.

WHAT OPS LEARNED TO ASK FOR THE REQUEST ITSELF CHANGED “BUILD US A BUTTON” EVERY CASE SPECCED UPFRONT WEEKS OF REQUIREMENT PING-PONG “BUILD US AN AI TOOL” DEFINE THE BOUNDARIES, NOT EVERY CASE EDGE CASES RESOLVED AS THEY ARRIVE DAYS OF MANUAL WORK → MINUTES PER BATCH · SPEC EFFORT SPENT ON BOUNDARIES, NOT ENUMERATION

LESSONS

What I learned

Rather than teach the system every scenario, teach it to admit what it doesn't know. A system that honestly says "I'm not sure about this one" is far safer than one that always produces an answer — and it turns launch from "betting we've enumerated everything" into "starting to accumulate".

  • Spend the scoping effort on boundaries, not enumeration. Six hard guards are worth arguing over for three days; the thirteenth refund rule isn't — it will walk in by itself.
  • Withdraw your own misjudgements in public. For the two I got wrong, I edited the requirement doc directly and attached the verification data. That's how the rule layer earns its credibility — and why ops later trusted this system with money.
WHAT I KEEP HONEST BEATS OMNISCIENT SPEND EFFORT ON BOUNDARIES, NOT ENUMERATION RETRACT YOUR OWN ERRORS IN PUBLIC A SYSTEM THAT SAYS “I'M NOT SURE” IS SAFER THAN ONE THAT ALWAYS ANSWERS

STACK

Stack

C# (5 MCP tools mounted on the existing back-office system, reusing the existing cancellation and invoice services), Claude Skill (GitHub version control + PR review), allowlist-based tool permissions (SQL-effective, no deployment), dual audit writes (SQL back-office edit history + Mongo operation log), 724 unit tests.

Internal tool; source not public.

HOW IT WAS BUILT 5 MCP TOOLS ON THE ADMIN SKILL GIT + PR REVIEW PER-TOOL ACCESS LIST FULL AUDIT SQL + MONGO ACCESS CHANGES BY SQL ROW — LIVE IN 60 SECONDS, NO DEPLOY