How I work
Watch it. Measure it. Then trust it.
Nobody has a secret model any more — you and your competitors can buy the same AI this afternoon. What's actually scarce is judgment about where to use it, and the discipline to prove it works before it touches anything that matters. That's what this page is.
01 · The audit
What does this work actually look like?
Before anything is built, I need to see how the job really gets done. That is almost never what the process document says.
Here is what happens every single time. A team describes a clean five-step process. Then you watch it for a day and find the work arrives from forty different senders in nine different formats, two of those senders get special treatment nobody wrote down, one step exists only because a system from 2011 can't accept a decimal point, and the person who knows all the exceptions is on holiday.
That gap is not a detail to tidy up later. It is the entire reason AI projects fail. A system built for the tidy version meets the real version on day one and starts producing confident, plausible nonsense.
The process document is never what actually happens.Why I start by watching, not building
What you get out of this step
Every branch and exception, with how often each one actually happens — so we automate the common case instead of the interesting one.
Hours, money, and error rate as they stand right now. Without this number, nobody can honestly claim an improvement later — including me.
The expensive step and the annoying step are usually not the same step. Teams tend to point at the annoying one.
Who gets hurt, how quickly you'd notice, and whether it can be undone. This decides how much freedom the system ever gets.
Two weeks of watching is the cheapest insurance you will ever buy against a six-month build of the wrong thing.
02 · The judgment call
Which parts should AI decide — and which absolutely shouldn't?
This single decision separates systems that survive from systems that embarrass you. Most failed projects reached for AI everywhere, and turned problems that had exactly one right answer into problems that now only usually get the right answer.
Every step in your process belongs in one of these four buckets. Put a step in the wrong bucket and no amount of clever engineering will save it later.
| Use | For | Why |
|---|---|---|
| Ordinary softwaredeterministic code | Anything with exactly one right answer — totals, dates, tax rules, whether a field is filled in | There is a correct answer already. Putting AI here can only introduce mistakes into something that was certain. |
| AI judgmentan LLM | Reading messy human writing — sorting it, pulling facts out of it, summarising it | The input is unpredictable and the rules can't be written down in advance. This is the only place AI actually earns its keep. |
| A person decideshuman review | Anything expensive to get wrong, or impossible to undo | The cost of being wrong is higher than the cost of waiting, and somebody has to be accountable for the call. |
| All three togetherhybrid | Most real processes, once you look closely | AI drafts, ordinary software checks the draft, a person approves, the system acts. Each step catches the previous step's mistakes. |
Use AI where the uncertainty is real and valuable. Use plain old software everywhere the answer is already knowable. Anyone selling you AI for the second category is selling you risk.
03 · The test
How will you know it's working?
If you take one thing from this page, take this. The reason most AI projects quietly die is that nobody could ever answer the only question that mattered: how often is it wrong?
The fix is unglamorous. Before building, we write down what “correct” means for this task, and collect real examples from your business where the right answer is already known. Then we can score it honestly — and keep scoring it after launch, when reality starts throwing things at it that we never anticipated.
The industry calls this evals, short for evaluations. It is the difference between a demo and a system.
Real cases where we know the answergolden dataset
Taken from your actual work, never invented. Deliberately stuffed with both the boring majority and the weird exceptions, because the exceptions are what break things in month three.
An agreed definition of “correct”rubric
Written down before anything is tuned, so the target can't move to meet the result. “The demo looked good” is not a standard you can hold anyone to — including me.
Every way it breaks, named and countedfailure taxonomy
A single accuracy score hides which problem is actually hurting you. Knowing it's “92% accurate” tells you nothing. Knowing that most of the 8% is one specific document type tells you exactly what to fix on Monday.
What each run costs, and how long it takescost & latency budget
Measured at the volume you actually run. Plenty of systems are accurate and still not worth it, and finding that out in week two is enormously cheaper than finding it out in month six.
Ask any AI vendor how often their system is wrong, and what happens when it is. The good ones have a document ready. The rest change the subject.
04 · Earning trust
How does it earn its way in?
Trust is granted one step at a time, and every step has to be paid for with evidence. Nothing I build starts by touching your live process on day one.
Think of it as a new hire. They shadow someone, then draft work for review, then work with a check, then work alone — and only because they earned each step. Software should be held to the same standard.
The middle three get confused constantly, because they differ on two things rather than one. Going from shadow to suggests changes who can see the answer. Going from suggests to approves changes who does the work — and flips the default, because below that line nothing happens unless a person acts, and above it things happen unless a person intervenes. The last column is the fastest way to tell them apart.
| Stage | What it's allowed to do | What it has to prove first | If it gets it wrong |
|---|---|---|---|
| Practises on old workoffline replay · level 0 | Runs against work that already happened, where the right answer is already known. It never sees anything live. | We have real examples with known answers, and we agree what correct means. | Nothing. It is a rehearsal — that is what the rehearsal is for. |
| Shadows the real work, unseenshadow mode · level 1 | Sees the same live work your team does and records what it would have done — but shows that answer to nobody. Your team works exactly as if it were not there. | It scored well enough on the practice cases. | Nothing, and nobody is misled — because nobody saw it. That is precisely why the agreement figure it produces can be trusted. |
| Suggests — your team still does the workassist · level 2 | Now its answer is visible. Your team sees a draft before they start, and is free to use it, change it, or ignore it completely. They still do the job. | Its shadow-mode answers matched your team's on real work, not just practice cases. | Someone loses a few seconds reading a bad suggestion and discards it. The work itself is unaffected, because a person was always going to do it. |
| Does the work — a person approves itsupervised autonomy · level 3 | The system does the work now, not your team. Someone reviews and signs off before anything lands. The default has flipped: things happen unless a person stops them. | Your team stopped correcting the drafts, and approving became the slow part. | It reaches a real customer if the reviewer misses it. This is the biggest jump on the ladder — the first rung where a human being tired costs you something. |
| Acts alone, inside limitsautonomous · level 4 | Acts without waiting for anyone, inside boundaries that cap how much a single mistake can cost. Checked afterwards by sampling rather than beforehand. | A long track record, plus a shut-off switch that has actually been tested. | It already happened. You find out from the sample, a complaint, or the monitoring — which is why this rung needs a tested way to switch it off, not just a plan for one. |
Hiding the answer in shadow mode isn't timidity. The moment a suggestion is visible, the person deciding is influenced by it — so you can no longer tell whether they'd have agreed on their own, which is the exact measurement the next step is supposed to be bought with.
At every stage there is a way to see exactly what it did and undo it. A system nobody can inspect or switch off is not automation, it's exposure.
There is one way for this to go right and about a thousand ways for it to go wrong.Which is where most of the work goes
The version that works on a good day takes an afternoon. The rest of the engagement is everything else: input arriving malformed, a supplier timing out halfway through, an answer that looks perfectly structured but is quietly wrong, confident nonsense about a case the system has never seen, the same request arriving twice and being paid twice.
That is where systems die in production, so that's where the time goes — and it's the part that never makes it into a demo.
05 · What you keep
Everything gets handed over.
The point is that your team can run this without me. If the only person who understands the system is the person you hired to build it, you haven't bought anything — you've started paying rent.
This is also a checklist you can hold any AI vendor to, not just me. Ask which of these they will hand over, and watch how specific the answer gets.
| Stage | What you get |
|---|---|
| DiscoveryUnderstanding how the work really happens today |
|
| ArchitectureDeciding what gets built, and who is allowed to do what |
|
| EvalsProving it works, with numbers instead of impressions |
|
| DeploymentTurning it on carefully, and being able to turn it off |
|