- Map how the business works before proposing anything, then let the client pick the first bottleneck.
- Pilot one team and one workflow, on a shared foundation you can extend later.
- Measure the current process before the build, and write five or six success criteria someone else could check.
- Agree system access before dates. Quote a target window and say what it depends on.
- Keep a named person on every decision that moves money or reaches a customer.
- Make each step a separate agreement the client can stop, and publish the limits with the result.
Most agent projects go wrong before any code is written. The scope is a platform for the whole company, the success measure is a feeling, and nobody has agreed what the agent is allowed to touch. A well-scoped pilot fixes all three. This is the structure we use for custom builds, and the one our own deployments follow.
Why agent pilots fail
The failures we see are rarely about the model. They are about scope:
- The scope is the whole company, so the first result is months away.
- Success is described as "faster" or "smarter", with no baseline to compare against.
- Nobody agreed which systems the agent can read, so the timeline depends on access that hasn't been granted.
- The agent can act without a person checking, so one wrong action ends the project.
- The proposal promises things nobody can measure, such as catching every fraudulent invoice.
1. Map the business before you propose anything
Start by mapping the departments and the handoffs between them: who creates each document, who checks it, who approves it, where it is stored and where it waits. In a company with several departments, the expensive problems usually sit in the handoffs. A purchase order, a delivery record and an invoice live in three places, and someone reconciles them by hand.
Then take the map back to the client and ask one question: of everything here, which bottleneck would you most want solved first? Their answer picks the pilot. A good answer usually names a workflow that already has a person checking it, because that check is what the agent will learn to prepare.
Questions for the discovery call
- Where do handoffs break between teams, and which problems pull a manager into the process personally?
- Which software runs each step, and where do the documents actually live?
- Who checks the work today, and what happens when two records disagree?
- Tell us about the last dispute or error. How long did it take to reconstruct what happened, and what was at stake?
- Roughly how many items does the team handle a month?
- Who signs off spending on a pilot?
Listen more than you talk. The question about the last dispute matters most: it gives you a real incident, a real cost and a real measure of how long the current process takes when it goes wrong.
2. Pick one team and one workflow
A company-wide platform is a plan. A pilot is one team's real work running through part of it. Choose a workflow that:
- happens often enough to measure in a month,
- runs on documents or records that already exist,
- already has a person checking it,
- stays inside one team, and
- can be switched off without unwinding anything.
Purchase order and invoice checking, booking-to-invoice reconciliation and support triage usually pass. A workflow that touches every department, or one with no existing check to compare against, usually doesn't.
3. Build the foundation once, switch on one service
Most of the work in a custom agent sits underneath the first workflow: bringing documents in, keeping one searchable record, routing approvals and keeping an audit trail. That foundation is shared. Procurement, project and finance services can all sit on it later.
So the pilot builds the foundation properly and switches on one service. The rest of the roadmap stays on paper until the first service has numbers. The pilot stays small, and nothing built in it is thrown away.
4. Measure the current process first
You cannot show an improvement without a baseline. Before the build, record how the team works today:
| Measure | How to get it |
|---|---|
| Time per item | From arrival to approval, using timestamps in the existing system or a timed sample |
| Rework rate | How many items are sent back or corrected in a month |
| Exceptions | How many items raise a question, a mismatch or a dispute in a month |
| Retrieval time | How long it takes to answer a question about an item from last quarter |
| Volume | Items handled per month, so the pilot month is comparable |
5. Write success criteria someone else could check
Agree five or six checks that a person outside the project could verify. For a document workflow they look like this:
- A month of real documents goes through the workflow, not samples.
- Each output is checked against its supporting evidence before a person approves it.
- Each answer links back to the source document it came from.
- Exceptions go to a named reviewer, and the decision is recorded.
- Accuracy, processing time and manual work are measured against the baseline.
A criterion that cannot be checked is a hope. Cut it or rewrite it.
Write down what is out of scope as well. If historical records are incomplete, the pilot answers questions about the period it covers. If a harder check is uncertain, such as comparing staged claims for work against the agreed scope, run it as a sampled test inside the pilot and report it separately.
6. Agree access before dates
How long a pilot takes depends on what the agent can read. In the first week, collect:
- the list of systems, and which ones have an API or an export,
- sample documents of each type,
- monthly volumes,
- who holds the permissions, and
- who approves at each step.
Then quote a target window, such as six weeks, and state that it depends on access. Dates set before access is confirmed are the usual reason pilots slip.
7. Keep a person on every decision that moves money or reaches a customer
The agent finds the discrepancy, drafts the reply or prepares the change. A named person approves it. Chad works this way: it prepares website changes as a preview link, and nothing goes live until someone approves it. Voice Shield answers every call after hours and escalates emergency and high-intent callers to a person in real time.
Write the approval rules into the pilot: which actions the agent can prepare, who approves each one, and where the approval is recorded. Keep the approval step after the pilot too. It is what lets a finance team trust the output in the first month, and it keeps the audit trail complete.
8. Promise only what the pilot can count
Do not promise fraud detection, perfect answers about historical records, or a delivery date that depends on someone else's systems. Promise what the pilot will count: documents processed, discrepancies flagged, minutes per item, exceptions decided.
Ask the client which value matters most to them: time saved, payments that should not have gone out, or disputes avoided. Report against that one first.
9. Make each step its own agreement
Split the commercials into steps: a scoping session with the client's finance and operations leads, then a fixed-scope pilot with delivery milestones, then a decision. Nothing should oblige the client to take the next step.
Set a go/no-go review on a fixed date. State what happens if they stop: their data, documents and content stay theirs, and are handed back. If the first step is priced low because the hard work is front-loaded, say so, and say where the cost is recovered.
10. Write down what can go wrong
Clients read the risks page first, so make it easy to find. List each risk with the response next to it:
| Risk | Response |
|---|---|
| System access arrives late | The window moves, and the client knows in week one |
| Source records are incomplete | The report shows coverage alongside accuracy |
| Too many false exceptions | Thresholds are tuned with the reviewer during the pilot |
| The team does not use it | Usage is a reported number, reviewed weekly |
| Scope grows mid-pilot | New requests go on the roadmap, not into the pilot |
11. Report the result with its limits
The end-of-pilot report covers what went through the workflow, accuracy and time against the baseline, the exceptions raised and how they were decided, what the agent could not do, and a recommendation for the next step.
Every case file we publish carries a limits line, and a pilot report should too. Two examples:
- Voice Shield, six months at a London dental practice: 296 calls answered and 129 bookings, a 44% conversion rate. The limit: one practice over six months, with revenue per booking not reported. See the Envy Smile case file.
- A custom agent inside Cuvama's Jira workflow: Cuvama estimates it ships roughly 40% faster end to end. The limit: that figure is Cuvama's own estimate, not an independent measurement. See the ProductPlease case file.
A result with its limits attached is easier to trust, and easier to repeat. For how we read results like these, see how to read the numbers in an AI agent case study.