Guide · Custom builds

How to scope an AI agent pilot.

One team, one workflow, a fixed window and a number to beat. The structure we use before building anything that touches a company's money or customers.

In short
  • Map how the business works before proposing anything, then let the client pick the first bottleneck.
  • Pilot one team and one workflow, on a shared foundation you can extend later.
  • Measure the current process before the build, and write five or six success criteria someone else could check.
  • Agree system access before dates. Quote a target window and say what it depends on.
  • Keep a named person on every decision that moves money or reaches a customer.
  • Make each step a separate agreement the client can stop, and publish the limits with the result.

Most agent projects go wrong before any code is written. The scope is a platform for the whole company, the success measure is a feeling, and nobody has agreed what the agent is allowed to touch. A well-scoped pilot fixes all three. This is the structure we use for custom builds, and the one our own deployments follow.

Why agent pilots fail

The failures we see are rarely about the model. They are about scope:

1. Map the business before you propose anything

Start by mapping the departments and the handoffs between them: who creates each document, who checks it, who approves it, where it is stored and where it waits. In a company with several departments, the expensive problems usually sit in the handoffs. A purchase order, a delivery record and an invoice live in three places, and someone reconciles them by hand.

Then take the map back to the client and ask one question: of everything here, which bottleneck would you most want solved first? Their answer picks the pilot. A good answer usually names a workflow that already has a person checking it, because that check is what the agent will learn to prepare.

Questions for the discovery call

Listen more than you talk. The question about the last dispute matters most: it gives you a real incident, a real cost and a real measure of how long the current process takes when it goes wrong.

2. Pick one team and one workflow

A company-wide platform is a plan. A pilot is one team's real work running through part of it. Choose a workflow that:

Purchase order and invoice checking, booking-to-invoice reconciliation and support triage usually pass. A workflow that touches every department, or one with no existing check to compare against, usually doesn't.

3. Build the foundation once, switch on one service

Most of the work in a custom agent sits underneath the first workflow: bringing documents in, keeping one searchable record, routing approvals and keeping an audit trail. That foundation is shared. Procurement, project and finance services can all sit on it later.

So the pilot builds the foundation properly and switches on one service. The rest of the roadmap stays on paper until the first service has numbers. The pilot stays small, and nothing built in it is thrown away.

4. Measure the current process first

You cannot show an improvement without a baseline. Before the build, record how the team works today:

MeasureHow to get it
Time per itemFrom arrival to approval, using timestamps in the existing system or a timed sample
Rework rateHow many items are sent back or corrected in a month
ExceptionsHow many items raise a question, a mismatch or a dispute in a month
Retrieval timeHow long it takes to answer a question about an item from last quarter
VolumeItems handled per month, so the pilot month is comparable

5. Write success criteria someone else could check

Agree five or six checks that a person outside the project could verify. For a document workflow they look like this:

A criterion that cannot be checked is a hope. Cut it or rewrite it.

Write down what is out of scope as well. If historical records are incomplete, the pilot answers questions about the period it covers. If a harder check is uncertain, such as comparing staged claims for work against the agreed scope, run it as a sampled test inside the pilot and report it separately.

6. Agree access before dates

How long a pilot takes depends on what the agent can read. In the first week, collect:

Then quote a target window, such as six weeks, and state that it depends on access. Dates set before access is confirmed are the usual reason pilots slip.

7. Keep a person on every decision that moves money or reaches a customer

The agent finds the discrepancy, drafts the reply or prepares the change. A named person approves it. Chad works this way: it prepares website changes as a preview link, and nothing goes live until someone approves it. Voice Shield answers every call after hours and escalates emergency and high-intent callers to a person in real time.

Write the approval rules into the pilot: which actions the agent can prepare, who approves each one, and where the approval is recorded. Keep the approval step after the pilot too. It is what lets a finance team trust the output in the first month, and it keeps the audit trail complete.

8. Promise only what the pilot can count

Do not promise fraud detection, perfect answers about historical records, or a delivery date that depends on someone else's systems. Promise what the pilot will count: documents processed, discrepancies flagged, minutes per item, exceptions decided.

Ask the client which value matters most to them: time saved, payments that should not have gone out, or disputes avoided. Report against that one first.

9. Make each step its own agreement

Split the commercials into steps: a scoping session with the client's finance and operations leads, then a fixed-scope pilot with delivery milestones, then a decision. Nothing should oblige the client to take the next step.

Set a go/no-go review on a fixed date. State what happens if they stop: their data, documents and content stay theirs, and are handed back. If the first step is priced low because the hard work is front-loaded, say so, and say where the cost is recovered.

10. Write down what can go wrong

Clients read the risks page first, so make it easy to find. List each risk with the response next to it:

RiskResponse
System access arrives lateThe window moves, and the client knows in week one
Source records are incompleteThe report shows coverage alongside accuracy
Too many false exceptionsThresholds are tuned with the reviewer during the pilot
The team does not use itUsage is a reported number, reviewed weekly
Scope grows mid-pilotNew requests go on the roadmap, not into the pilot

11. Report the result with its limits

The end-of-pilot report covers what went through the workflow, accuracy and time against the baseline, the exceptions raised and how they were decided, what the agent could not do, and a recommendation for the next step.

Every case file we publish carries a limits line, and a pilot report should too. Two examples:

A result with its limits attached is easier to trust, and easier to repeat. For how we read results like these, see how to read the numbers in an AI agent case study.

ZeroLayer builds custom agents on this structure. One team, a month of real work, a person approving each decision, and the limits published with the result.
Book a call
FAQ

Common questions

How long should an AI agent pilot run?

We usually scope one team and one workflow over about six weeks, subject to access to the team's systems. The measure that counts is a full month of real work processed through it.

What should an AI agent pilot measure?

Accuracy, processing time and the amount of manual work removed, each against a baseline taken before the build, plus the number of exceptions routed to a person and decided.

Should the agent replace our existing software?

No. In a pilot the existing systems stay the official record. The agent reads from them, checks against them and routes exceptions for approval. See the guide to keeping your systems as the record.

What should be left out of a pilot?

Anything that cannot be measured in the pilot month: company-wide rollout, promises about fraud, and answers about records the agent has no access to.

Planning an agent build?