Why do most AI pilots end without a decision?
Most AI pilots end without a decision because no one recorded what normal looked like before they started. Without a baseline there is nothing to compare against, so the conversation at the end turns on impressions, and impressions favour whoever ran the pilot.
The second common cause is scope creep. A pilot that starts on one workflow and grows to four produces results nobody can attribute, which leads to an extension rather than a conclusion.
What should a pilot cover?
A pilot should cover one function, one property, and one measurable target. A property here means a specific website, store, ad account, or ledger rather than the whole estate, so results are attributable and mistakes are contained.
Choose the function where coordination overhead is highest relative to judgement, because that is where the effect shows fastest. Resist adding a second function even when the first goes well; the comparison is the asset you are building.
What baseline should you record?
Record three things before the first run: the output volume of the function, the elapsed time from request to delivery, and the hours the responsible people currently spend. The third is the one teams skip and the one that usually moves first.
Write them down in a shared document with a date. Memory of the previous state degrades quickly once the new state exists, and an undocumented baseline becomes whatever the most confident person in the room remembers.
- Output volume: how much does this function currently produce per week?
- Cycle time: how long from request to delivered work?
- Human hours: how much time do the responsible people spend?
- Quality bar: what does an acceptable output look like, in writing?
How long should a pilot run?
Six to eight weeks is usually enough for a continuous function, because it covers the ramp and at least a month of steady operation. Shorter than four weeks and you are measuring onboarding rather than performance.
For a function with a monthly cycle, such as reconciliation and close, run across two full cycles. One cycle tells you whether it worked once; two tell you whether the second was easier than the first, which is the question that actually matters.
What is the decision rule?
Write the decision rule before the pilot starts, including what a no looks like. A rule states the threshold on the target metric, the quality bar, and the review-time budget, and it commits both sides to acting on the result.
The review-time clause is the one most often omitted and the one that most often decides the outcome. A pilot that hits its output target while consuming more management time than it saved has not succeeded, and only an explicit rule catches that honestly.
What should you do at the end?
At the end, compare against the recorded baseline, apply the decision rule, and write down what you learned about the operating model regardless of the outcome. The operating knowledge is valuable even when the answer is no.
If the answer is yes, record the one-page model before expanding: owner, remit, targets, approval routing, access scope, autonomy level, and review rhythm. That page is what makes the second role a fast decision rather than a repeat of the first project.
What should the pilot's weekly rhythm look like?
The pilot's weekly rhythm is short and fixed. Every morning the manager reads the Digital FTE's brief in about five minutes, approves or sends back the waiting proposals with reasons, and decides any escalations. Every week the manager reads the send-back reasons grouped by category and adjusts the remit where the same reason repeats, because a repeated send-back is a rule the remit is missing rather than a failing of the FTE. Nothing is promoted on the autonomy ladder in the first cycle.
At the end of each cycle the manager reads the FTE's own report against the targets set at the start and checks it against the evidence in Targets, then records three numbers alongside the baseline: outcomes delivered, approval rate, and tokens consumed, plus the human hours spent. The rhythm matters because it is what the pilot is testing. A pilot run by reading outputs when there is time produces no record and no decision; one run on this rhythm produces both, and shows the manager what running the role for real would ask of them.