Vora
Write the buying question first
For a hypothetical after-hours receptionist pilot, the question could be whether structured callback requests are useful enough to justify a recurring intake service. That is narrower than asking whether AI works for the business.
Specify the service the client would buy after the pilot. In this example, the agent collects approved details and creates a request for staff review. It does not confirm appointments or give binding estimates.
Use the first-client guide if that boundary is still unclear. A pilot cannot settle a commercial question when the proposed service keeps changing.
Ask who has authority to make the buying decision and who will judge the working result. Those may be different people. The owner may approve spending while an office manager checks whether requests are usable.
Include both in the plan. A technical review with someone who cannot approve the next step can leave the commercial decision unresolved even when the evidence is complete.
Agree on the evidence and its limits
Separate technical acceptance from business usefulness. Technical acceptance might require the final callback number to appear in the staff queue. Business usefulness might depend on staff using those requests and spending less time reconstructing messages.
The GOV.UK Service Manual's performance guidance recommends choosing metrics from the service's purpose and the benefits you expect. Use that approach to avoid measuring call counts that cannot answer the buyer's question.
For this hypothetical pilot, agree on a small evidence set:
| Question | Observation to record | Who checks it | Limitation to keep visible |
|---|---|---|---|
| Did eligible calls produce usable requests? | Eligible calls, saved requests and incomplete requests | Agency and office manager | Define eligible before counting |
| Were details accurate enough for staff? | Reviewed requests and corrections required | Office manager | Review all requests or label the sample used |
| Did exceptions reach a person? | Failed submissions and observed handoffs | Agency and receiving staff | A sent alert does not prove staff acted |
| How much work did the service require? | Agency support time and staff correction time | Each party records its own work | Estimate separately when time was not recorded |
| Is a recurring purchase justified? | Evidence against the buying question and quoted terms | Authorized buyer | A short period may leave demand uncertain |
Write the agreed decision rule
Record these rules beside the observations before starting. Let the client agree any numerical tolerance; a template cannot supply one for their business.
| Decision part | Agreement to write |
|---|---|
| Mandatory technical cases | List required cases, pass criteria and accepting owner. Require those cases to pass before handover. |
| Staff usability | Name the reviewer, review set and usable-request definition. Agree any allowed correction tolerance in advance. |
| Pause conditions | Name the failures that pause the pilot, stop authority and fallback owner. Keep unresolved pause conditions visible. |
| Recurring purchase | The authorized buyer decides whether the demonstrated service justifies its recurring terms. Technical acceptance alone does not approve the price. |
| Missing evidence | Name what remains unknown. A funded extension needs specified work, a fee, an end date and a new decision rule. |
A usable request needs a written definition. For example, it contains the final callback number, reason for calling and necessary location detail, and staff can find it in the agreed queue. Define what counts as incomplete and whether a request requiring one correction remains usable.
If only some requests receive review, record the selection rule and sample size. Avoid reporting the sample's outcome as though someone inspected every call.
Record the starting process
Before the pilot, inspect how the business handles comparable enquiries. Record the volume, missing details and staff work where the evidence exists. Use the same definitions that you will use during the pilot.
Historical records may be incomplete. Say which baseline observations are available and which remain unknown. You can still inspect whether the new workflow saves accurate requests, but you cannot confidently claim a reduction from a starting value you never measured.
Keep changes in context beside the results. A promotion, holiday closure or altered staff coverage can change the types and number of calls. If you compare two periods, explain those differences and avoid attributing all movement to the pilot.
Revenue needs a separate chain of evidence. A saved callback request is not a sale. If the business wants to evaluate sales, agree how staff will connect each request with its later outcome and what attribution limits apply. A short intake pilot may not be able to answer that question.
Bound the pilot fee and work
Quote the pilot as a defined engagement with setup, controlled tests, limited use, support and a review. Specify what access, approved information and staff participation the client must provide.
The fee should account for the work the agency expects to do. The service-cost guide separates setup, recurring delivery and usage so those costs remain visible.
State whether any pilot payment is credited toward later setup. State the likely recurring fee or pricing method, allowance and support scope. If those terms may change after usage is measured, identify what can change and how the revised quote will be decided.
Without that information, the buyer may like the result and still reject the price at the final meeting. The pilot has tested a service whose purchase terms arrived too late.
Choose duration and volume conditions for the question you need to answer. Three weeks is a hypothetical period here, not a universal recommendation. A quiet business may need longer to encounter the relevant calls. Extending the calendar does not guarantee that a rare failure or seasonal use case will appear.
Prove the technical path before admitting ordinary callers
Run controlled tests with synthetic details. Inspect saved results through the staff view and test corrections, incomplete details and the agreed failure path.
Voice agent workflow testing gives examples of checking what happened after the call. A mocked successful response can help test conversation behavior; the actual destination still needs a controlled integration check.
NIST's AI Risk Management Framework Core describes decisions about whether development or deployment should proceed, supported by mapped and measured risks. A pilot plan should likewise allow the team to stop when a required condition fails.
Write the pause conditions before starting. These could include missing records, incorrect promises, an inaccessible exception queue or a dependency the team cannot recover. Agree who can pause the service and what handles incoming work while it is paused.
A pilot is still a service people may use. The client needs a working fallback and a responsible owner during the evaluation period.
If the pilot replaces an existing workflow, agree on cutover and rollback separately. The migration checklist covers comparing records, preserving a fallback and checking dependencies. A pilot's short duration does not remove that transition work.
Keep the question stable while the pilot runs
Record defects and proposed additions separately. Fixing an accepted behavior is different from expanding the service to another location or introducing direct booking.
If the client requests an addition, assess it using scope-change guide. Decide whether it belongs in a later phase, a revised pilot or a separately priced experiment. Record the version and dates for any accepted change so the review can distinguish results from different configurations.
Keep a short evidence review during the pilot. Inspect exceptions while they can still be reproduced, rather than waiting for the final meeting. Update the buyer when a finding changes the delivery plan or the likelihood of acceptance.
Avoid repairing the data after the fact without a record. If staff fix a callback number manually, count the correction. It tells the buyer something about the actual service work.
Make the final meeting a decision
At the review of a hypothetical three-week AI receptionist pilot, the owner likes the calls and staff say the requests seem useful. Nobody knows how many requests reached the office queue, how many needed correction or what the recurring service would cost. The discussion ends with another trial period.
Bring the agreed buying question, acceptance results, observed usage, support work and proposed recurring terms. Show incomplete observations alongside the findings.
If the service passes technical acceptance and the buyer sees enough value at the proposed price, document the purchase and handover. If it fails a required condition, identify the repair needed before another decision. If the buyer no longer wants the service, close the pilot and follow the agreed offboarding process.
An extension should name the missing evidence, the change that will produce it, the additional fee and the next decision date. Another month of the same process may leave the same question unanswered.
Before the pilot starts, put those decision options into the review invitation. Add the person who can approve the recurring service and the staff member who will bring the saved-request evidence.