Piloting AI inside an existing business system

A bounded AI pilot is the controlled way to introduce and evaluate an AI capability inside a system that already exists. The pilot is defined before the technology: a narrow operational problem, an explicit system boundary, a limited AI role, permitted inputs and outputs, an authority set that stops short of execution where consequences demand it, observable evaluation criteria, and outcomes that can be reversed. It runs for a defined period, is measured against the criteria set in advance, and ends with one of five decisions: expand, keep bounded, redesign, defer, or remove the AI capability entirely. Successful adoption is one possible outcome, not the required one — the pilot exists to generate evidence for the next system decision.

Define the operational problem before any AI capability, model, or tooling is considered. The problem is stated in the language of the operation: a task that is slow, inconsistent, or hard to staff, a step where language or pattern work exceeds what rules can do, or an output that people currently produce by hand. The problem statement names the task, the data it touches, the people involved, and what would count as an improvement. If the problem cannot be stated without naming a model or a product, it is not yet a problem statement. What a coherent business system is — and how the host system of the pilot fits that picture — is covered by the business-systems guide (/resources/what-business-systems-include); the pilot starts from the concrete task, not from the definition.

The pilot names where the AI capability lives inside the existing system and what remains outside it. The boundary lists the systems the capability may read from, the one place where its output enters the operation, and everything it must not touch: other systems, other data, other processes, and other decision points. The rest of the existing system keeps operating exactly as it does today — the pilot does not redraw the system, does not replace a workflow, and does not require other teams to change how they work. This is deliberately narrower than the coherent-system work described in the business-systems guide (/resources/what-business-systems-include): that guide defines the operating whole, while the pilot claims only a bounded corner of it.

The AI capability in a pilot performs one narrow kind of work: interpreting, classifying, generating, extracting, or assisting — not the whole task and not the whole system. The role is described as a job, not as a technology: reading incoming requests and extracting the case data, rather than an endpoint or a model name. Whatever does not fit that narrow role stays in the existing system, with the existing tools and the existing people. The pilot must not let the AI capability implicitly become the system: it is a bounded component with a named job, surrounded by a system that already works without it.

The pilot runs on a defined set of inputs and no more. Name the source of truth for each input — the system and record it comes from — and the permitted context the capability may see: which fields, which records, which documents, and what is explicitly excluded. If the pilot retrieves context instead of receiving it, the retrieval boundary is part of the design: what may be searched, what may not, and how the selected context is bounded before it reaches the model. Deterministic inputs the task already has — statuses, identifiers, reference data — stay deterministic and are passed alongside the AI work rather than regenerated.

Capability is not authorization. A capability that can generate an answer is not thereby allowed to commit it, and a capability that can recommend an action is not thereby allowed to execute it. The pilot defines what the AI may produce — classifications, extractions, drafts, summaries, recommendations, structured results — and what it may not do: write to records, trigger workflows, send communication, or take any action that changes the operation, unless that specific authority was granted for that specific task. Where the pilot's output is decision support, the decision stays with the person; where the output is material for a step, the step itself remains where it already lives.

The pilot needs exactly one bounded integration point: a defined contract where its structured output enters the existing system, and a defined fallback when the output cannot be produced. The contract is minimal and explicit — the output schema, the expected values, the format, and what the receiving step does with the result. The fallback is part of the same contract: when the capability fails, times out, or produces output that fails validation, the work continues on the existing path, handled by the person or the process that did the task before the pilot. This single integration point is the pilot's only claim on the existing system; everything else stays untouched.

Human control in a pilot follows the consequence of each action the capability influences, not a universal rule that every output must be approved. Where the pilot's output feeds a step with high or irreversible consequences — external communication, record changes with legal or financial weight, decisions that commit the business — a person reviews or executes. Where the output is low-consequence and easily corrected — a draft, a classification for triage, a summary for internal review — the pilot can flow directly and still be reversible. The point of the pilot is to learn where the consequence line actually sits for this task and to set the control points accordingly. The full catalogue of control categories for AI systems is the subject of the practical guardrails note (/resources/practical-guardrails-for-ai-systems); the pilot applies the subset its consequences demand.

The pilot should leave enough of a trail that a later review can reconstruct what happened: the inputs each request received, the context that was actually used, the output that was produced, the validation and fallback decisions, the human reviews that were involved, and the exceptions and failures that occurred. Observability is not about logging everything — it is about making the evaluation criteria answerable. If the pilot's outputs are later judged, the material for that judgement must exist. What must be visible is decided per signal: inputs and outputs for usefulness, exceptions and failures for error modes, corrections for human burden, and the full trail for auditability.

Operating cost and latency are first-class constraints in a pilot, not afterthoughts. The design sets a token or context budget for the task, an upper bound on what each run may consume, and a latency budget for the step the capability serves. These are operational parameters the team sets for its own task and environment — there are no universal numbers, and none should be borrowed from elsewhere. What matters is that the pilot names its budgets in advance and records what it actually consumed, so the evaluation can weigh the cost and latency of the capability against the existing path it would replace.

The pilot defines how the capability fails before it fails. Timeouts bound how long a run may take, retry boundaries bound how often it may be tried, and validation catches output that is structurally unusable. The fallback is the existing path: when the capability cannot produce an acceptable result within its boundaries, the task continues the way it did before the pilot. For most pilot tasks a person performs the step directly; where a deterministic path exists, that path serves as the fallback instead. A pilot that fails silently is not a pilot — every failure is visible and leaves the operation either unaffected or better informed.

Task suitability: whether the narrow role defined in the pilot actually fits the operational problem, or whether the problem was misidentified.

Output usefulness: whether the outputs are usable in practice, not only in a test set.

Error modes: the patterns in where and how the capability fails.

Human-correction burden: how much review and correction the outputs actually require.

Workflow fit: whether the integration point and the fallback fit how the work really runs.

Integration reliability: whether the bounded integration contract behaves as designed.

Latency: whether the step completes within the latency budget the operation needs.

Operating cost: what the capability actually consumes against the budget set in advance.

Fallback behavior: whether failures route to the existing path cleanly and visibly.

Auditability: whether the recorded trail is sufficient to reconstruct what happened.

Reversibility: whether every outcome of the pilot can be rolled back without residue.

Expansion readiness: what the pilot learned about whether, and how, a broader capability would fit the system.

The pilot evaluation does not need a universal score or a fixed pass threshold. It needs evidence against the criteria named before the pilot started, gathered from the operation itself.

Expand: the evidence supports moving the capability to a wider scope — a larger slice of the task, more inputs, more authority — as its own next bounded step.

Keep bounded: the capability is useful exactly where it is, at its current scope, and staying there is a legitimate outcome.

Redesign: the task or the boundary was wrong, and the pilot should be reshaped before anything is judged.

Defer: the problem is real, but the evidence says the conditions — data, timing, integration, cost — are not right yet.

Remove AI entirely: the honest conclusion is that the existing path is better, and the AI capability should be taken out.

The pilot exists to generate evidence for the next system decision, not to justify AI adoption. Removing the capability is a successful pilot if that is what the evidence supports.