From demo to an agent you can let act
Most agent projects reach the same point. A prototype, built in a few days, calls a handful of tools and completes the task in a demo. Then someone asks what happens when it is given a malformed record, when the API it depends on is slow, when a retrieved document contains instructions, or when it decides to update a thousand rows instead of one. The prototype has no answer, because it was never designed to have one.
The work on this page is building the parts around the model that make an agent safe to run: the runtime, the tool boundary, the permissions, the approvals, the evaluation, the tracing and the recovery paths. The model is one component. Most production failures I have seen come from the parts around it.
What the work involves
I build agents on a principle published as the Substrate Pattern: the agent proposes, and a deterministic layer underneath decides what is allowed to happen. In practice that means:
- Typed tools. Each tool has a schema for its inputs and outputs, and calls that do not validate are rejected before they reach your systems.
- Permissions outside the model. The agent acts with credentials scoped to the task and to the user it is acting for. A prompt cannot widen them.
- Approval steps. Actions that are expensive, irreversible or customer-facing pause for a human, with the proposed action and its reasoning shown in plain language.
- Idempotency and compensation. Retries do not duplicate side effects, and actions that partly complete can be rolled back or completed safely.
- Tracing. Every run records the inputs, model calls, tool calls, approvals and outcome, so an incident can be reconstructed afterwards.
Evaluation runs alongside the build. I assemble task examples with the people who do the work today, add adversarial cases, and wire the suite into CI so that a prompt or model change cannot ship without being measured.
The signature deliverable
You receive agent implementation, controls and recovery engineering: the agent code, the action register, the evaluation suite and a runbook. Illustrative example of an action register extract:
| Tool | Effect | Credential scope | Approval | Recovery if it fails midway |
|---|---|---|---|---|
lookup_order | Read | Orders API, read-only, current customer | None | Retry with backoff |
issue_refund | Write, financial | Payments API, refund-only, capped value | Human approval above agreed threshold | Idempotency key; reconcile against payment log |
send_customer_email | External message | Email service, templated sends only | Approval for first release, sampled afterwards | Queue holds message until confirmed |
update_crm_note | Write | CRM, notes field only | None | Overwrite-safe append |
Illustrative example. Not taken from a client engagement.
How acceptance is judged
Acceptance criteria are agreed before the build: the task set, the success threshold on the evaluation suite, the actions that must be blocked or approved, and the recovery scenarios that must pass in a rehearsal. Your platform lead and the risk owner for each consequential action sign off. The evaluation report is part of the deliverable, including the cases the agent still gets wrong.
Ownership and handover
The code runs in your repositories and infrastructure from the first commit. At handover your named owner receives a walk-through of the runtime, the action register and the evaluation suite, plus a runbook covering alerts, the kill switch and common failure modes. If you want help keeping it healthy afterwards, maintaining production AI workflows is a bounded arrangement with a stated capacity, not an open-ended support promise.
Narrower routes
This page covers the full build. If your need is narrower, one of the routes in the comparison above will fit better: MCP integration for connecting an existing agent to your tools, governed AI agents where approval and audit obligations lead the design, and reliability and recovery engineering for an agent that already runs but fails unpredictably. If you want an engineer under your own manager instead, see hire an AI agent engineer.