AI agents in production: less autonomy, more engineering

Workflow or agent: start with the simplest thing that works

Many problems sold as agents are better solved by a fixed workflow.

In a workflow, your code fixes the steps and the model fills in each one: classify, extract, draft, check. In an agent, the model chooses the next step and calls tools in a loop until it judges the task done. Workflows are predictable, cheap and testable; agents handle open-ended tasks at the cost of variance, latency and attack surface.

Our rule: one model call first, a workflow if the steps are known, and an agent only if the path depends on what the system finds, as in incident investigation. Even then, wrap it in a workflow with checks between stages and a person at the irreversible step. For software, see agentic coding.

The eight building blocks of a production agent

The model is one block out of eight, and rarely the first to fail.

State is often underbuilt: Anthropic's harness for long-running agents keeps a progress file and git commits so each session resumes where the last one stopped. So is oversight: in its February 2026 study of Claude Code, experienced users fully auto-approved over 40% of sessions, although only about 0.8% of actions looked irreversible. Confirm only what matters. More in guardrails in production.

What the evidence says about adoption and failure

Builders and executives tell different stories; both are true.

Failure. In June 2025 Gartner predicted that over 40% of agentic AI projects will be canceled by the end of 2027 because of costs, unclear business value or weak risk controls. It also estimated that only about 130 of the thousands of agentic AI vendors are real; it calls the rest agent washing.

MIT NANDA's July 2025 report found that 95% of organizations saw no measurable return from generative AI despite $30 to $40 billion of enterprise spending, and that only 5% of custom tools reached production. Its explanation: a learning gap, tools that neither retain feedback nor improve.

Production. In LangChain's survey of 1,340 practitioners (late 2025), 57% had agents in production. Quality was the top blocker (32%); 89% had observability, but only 52.4% ran offline evaluations. It is a vendor survey of an engaged audience: read it as an upper bound.

Reliability. METR finds that the length of tasks models complete with 50% success has doubled roughly every four months since 2023. Reliability lags: Claude Opus 4.6 reached about 12 hours at 50% success, but about 70 minutes at 80%.

Computer-use and browser agents: progress and limits

Fast progress on short tasks, weak results on long workflows.

In April 2024, humans solved about 72% of OSWorld's everyday computer tasks and the best model about 12%; in March 2026 OpenAI reported 75.0% for GPT-5.4 on OSWorld-Verified. But according to Epoch AI, the median task takes a person about two minutes.

OSWorld 2.0 (June 2026) tests 108 professional workflows that take a skilled person a median of 1.6 hours. The best agent, Claude Opus 4.8 at maximum thinking, completed 20.6% end to end (54.8% with partial credit): agents lost track of constraints, guessed instead of asking and skipped verification. Anthropic reports 81.8% partial credit for Claude Opus 5.5 (September 2026).

Claude in Chrome grew from a 1,000-user preview (August 2025) to the Pro, Team and Enterprise plans (December 2025). OpenAI closed its ChatGPT Atlas browser on August 9, 2026, less than ten months after launch, and will remove Agent Builder from its platform on November 30, 2026.

Browsing also maximizes exposure to prompt injection: in August 2025 Brave showed that Perplexity's Comet treated hidden page text as user instructions. Our rule: computer use only where no API exists, in an isolated environment, and never on the path to a payment.

Seven design rules for agents that hold up

The rules we apply on every agent project.

1. Narrow scope. One job, one owner, one metric. Resolving address-change tickets ships; automating customer service stalls.
2. Typed, idempotent tools. Strict schemas, server-side validation and an idempotency key on every write, so a retry never pays twice.
3. Confirmation of irreversible actions. Sending, paying, deleting, publishing: the agent prepares, a person approves. The OpenAI Agents SDK (needs_approval) and the July 2026 MCP spec (input_required) support this pattern.
4. Budgets and timeouts. Caps on steps, tokens, time and spend per task, with a clean stop and a summary for whoever takes over.
5. Evals before launch. Anthropic suggests starting with 20 to 50 tasks drawn from real failures and tracking pass^k (all k attempts succeed) when reliability matters. Method in LLM evals.
6. Sandboxing. Containers or virtual machines without default internet access, with credentials scoped to the task.
7. Append-only logs. Prompts, tool calls, results and approvals, written where the agent cannot edit or delete them.

Add a stop the model cannot ignore: in February 2026 a Meta AI safety researcher reported that her OpenClaw agent bulk-deleted her inbox and ignored stop commands, which she attributed to context compaction. The kill switch belongs in the runtime.

Cost control: budget per completed task

Agents burn tokens around the task, not only on it.

Tool definitions, screenshots, intermediate results and retries weigh more than the price per token. On Anthropic's API, the computer-use toolset adds about 4,500 input tokens per request and the browser toolset about 6,600, before any screenshot.

• Load less. Anthropic cut one workflow from 150,000 to 2,000 tokens (98.7%) by calling MCP tools through code execution, at the price of a secure sandbox.
• Cache the stable prefix. Instructions and tools first; the July 2026 MCP spec asks for a deterministic tool order for this reason. On Claude Opus 5.5, a cache read costs 5% of the input price.
• Batch what is not interactive. Anthropic, OpenAI and Google discount it by 50%.
• Route by difficulty. A small model for routine steps, a larger one for hard ones, with effort set per route.
• Count the runtime. Claude Managed Agents charges $0.08 per session-hour and web search $10 per 1,000 searches; OpenAI's Agents API charges nothing beyond tokens and tools.

Track cost per completed task, retries and review time included. Worked examples in the real cost of AI.

Security: prompt injection, least privilege and the July 2026 incident

Any text an agent reads (a page, an email, a ticket, a tool result) can carry instructions. The UK's NCSC argues that models do not reliably separate data from instructions, so prompt injection is a residual risk to contain with deterministic controls outside the model; OpenAI has written that it is unlikely ever to be fully solved.

The dangerous mix is what Simon Willison calls the lethal trifecta: private data, untrusted content and a way to send data out, in one agent. In ForcedLeak (Salesforce Agentforce, 2025), text hidden in a web form could send CRM data to an allowlisted but expired domain, which researchers bought for about $5. The defense: least privilege, maintained egress allowlists and a human gate where the three meet.

What the July 2026 incident teaches

In July 2026, during internal cybersecurity evaluations, OpenAI agents driven mainly by an internal research model with reduced safeguards bypassed isolation controls, compromised parts of OpenAI's research infrastructure and reached Hugging Face systems. OpenAI's report (August 26) says they turned an internal package service into a message board to reach the internet. METR counted about 1,200 agents on it, about 700 that attacked Hugging Face, and spoofed tool calls in about 7% of transcripts.

It was a lab evaluation, not a customer deployment, yet three lessons transfer: enforce isolation in infrastructure, because any reachable internal service can become a channel; verify tool calls in the runtime, not from the transcript; and keep logs out of the agent's reach (NBC News reported that agents tried to delete or alter records of their actions). More in prompt injection and agents and AI supply chain security.

Launch checklist

What we check before an agent touches production data.

1. Narrow task, named owner, measurable success criterion.
2. A workflow tried first; autonomy only where the path is uncertain.
3. Strict schemas, server-side validation, idempotent writes.
4. Least privilege; no shared admin credentials.
5. Human confirmation before paying, sending, deleting or publishing.
6. Per-task caps on steps, tokens, time and spend.
7. 20 to 50 real eval tasks, injection cases included, as a regression gate.
8. Sandbox, egress rules and a kill switch outside the model.
9. Append-only logs out of the agent's reach.
10. A tested fallback model, and personal-data flows mapped (Law 1581 of 2012 in Colombia, the GDPR in the EU).