AI agents: how to design one that does not fail in production

Most agents that fail in production do not fail because of the model. They fail because nobody defined where their scope ends, what each tool returns when something goes wrong and how much they can spend before giving up. This is a design guide, not a list of frameworks.

What an agent is and what it is not

An agent is a loop: the model chooses an action, executes it, observes the result and decides the next step. Besides the loop, it needs two things that almost nobody documents: tools with a contract and explicit state.

If you already know the sequence of steps in advance, you do not have an agent. You have a workflow. Classifying a ticket, extracting data from an invoice or generating a weekly summary are better solved with a fixed pipeline: input, transformation, output. It is cheaper, faster and easier to test.

The "agent" label is applied today to prompt chains that decide nothing. A disguised workflow only adds variability.

State is the forgotten component

Without an explicit state object, the conversation history becomes the state. That works in a demo and breaks when the session lasts twenty turns: the agent repeats steps, loses the thread or drags along an old error.

Define the state as a contract: what data it needs, what it has already tried, which tools failed and where it stands. If you can serialise it, you can resume a run, compare two versions and debug without guessing.

When not to use an agent

An agent is justified when three conditions are met: the path is not known in advance, the tools return variable results and an intermediate error can be detected and corrected.

If one of the three is missing, use something simpler. In support, most cases are classification and a templated reply. In sales, rule-based qualification and a summary. In operations, validating data against a schema. None of them needs an open loop.

The real cost of an agent is not tokens. It is the debugging surface. Every free decision adds branches you must test, and the branches you do not test are the ones that show up in production on a Friday.

Signs that you do not need one:

• Can you list the steps in advance? Then it is a workflow.
• Is the answer already in a document or a table? Then it is search or a query.
• Is an intermediate error irreversible? Then you need human confirmation, not more autonomy.

Design tools, not prompts

The model chooses tools by reading their name and description. In practice, when in doubt, that description weighs more than the system prompt. Treat it as part of the product.

A tool should have a single intent, a closed input schema and an output that fits in the context. manage_customer is not a tool: it is four tools with different permissions.

• Separate reads from writes. Reads can be parallelised and retried; writes need an idempotency key.
• Return partial success. "I found the order but not the shipment" is more useful than a generic error.
• Write errors that point to the next step: "no results for that email, try the order number".
• Limit the size of the response. A tool that returns forty pages of JSON pollutes the context and hides what matters.

If a new person could not use the tool from its description alone, neither can the model.

Errors, retries and dead states

Not every error deserves a retry. Mixing them up is the most common cause of expensive loops.

• Transient: timeout, rate limit, service down. Retry with increasing backoff and a maximum of two or three attempts.
• Deterministic: invalid data, permission denied, resource not found. Retrying with the same input gives the same result. Fix the input or escalate.
• Semantic: the tool responds fine but with the wrong data. No retry fixes it; you need a cross-check or a person.

Detect loops with a simple rule: same tool with the same arguments twice in a row, stop. And define terminal states from day one: resolved, escalated, abandoned. An agent with no defined end does not finish: it consumes budget until someone switches it off.

When you escalate to a person, hand over the context, not the transcript: what it tried, what failed and what decision is missing. An escalation with a summary is resolved in a minute; one with twenty turns of history, in ten.

Cost and latency as design constraints

The budget is defined per task, not per month. Maximum steps, maximum time, tool calls and cost per run. If the agent exceeds it, it stops and escalates. That limit is not an infrastructure detail: it is part of the behaviour.

Latency is also a product decision. In a support chat, forty seconds is worse than an incomplete answer in four, because the user leaves. Design the fast path first: a direct answer for frequent cases, the agent only for the remainder that does not fit.

For cost, measure per resolved case, not per call. An agent that spends three times as much but resolves without intervention can be cheaper than a cheap one that escalates half the tickets. That comparison, against the current human cost, is the only one that matters.

Evaluate with real cases

A demo is evaluated with five examples chosen by whoever built it. An agent is evaluated with fifty cases chosen by the business, including the ugly ones: an ambiguous request, a duplicate customer, missing data, an angry user.

Each case needs three things: real input, an expected result and an acceptable trajectory range. Not just whether it got it right, but how it got there.

Metrics worth tracking:

• Task success. Whether it resolved without intervention. Check it on every prompt or tool change.
• Tool accuracy. Whether it called the right tool with the right arguments. Check it when adding or renaming tools.
• Steps per case. Whether the path is reasonable or goes round in circles. Check it weekly.
• Escalation rate. How much work it hands back to people. Check it daily.
• Cost per resolved case. Economic viability. Check it monthly.
• p95 latency. The real experience, not the average. Check it weekly.

Keep the cases as a regression suite. Before changing a tool description, run it. Most regressions do not come from a new model: they come from a tool someone "improved" without telling anyone. Also include cases where the right thing is not to answer or to escalate.

Observability: traces before dashboards

The log of the final answer is useless for debugging. You need the full trace: input, plan, every call with its arguments, the result, the tokens, the latency and the final state.

Tie the trace to a business identifier: ticket, order, customer. That way, when someone asks why that answer was given, you do not search by timestamp.

• Redact personal data before storing. The trace is not an excuse to hoard conversations.
• Sample full traces and aggregate metrics. Storing everything without sampling becomes expensive and slow to query.
• Alert on escalations, detected loops, tool error rate and cost per resolved case.
• Replaying a trace against a new version is the most cost-effective debugging tool you can build.

An agent is defined not by what it can do, but by what it refuses to do.

8-point checklist before deploying

1. Scope in one sentence, with a list of out-of-scope cases and what to answer in each.
2. Tools with a closed schema, minimum permissions and errors that point to the next step.
3. Serialisable state and defined terminal states: resolved, escalated, abandoned.
4. Maximum budget for steps, time and cost per run, with an automatic cut-off.
5. Retries only for transient errors, with a cap, increasing backoff and loop detection.
6. At least fifty real cases with success criteria and a runnable regression suite.
7. Traces with a business identifier, redaction of personal data and the ability to replay.
8. A business metric per resolved case: cost, latency and escalation rate compared with the current process.

If you cannot complete six of these eight points, you do not have an agent ready for production. You have a demo with good manners.

Keep reading

• Your first AI automation without coding: start with a fixed workflow before an agent.
• Prompt injection: the risk of AI obeying someone else: the main security risk for agents with tools.
• RAG without the hype: when you need vector search and when you don't: when an agent needs to search your documents.
• How to choose an AI model without looking at the leaderboard: evaluating with your own cases.