AI Agents & Orchestration

First, do not build an agent

The most valuable judgement here is knowing when not to.

An agent is an LLM in a loop with tools, deciding its own path. That autonomy is the point and the cost — it is slower, dearer, non-deterministic, and hard to debug.

Four questions, and a no to any of them means build something simpler:

1. Is the path genuinely unknown ahead of time? If you can enumerate the steps, encode them. A fixed workflow with an LLM at each node is cheaper, faster, testable and debuggable.
2. Does the value justify the cost? Agents run many model calls per task.
3. Is the model actually capable here? Test the hard cases before committing to the architecture.
4. Are errors recoverable? Can you detect, roll back, review?

The commonest expensive mistake at this level is an agent doing work a three-step pipeline would have done better.

The loop, and its economics

history = [system_prompt, user_request]

for turn in range(MAX_TURNS):
    response = model(history, tools)

    if response.stop_reason != "tool_use":
        return response.text

    results = []
    for call in response.tool_calls:          # may be several — run concurrently
        if requires_approval(call):
            if not await human_approval(call):
                results.append(rejection(call))
                continue
        results.append(execute(validated(call)))

    history.append(response)
    history.append(results)                   # all results in ONE message

raise TurnLimitExceeded()

Four details in that sketch are load-bearing:

Parallel tool calls belong in one message. A model may request several tools at once. Execute them concurrently and return all results in a single message. Splitting them across messages teaches the model, turn by turn, to stop requesting parallel calls.

Failed calls must still return a result. Every request needs a matching response, flagged as an error. A missing result corrupts the conversation structure.

Validation sits between the model and execution. The model produces arguments; it does not produce trustworthy arguments.

The turn cap is not optional. It is the outermost guard against a loop that never terminates.

Why cost grows superlinearly

The API is stateless. Every turn resends the entire accumulated history — which grows with every turn.

Token spend therefore scales roughly with the square of turn count. An agent taking eight turns instead of five costs far more than 60% extra.

The practical consequence: reducing turns is worth more than reducing per-token price. Better tool design that lets the model finish in three turns beats a cheaper model that needs seven.

Tool design

Tools are the agent’s entire interface to the world. This is where most agent quality lives.

Descriptions are the API

The model reads descriptions to decide. A description states four things: what it does, when to use it, what it returns, and what it is not for.

{
  "name": "search_orders",
  "description": "Search a customer's orders by date range or status. Returns up to 20 orders with id, date, status, total in GBP, and items. Use when the user asks about past purchases, deliveries, or returns. Does NOT return payment details — use get_payment_method. Does NOT modify anything.",
  "input_schema": {
    "type": "object",
    "properties": {
      "customer_id": { "type": "string", "description": "Internal customer ID, format CUS-XXXXXX" },
      "status": { "type": "string", "enum": ["pending", "shipped", "delivered", "returned"] },
      "since": { "type": "string", "description": "ISO 8601 date. Omit for all history." }
    },
    "required": ["customer_id"],
    "additionalProperties": false
  }
}

The negative clauses prevent a whole class of wrong-tool selection. The enum removes an entire category of invalid arguments. And where your provider supports strict schema enforcement, enable it — it makes structurally invalid arguments impossible rather than merely unlikely.

Errors are instructions

The model can only act on what it reads. Compare:

✗ ValueError: invalid literal for int() with base 10: 'CUS-4521'
✗ {"error": true}
✓ "customer_id must be numeric, e.g. 4521, not the CUS- prefixed form.
   Call lookup_customer with the prefixed ID to get the numeric one."

The third tells the model exactly what to do next. This single habit removes more retry loops than any other change.

Keep the surface small

Selection accuracy degrades as the tool list grows. Ten well-scoped tools beat forty overlapping ones. If you genuinely need many, look at deferred loading or tool search, where the model retrieves relevant tool definitions rather than holding all of them in context.

Prefer few general tools over many specific ones. One query_database with a constrained schema usually beats fifteen narrow getters — fewer selection decisions, fewer descriptions competing for attention.

Connecting tools has begun to standardise around the Model Context Protocol, which lets a tool written once be exposed to different agent hosts. Worth adopting for anything you expect to reuse.

Memory and context

Long runs exhaust the window. Three mechanisms, and they are not interchangeable:

• Compaction — summarise older history into a digest. Preserves the shape of what happened; loses detail.
• Context editing — clear old tool results outright, keeping the reasoning. Cheaper and lossier. Right when tool outputs are bulky and no longer needed.
• External memory — write findings to a store, retrieve on demand. Best fidelity, most machinery, and the only option that survives across sessions.

A practical pattern: keep the current task in context, write durable findings to external memory, and compact everything else.

Note the caching interaction from lesson 4. Compaction rewrites history, which invalidates the cached prefix from that point. That is usually still worth it, but it should be a decision rather than a surprise on the bill.

Multi-agent patterns

Multi-agent systems are widely oversold. Each pattern has a specific justification, and “it seems more sophisticated” is not one.

Parallel fan-out is the pattern that most reliably pays. Research across ten sources, per-file analysis, per-record enrichment: each worker gets its own clean context, they run concurrently, and the coordinator sees only summaries. Both wall-clock time and context pressure improve.

Two things to be honest about. Context isolation is often the real benefit — not “specialisation”. A worker with one job and a clean window outperforms one agent juggling ten threads, regardless of prompting. And every handoff loses information: the manager sees a worker’s summary, not its reasoning. Errors compound quietly across boundaries.

A sensible progression: single agent → parallel workers for the fan-out part → specialised workers only where tool or permission differences demand it.

Bounding the system

These are the controls that hold when the model does not behave. Prompt instructions are not in this category.

Reversibility gates. Irreversible actions require approval — sending, paying, deleting, publishing, committing. Not “risky” actions; irreversible ones. A wrong read costs a retry.

Least privilege, enforced in code. Each agent gets the minimum tool set. Authorisation is checked in your execution layer against the actual user, never delegated to the model. “The model was told not to” is not access control.

Turn and budget caps. A hard turn limit, and where available a token or spend budget so a runaway session terminates on its own.

Timeouts everywhere. Per tool call and per overall task.

Full tracing. Every call, argument, result, and the model’s stated reasoning. Without a trace, debugging an agent is guesswork — this is the observability that makes the rest possible.

Prompt injection in agent systems

The sharpest version of the problem from lesson 3, because now the model can act.

An agent that reads untrusted content — web pages, emails, documents, code comments — and also holds a dangerous capability is exploitable, and no prompt fixes it. Instructions and data share one channel.

The control is architectural: separate the capability from the untrusted context. One component reads and produces a structured, validated summary with no privileges. Another acts on that structure and never sees the raw content. Everything else — delimiting, filtering, output checks — is defence in depth behind that split.

Failure modes

Evaluating agents

Single-call evals do not transfer. Agents need trajectory-level measurement:

• Task completion rate — the headline number.
• Turns to completion — the cost driver.
• Tool selection accuracy — right tool, right arguments, per step.
• Recovery rate — after a tool error, does it correct or spiral?
• Boundary violations — did it attempt anything outside its scope? This should be zero, and it should be alerted on.

Fix a seed set of tasks and run the whole set on every prompt or tool change. Because agents are non-deterministic, run each task several times and look at the distribution — a single pass proves nothing.

Try this: Take an agent task and log every tool call and result for twenty runs. Count the turns and find the modal failure. In most systems the answer is a single tool with an unhelpful error message causing a retry loop — and fixing that one message is worth more than any model upgrade.