Putting AI Into a Workflow

The shape of every AI workflow

Strip away the tooling and nearly all of them are the same three parts:

Trigger → AI step → Action

• Trigger — something happens. A form is submitted, an email arrives, a timer fires.
• AI step — something is judged, extracted, classified or written.
• Action — something changes. A row is updated, a message is queued, a ticket is routed.

The tools that host this change constantly — the visual builders, the assistant platforms, the code frameworks. The pattern does not. Learn the pattern and the tool is an implementation detail.

The one thing that makes AI steps different

Ordinary automation is deterministic. The same input produces the same output, forever. Everything about how we build and operate automation assumes this: retries are safe, tests are stable, a passing run means a working system.

An AI step breaks that assumption. It samples from a distribution. The same input can produce different output on Tuesday than it did on Monday. And the system can change under you when a provider updates a model.

Four consequences follow, and they are the whole lesson.

1. Retries are not automatically safe

The classic bug: the AI step drafts a reply, the send succeeds, recording the send fails, the workflow retries — and the customer gets two emails.

With a deterministic step you can often retry blindly. Here you cannot. You need an idempotency key — a record of “this trigger has already been actioned” checked before acting, not after.

2. You cannot test by running it once

One good run tells you almost nothing. What you need is a fixed set of representative inputs with known-correct outputs, run whenever anything changes.

Twenty cases in a spreadsheet is a legitimate evaluation set and puts you ahead of most teams. That is the next lesson.

3. Confidence should route the work

Accuracy is never uniform. A classifier at 85% overall might be 98% on clear cases and 60% on ambiguous ones. Treating all outputs identically throws away that structure.

Ask the model for a confidence signal alongside its answer and route on it:

high   → act automatically
medium → act, but flag for review
low    → send to a human, do not act

A caveat worth stating plainly: self-reported confidence is weakly calibrated. It is a useful ordering signal, not a probability. Set your thresholds by measuring what actually happens at each level, not by trusting the number.

4. Log the input and the output, always

When something goes wrong two months from now, “the AI classified it wrong” is not debuggable. What you need on every run:

• the exact input that was sent,
• the exact output that came back,
• which prompt version and model produced it,
• what action was taken as a result.

That last item matters most. Without it you cannot answer “did we email this person already?” — which is the question you will be asked.

Three levels of autonomy

Match the level to the cost of being wrong. Most teams jump straight to the first and regret it.

A sound sequence: ship at suggest, measure real accuracy for a fortnight, then promote to review, and only reach automatic for the specific case where the numbers justify it.

Four workflows that genuinely work

These are durable shapes, not product recommendations.

Triage and routing. Ticket arrives → classify topic and urgency → route. Low risk: misrouting is annoying and instantly fixable. An excellent first project.

Extraction into structure. Invoice or form arrives → pull out vendor, amount, date, line items → write to a system. High value, and easy to validate — the numbers either reconcile or they do not.

Summarising for a human. Long thread or transcript → summary plus action items → posted for review. Output goes to a person who can judge it, so the risk is inherently contained.

Draft generation. New content published → draft variations for other channels → land in a queue for approval. Suggest-level by construction.

Notice the common thread: each produces something a human either checks or can trivially correct. That is not a coincidence — it is why they work.

What to avoid early on

• Chains where every step is an AI step. Errors compound. Four steps at 90% each is 66% end to end.
• Automating something you have never done manually. If you cannot describe the correct answer, you cannot evaluate it.
• Silent failure. A workflow that stops working without telling you is worse than no workflow.
• Anything irreversible without a gate. Same rule as agents — reversibility decides.

Try this: Take a repetitive task and write down its trigger, its AI step and its action on one line each. Then write the four inputs where you would most fear a wrong answer. If you cannot state what the correct output is for those four, that is the work to do before automating anything.