When should you use an AI agent instead of a workflow?
An agent decides its own steps; a workflow follows steps defined in advance. Both can do the same job, and they spend money in different places: an agent resends its conversation on every turn, while a workflow can turn thinking off for a step that does not need it, check for rejects before the expensive step, and run as a batch at half price. This guide goes through where the costs differ, what we measured when we ran both on the same job, and how to tell which your job needs.
Two ways to run the same job
An agent is a model given a goal and some tools. It decides what to do next, calls a tool, reads the result and decides again, until it judges the job done. A workflow is a fixed sequence of steps: extract the fields, check them, fix what failed, write the description. Code decides what happens next, and the model does one step at a time.
When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed.
The same piece notes that agentic systems often trade latency and cost for better task performance. The question for a given job is whether it needs what the agent buys: the freedom to choose its own steps.
Where the costs differ
Resending the conversation
A model has no memory between requests, so an agent sends its whole conversation so far on every turn: the instructions, the description of every tool, every earlier answer. A workflow step sends only what that step needs.
The Messages API is stateless, which means that you always send the full conversational history to the API.
It adds up. Describing their own research system(opens in a new tab), Anthropic’s engineers wrote that “agents typically use about 4× more tokens than chat interactions”.
Prompt caching softens this. Providers charge a fraction of the input price for the start of a prompt they have seen recently, and an agent’s growing conversation is exactly that kind of repeated text. Anthropic(opens in a new tab) and OpenAI(opens in a new tab) each charge a tenth of the input price on most of their current models; Anthropic keeps the text for five minutes by default.
No provider promises the discount. OpenAI’s documentation(opens in a new tab) says a request reuses cached text only if it “reaches a machine holding a matching entry that has not expired”, and that “maintaining a session doesn’t guarantee a cache hit”. Anthropic’s(opens in a new tab) says requests sent at the same time do not share a cache until the first has started to answer. So an agent that takes many turns is only as cheap as the caching it happens to get.
Batching
A batch is a set of requests the provider answers within a day instead of at once, at half price: Anthropic(opens in a new tab), OpenAI(opens in a new tab) and Google(opens in a new tab) all charge 50% of their normal prices for batch work and aim to finish it within 24 hours. A workflow step can go as a batch, because its requests do not depend on each other: every item’s extraction can run at the same time. An agent can only be batched a turn at a time: every item’s first turn in one batch, every second turn in the next, because each turn needs the answer to the one before. That takes as many batches in a row as the longest item takes turns, each of which can take a day, and the agent’s cache may not last from one to the next. Batching only applies to work that can wait.
Model size and thinking, step by step
A workflow can send easy steps to a small, cheap model and keep a larger one for the hard ones. It can also turn thinking (the model reasoning before it answers, which is billed like the rest of its answer(opens in a new tab)) on for the step that needs it and off for the rest. An agent makes every decision with one model and one thinking setting. Sending each request to the cheapest model that can handle it has been studied: a 2023 paper, FrugalGPT(opens in a new tab), reported that it could “match the performance of the best individual LLM (e.g. GPT-4) with up to 98% cost reduction”. In our runs, turning thinking off for one step paid off. Sending easy steps to a smaller model did not: the results were not accurate enough to use, so treat that one as something to test on your own job.
Stopping early
When some items should be turned away (duplicates, things you do not sell), a workflow can check for them before its expensive steps, and an item that fails stops there. An agent can stop early too, once it decides to, and a reasonable one often does. The workflow’s advantage is that you choose where the check goes. Put it after the expensive step and it saves nothing.
What a benchmark measured
We built an open benchmark to measure these differences rather than argue them: the same 500 product listings, the same models, tools and scoring, run as a minimal agent per item, an off-the-shelf agent handed the whole file, a single prompt, and a workflow. Each side had one round of improvements before the runs reported here. The agent’s mattered: as first built it checked each record and then sent it again in a second turn, and giving it one tool that does both cut its cost by up to 49%. The agent figures below are the improved agent, averaged over its two runs. Every request was counted as it went out, and costs were priced afterwards from public price lists, with the agent given the provider’s full caching discount. The design, the code and every run are public on GitHub(opens in a new tab).
| Run | Right: workflow vs agent | Workflow | Agent | Workflow, batch | Agent, batched in rounds |
|---|---|---|---|---|---|
| Gemma 4 12B, thinking on | 97% vs 95% | $1.23 | $0.89 | $0.61 | $0.44 |
| Gemma 4 12B, thinking off | 94% vs 94% | $0.22 | $0.15 | $0.11 | $0.07–$0.08 |
| Claude Haiku 5.5, thinking on (its default) | 97% vs 95% | $0.66 | $0.48 | $0.33 | $0.24–$0.27 |
| Claude Haiku 5.5, thinking off | 95% vs 95% | $0.42 | $0.24 | $0.21 | $0.12–$0.15 |
- The workflow and the agent were within two points of each other on accuracy in every run. The workflow was never measurably worse.
- Sent for an answer right away, the agent cost less than the workflow in every run. It finished most items in one request, where the workflow made two or more and sent and wrote more text. On Haiku, prompt caching lowered the agent’s bill further, but the agent would have cost less without it too.
- A single prompt with no tools cost a little less than the agent. On one model it was as accurate; on the other it got 7 to 15 points fewer items right, because nothing checked its records against the rules.
- Sent as a batch, the workflow cost 12% to 31% less than the agent sent right away.
- Batched a turn at a time, the agent came out cheaper than the workflow in batch, at the price of up to eight batches in a row, a cost that depends on how many turns each item needs, and storing and resuming every item’s conversation between them, which agent frameworks do not do for you. Sent that way for real once, a simple job took three rounds and six minutes, and kept its caching discount.
- Turning thinking off for the one step that writes the description cut the workflow’s cost by 15% to 30% with thinking on, a choice an agent with one setting cannot make.
- With items to turn away, a workflow that checked first cost less than one that checked after its most expensive step. On a small model, the workflow also kept far more of the new items than the agent did. On this task the agent took more turns, and without caching it would have cost more than the workflow in every run.
- The off-the-shelf agent, handed the whole file, cost $0.79 to $2.25 per 1,000 items across three runs, against the minimal agent’s $0.48, for about the same accuracy. Its cost varied more than anything else we ran.
How to decide
Ask these about your job, in order:
- Can one request do the whole job? Try a single prompt first, and check its answers against your rules. In our benchmark it was the cheapest option on one model and missed too often on the other.
- Can you define the steps before it runs? If the next step depends on what the last one found, in ways you cannot list in advance, use an agent. That is what agents are for.
- Can the answers wait? If so, a workflow sent as a batch costs less than an agent sent right away, and you know its cost before it runs.
- If the answers cannot wait, measure both on a sample of your own items. An agent that finishes in one or two turns can cost less than a workflow that makes a request for every step.
- Count the agent’s turns. A turn that only re-checks or re-sends what it already has resends the whole conversation; removing them is often the cheapest improvement.
- Do some items need to be turned away? Put the cheapest check that can reject them first, before the step that costs the most.
- Are some steps easier than others? Give those a smaller model, or turn thinking off for them.
The two also combine. An agent can decide, while it works, that a known set of steps should run over a set of items, and hand those steps to a workflow: the agent keeps the open-ended decisions, the workflow does the repeated work at batch prices.
How to measure your own job
Agent benchmarks often report accuracy and leave cost out. A 2024 study of them, AI Agents That Matter(opens in a new tab), found “a narrow focus on accuracy without attention to other metrics”, and concluded that as a result “SOTA agents are needlessly complex and costly” (SOTA means state of the art: the best published). Its remedy is the one this guide follows:
Our focus on cost in addition to accuracy motivates the new goal of jointly optimizing the two metrics.
- Count tokens as each request leaves, with a small program between your code and the model, rather than trusting the totals your agent library reports.
- Record tokens, not dollars, and price them afterwards, so a price change does not need a rerun and you can price one run as any model.
- Give the agent its provider’s caching discount in full; without it the agent looks more expensive than it is.
- Run both on the same items and compare item by item. Most of the variation is between items (a long listing costs more for everyone), and pairing removes it. Evan Miller’s paper Adding Error Bars to Evals(opens in a new tab) calls comparing this way a “free” reduction in variance, which here means less noise for the same number of items.
- Report cost per correct result beside cost per item, so neither side can look cheap by failing.
The benchmark’s meter, scoring and report generator are open source and work with any model you call the way you call OpenAI’s or Anthropic’s API, so you can point them at your own job.
Running known steps on Otium
Otium AI runs work that can wait, single requests or whole multi-step workflows, on spare cloud GPUs when they are cheapest, within a deadline you choose. On Otium you write a workflow as one short document of steps, most of them prompts; Otium works out which step needs which result and runs each step as batch work, which is where the benchmark found the workflow’s saving. The benchmark itself does not use or need Otium.
Sources
Frequently asked questions
Is an AI agent always more expensive than a workflow?
No. In our benchmark, sent for an answer right away, a well-built agent was cheaper than the workflow: it finished most items in one request where the workflow made two or more, and prompt caching lowered its bill further. Sent as a batch, the workflow was cheaper than the agent sent right away. An agent can be batched a turn at a time too, which on simple work made it cheaper still, at the cost of many batches in a row and the bookkeeping to resume each item between them.
Is a workflow less capable than an agent?
Not for work whose steps are known. On the same job and model, the workflow in our benchmark was never measurably less accurate, and on a smaller model it was more accurate on one of the two tasks. For work whose steps depend on what is found along the way, an agent is the right tool.
Does giving an agent a skill or a procedure close the gap?
In our runs it did not. An off-the-shelf agent given a written procedure for the task cost more and scored about the same as without it. A procedure in prose is still read and followed turn by turn, with the conversation resent each time.
Related guides
Prompt engineering shapes one call, loop engineering the control flow, graph engineering the whole job as a DAG. What each is, and when it is the right tool.
Prompt caching reuses the computed state of a repeated prefix so you stop paying full price for it. How it works, why batch benefits most, and how it is billed.
Batch AI inference runs large volumes of requests against a deadline instead of in real time. How it differs from interactive inference, and why it costs less.
Most useful AI work is several model calls in a row. The common patterns, where chained jobs lose time and money, and how to design steps that hold up.
From tokens to dollars for a whole pipeline: measuring token counts, adding retries, branches and per-call fees, and planning for the expensive case.