Many drafts, one editor: the mixture-of-agents pattern
Ask a few models, or one model a few times, to answer the same request. Then give every answer to one more model and ask it to write the final version from the best parts. Researchers call this a mixture of agents. It can beat a single answer from a much larger model, and it can also cost twice as much for nothing. Which one you get depends on choices you make when you design it.
The pattern
There are two roles. The drafters (the papers call them proposers) each answer the request on their own. The editor (the aggregator) reads every draft and writes one answer. It is not asked to pick a winner; it is asked to keep the strongest opening from one draft, the clearest explanation from another and the best example from a third, and make them read as one.
Solid edges are data dependencies the compiler derived. Dashed edges are after: or after_settled: — ordering you asserted, with no data flowing.
A thicker edge is a barrier: the scheduler stops chaining there. Square-cornered steps are tool
calls Otium makes to your endpoint.
Describe this graph in words
- draft_plain — otium-small, on the critical path. Starts the graph.
- draft_skeptic — otium-small. Starts the graph.
- draft_story — otium-small. Starts the graph.
- edit — otium-medium, on the critical path. Runs after draft_plain, across a model change barrier and after draft_skeptic, across a model change barrier and after draft_story, across a model change barrier.
It works for the same reason an editor improves a writer: spotting the best of several attempts is easier than writing the best attempt first time. A drafter that writes a weak paragraph seldom writes the same weak paragraph as the other drafters, so the editor has something better to choose from at almost every point.
The pattern is a fan-out followed by a join, the same shape as many pipelines. What makes it different is that every branch does the same job, and the join is where the quality comes from.
What the research found
The name comes from a 2024 paper led by researchers at Together AI. They had several open models answer each prompt, passed every answer to the next layer of models as extra context, and had a final model write the answer. Built only from open models, it scored 65.1% on AlpacaEval 2.0, a benchmark of open-ended instructions judged against a reference model, where GPT-4o scored 57.5%.
An LLM tends to generate better responses when presented with outputs from other models.
The paper stacks several rounds of drafting and merging, but its authors note that the first round of merging gives most of the gain, and that every extra round delays the first word of the answer. One round, a set of drafts and one editor, is the version worth building first.
A 2025 follow-up from Princeton asked whether the mixing matters. It mostly does not. Taking several drafts from the single best model, which the authors call Self-MoA, beat mixing different models by 6.6% on AlpacaEval 2.0 and by 3.8% on average across benchmarks including MMLU and MATH. Their explanation is a trade-off: different models make the drafts more varied, but they also lower the average draft, and the editor’s result tracks the quality of what it is given more than its variety.
Older work covers the simpler versions of the same idea:
| Variant | How the answer is chosen | Works for | Paper |
|---|---|---|---|
| Vote | Sample several answers and keep the most common one. | Questions with one right answer: maths, a label, a yes or no. | Self-consistency (Wang et al., 2022); More Agents (Li et al., 2024) |
| Pick | A model reads the samples and picks the one most consistent with the rest. | Free text where you want one answer, not a blend: a summary, code. | Universal self-consistency (Chen et al., 2023) |
| Rank and fuse | A ranker orders the candidates; a second model merges the top few. | Mixed-quality candidates from different models. | LLM-Blender (Jiang et al., 2023) |
| Mixture of agents | An editor model writes a new answer from every draft. | Open-ended writing, where the best answer combines parts of several. | Wang et al., 2024; Li et al., 2025 |
When it pays, and when it does not
It pays when the answer is open-ended and quality is judged as a whole: an article, a product description, an email, a summary written for a particular reader, an explanation. Several drafts give the editor real choices, and the result reads better than any one draft.
It does not pay when there is one right answer. Pulling a price out of a listing, choosing a category, answering yes or no: an editor has nothing to combine. A vote is cheaper and at least as good, or better still, a check that rejects a wrong answer and asks again (see the guide to structured output).
- Good fits: marketing and product copy, article drafts, reply drafts for support, reading-level rewrites, explanations for a stated audience.
- Poor fits: extraction, classification, anything a schema or a rule can check, and anything where a fact that is not in the drafts would be a mistake the editor cannot see.
- Latency matters: the edit waits for the slowest draft, so the pattern adds at least one model call to how long each answer takes. It suits batch work, where nobody is waiting.
What it costs
The drafts are rarely the expensive part. The editor is: it reads every draft as input, so its input grows with the number of drafts, and it runs on the larger model. Here is one article of about 900 words, at illustrative rates of $0.05 in and $0.25 out per million tokens for a small model and $0.80 in and $4.00 out for a medium one.
| Design | Per article | Per 10,000 |
|---|---|---|
| One medium-model draft | $0.0050 | $50 |
| Three small drafts, no edit (for comparison) | $0.0009 | $9 |
| Three small drafts and a medium edit | $0.0090 | $90 |
| Three medium drafts and a medium edit | $0.0231 | $231 |
Two things follow. First, the honest pitch is quality, not savings: three small drafts and a medium edit cost under twice a single medium draft, and about 40% of the usual version with large drafters. Second, adding drafts is cheap at the drafting end and expensive at the editing end. Going from three drafts to six barely moves the drafting cost and nearly doubles what the editor reads.
Where it goes wrong
- Drafts that are copies. The same model with the same prompt at a low temperature writes nearly the same draft three times, and the editor has nothing to choose between. Give each draft its own angle (plain, a worked example first, the sceptic’s objection first), and vary the temperature.
- Weak drafts. The follow-up research is clear that the editor cannot write much better than its best input. If the small model’s drafts are poor, more of them will not help; use a better drafter or fewer, better drafts.
- An editor that favours position or length. Model judges prefer the answer shown first, prefer longer answers, and prefer text in their own style; the MT-Bench study measured all three. Shuffle the drafts’ order, label them neutrally, and tell the editor what good looks like in concrete terms.
- Too many drafts. Models use the start and end of a long input better than its middle, so past a handful the middle drafts are read less carefully. Three to five is a common range.
- Facts that appear from nowhere. An editor asked to improve can add a figure that no draft or source contained. Give every step the same source notes, tell each to add nothing beyond them, and check the result against the notes before it is used.
- One bad draft failing everything. If a draft errors or comes back malformed, the editor should work from the rest rather than the whole item failing.
The choices you control
A hosted "ask several models and merge" endpoint makes these choices for you. Built as your own pipeline, each one is a line you can change and measure:
| Choice | Start with | Why |
|---|---|---|
| How many drafts | Three | Enough for real choice; few enough that the editor reads them all well. |
| Who drafts | The best small model you trust, several times | Quality matters more than variety. Add a second model only if it is as good. |
| How drafts differ | An angle each, plus a spread of temperatures | Variety without lowering quality. |
| Who edits | A larger model, at a low temperature | The edit is judgement and consistency, not invention. |
| What the editor sees | The source notes and every draft, labelled | The notes let it check the drafts, not just merge them. |
| What the editor returns | The article, plus which draft each part came from | A person reviewing the output can see what was kept. |
| What runs after | A check against the notes | Catches added facts before anything is published. |
Then measure it, because the research cannot tell you whether it beats a single larger draft on your work. Take fifty real items, produce both versions, hide which is which, and have a person (or a judge model, with the cautions above) choose. If the pattern does not win clearly, the single draft is cheaper and simpler. The guide to measuring quality covers how to run that comparison.
How it runs on Otium
On Otium the pattern is an ordinary workflow: a document you write, with each draft and the edit as a step, and the graph worked out from which step reads which result. Nothing is hidden from you. You choose the number of drafts, their prompts, models and temperatures, what the editor reads, and what checks run after it.
- The drafts read only the input, so they run side by side; the edit reads all three, so it waits for them. You do not write that ordering; the compiler derives it.
- The drafts share a model and a prompt prefix, so they run on the same warm machine over a cached prefix.
- Each draft is marked to carry on if it fails, and the editor’s prompt includes a draft only when it exists, so one bad draft costs one draft, not the article.
- Every answer is checked against its schema before the next step reads it, and a malformed one is retried in place.
- The whole job is priced before it runs, and its unit tests can drive every case (all drafts present, one missing) without a model.
Sources
Frequently asked questions
Is a mixture of agents the same as a mixture of experts?
No. A mixture of experts is inside one model: a router sends each token to a few of the model’s sub-networks. A mixture of agents is outside the model: several complete answers, and one more call that writes from them.
Do I need several different models?
Usually not. The 2025 Self-MoA study found several drafts from the single best model beat mixing models, because a weaker model lowers the average draft. Vary the prompt and temperature instead, and add a second model only if it is as good.
How many drafts should I use?
Start with three. More drafts cost little to write but grow the editor’s input, and models read the middle of a long input less carefully. Measure before going past five.
Is it cheaper than one large model?
Rarely. The editor runs on the larger model and reads every draft, so the total is usually above one large-model draft. Use it for quality on open-ended writing, and check on your own items that it wins.
Related guides
Most useful AI work is several model calls in a row. The common patterns, where chained jobs lose time and money, and how to design steps that hold up.
You cannot choose a model, quantization or prompt without measuring quality. How to build a small gold set and a test loop that turns choices into data.
What temperature, top-p, max_tokens, stop sequences, penalties and seeds do to a model’s answer, and sensible starting points for batch work.
Bigger is not automatically better. How to match model size to a batch task, weigh quality against cost, and send only the hard items to a bigger model.
An MoE model holds many parameters in memory but activates a fraction per token. Total drives memory, active drives speed, and that split changes the economics.
Schemas, validation, retries and escalation: how to get JSON from a language model that your code can trust, and what to do with the answers that fail.
Prompt caching reuses the computed state of a repeated prefix so you stop paying full price for it. How it works, why batch benefits most, and how it is billed.