How to choose a deadline for AI work
Most teams pick a deadline by habit: "as soon as possible", or whatever the service offers. But the deadline is often the biggest single lever on what AI work costs. Here is a way to choose one on purpose.
Start from whoever reads the output
Do not ask how fast the work could be done. Ask when the result is first used, and by whom. That moment is your real deadline, and it is usually much later than "now".
| The output feeds | When it is first read | A deadline that fits |
|---|---|---|
| A person waiting in a chat | Seconds | Not batch work: use a real-time API |
| A daily report or dashboard | Tomorrow morning | Hours |
| Product pages refreshed each week | Next week | Days |
| A backlog: tagging an archive, filling a search index | Whenever it is done | The longest on offer |
| A model evaluation before a decision meeting | The meeting | Work back from the date |
If the answer is "nobody reads it until next week", a one-hour deadline buys nothing but a higher bill.
A deadline is not how long the work takes
A deadline is the latest the result may arrive. The work itself may take minutes. The time in between is what lets a batch service wait for cheaper machines, gather your work with other work so a machine is not started just for you, and retry anything that failed.
So expect results to arrive anywhere inside the window, not at the start of it. If your code needs the result by 9am, set the deadline for 9am, and do not count on getting it at 3am just because it sometimes arrives early.
Deadlines in a pipeline add up
If each step of a pipeline is sent as its own job, each job gets its own deadline, and you can wait out every one of them. Three steps on a 24-hour deadline can take three days. Plan for that, or send the pipeline as one job so one deadline covers all of it.
Steps that wait for a person or another system need room too. If a reviewer approves drafts within a working day, a 24-hour deadline for the whole pipeline leaves no margin. Give the pipeline a deadline that covers the slowest realistic wait, not the typical one.
Split urgent work from the rest
Most jobs are a small urgent part and a large part that can wait. New products added today may need descriptions tonight; the 200,000 already in the catalog do not. Send them as two jobs with two deadlines and pay the faster price only for the part that needs it.
The same goes for a first run and its reruns. The first pass over a backlog can take the longest deadline. Fixing the few items that failed, while someone is waiting to check them, may justify a shorter one.
Leave room for things to go wrong
Set the deadline with enough margin that you can act if a job comes back with failures. If the output must be live by Monday and a job may return a few percent of items that need another pass, a deadline of Monday morning leaves you no time to send that pass. Friday does.
Also decide what a missed deadline means for you. Can the old data stay up another day? Can you show a placeholder? Knowing the answer tells you how much margin you actually need.
Questions to ask a provider
- Which deadlines do you offer, and what does each cost?
- Does the deadline cover each request, or the whole job? For a pipeline, does it cover every step or only one?
- What happens to the work that is not done at the deadline? Do I get the finished part back?
- Am I charged for work that misses the deadline?
- Can I see why work is waiting, so I can tell waiting for a cheaper machine from being stuck?
Deadlines on Otium
Otium sells a ladder of deadlines, called tiers. Live today: Remnant (< 7 days) and Standard (< 24 hours). You choose one with the Batch API's completion_window. A workflow's whole run gets one deadline, however many steps it has, and a step that waits for your system stops the clock that measures Otium's own work.
Otium checks the deadline when you submit. If the workflow’s slow-case (p95) estimate does not fit the window you chose, the batch is refused at once, instead of being found out days later. A run that runs out of time anyway keeps every step that finished. While a batch waits, its view says which steps are waiting, why, and the latest they will wait before starting regardless.
Once you have a deadline, send everything that shares it at once: the whole backlog as one batch file or one workflow run. There is no need to pace it. Otium holds the work, retries what fails and delivers by the deadline. And a big batch is the cheap way to use a deadline: Otium waits until there is enough work to fill a machine before starting one, so a whole backlog starts sooner than a trickle that has to be gathered.
Further reading
Frequently asked questions
How do I choose a deadline for a batch AI job?
Work out when the output is first used, and by whom, then subtract enough margin to fix anything that comes back failed. Choose the longest deadline that still meets that time.
Will my batch finish early?
Sometimes, but do not plan on it. A batch service uses the time before the deadline to find cheaper capacity, so results can arrive anywhere inside the window.
What deadline should a multi-step pipeline have?
Enough for the whole pipeline, including any waits for people or other systems. If each step runs as its own job, each one can take its full deadline, so the waits add up.
Once I have picked a deadline, how should I send the work?
All at once, in one batch file or one workflow run, rather than a few requests at a time. The service holds it, retries failures and delivers by the deadline, and a big batch fills machines sooner and costs less to run than a trickle.
Related guides
A deadline is the most valuable thing you can give a batch inference platform. How SLA tiers turn patience into price, and how to pick the right window.
Batch AI inference runs large volumes of requests against a deadline instead of in real time. How it differs from interactive inference, and why it costs less.
Most useful AI work is several model calls in a row. The common patterns, where chained jobs lose time and money, and how to design steps that hold up.
From tokens to dollars for a whole pipeline: measuring token counts, adding retries, branches and per-call fees, and planning for the expensive case.