Running workflows

Publish a workflow from the portal's editor, the API, or otiumctl. A published version never changes: editing the document and publishing again creates the next version, and every run keeps the version it started with. Publishing a document identical to the latest version creates nothing and returns that version, so running publish on every merge is safe.

# validate without saving anything (also: otiumctl workflow validate, offline)
curl https://api.getotium.ai/v1/workflows/validate \
  -H "Authorization: Bearer $OTIUM_API_KEY" --data-binary @summarize.yaml

# publish: 201 with a new version, or 200 and "created": false if nothing changed
curl https://api.getotium.ai/v1/workflows \
  -H "Authorization: Bearer $OTIUM_API_KEY" --data-binary @summarize.yaml

A document the compiler rejects comes back 422 with every finding, the same ones otiumctl workflow validate prints. Retiring a version is DELETE /v1/workflows/{name}/{version}: it stops new batches from using it, and runs already using it are unaffected.

A workflow is a document you review in a pull request like any other code. otiumctl workflow apply publishes a directory of them and changes only what changed:

otiumctl workflow apply -f ./workflows/ --dry-run      # what would change
otiumctl workflow apply -f ./workflows/ --strict --max-critical-path 4h

There is no state file: a published version can't change, so the server always knows exactly what is deployed. Removing a file from the directory deletes nothing, because live runs may still use that version. --max-critical-path fails the step if a change makes the workflow's p95 path longer than you allow, so a slow change is caught before it ships rather than when a batch is refused. In CI, set OTIUM_API_URL and OTIUM_TOKEN (an API key) instead of signing in.

A workflow runs through the same OpenAI-compatible Files and Batch API as a single model call. Each line of the file is one item, names the workflow, and carries the item's input in otium.input, which is what ${input.…} bindings read:

{"custom_id":"doc-1","otium":{"workflow":"summarize@1","input":{"text":"Otium runs batch inference on spare GPUs."}}}
{"custom_id":"doc-2","otium":{"workflow":"summarize@1","input":{"text":"…"}}}

The prompts live in the published workflow, so a line has no request body; one sent anyway is ignored. Every line in a file must name the same workflow and carry an input, and a file can't mix workflow lines with ordinary ones. name@version pins a version; a bare name uses the latest, which is handy while experimenting and risky in production. Each run records the version it used either way.

from openai import OpenAI

client = OpenAI(base_url="https://api.getotium.ai/v1", api_key=OTIUM_API_KEY)

batch_file = client.files.create(file=open("items.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=batch_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",  # or "7d"
)

Everything is on one host, https://api.getotium.ai/v1, with one API key: files and batches, and also publishing workflows and assets, following runs and resuming parked ones.

Put every item in one file rather than drip-feeding them: Otium is the queue, so there is no rate limiter or retry loop to write. The items share the workflow's prompts, which are served from cache for free once the first has run, and a full file starts a machine sooner than a trickle.

completion_window picks the tier: 24h (Standard) or 7d (Remnant). The longer the window, the less you pay; see pricing. A run then has two clocks:

• The execution budget is the window, spent while the run is Otium's to work on. It stops while the run is parked at an external step.
• The wall clock runs out when the batch expires, a window after you submit, and never stops.

Both are checked when you submit, against the compiled plan's p95 rather than its typical duration. A workflow whose p95 execution doesn't fit the window is refused with window_too_short; one whose p95 path, parks included, doesn't fit before the batch expires is refused with expiry_too_short. Refusing at the door is deliberate: a workflow that can't fit would otherwise be found out days later, item by item.

A run that runs out anyway ends with outcome budget_exhausted or deadline_exceeded, and keeps every step that succeeded.

Each model step is charged like a single request: tokens at the rate for its size and your window. Every attempt that gets an answer is charged, including one a validator rejects, so a step whose answer was rejected twice costs three attempts. An attempt that fails before answering, and a skipped step, cost nothing. A tool step costs $0.002 per call your endpoint answers (details).

When a batch is accepted, each run reserves its worst-case (p95) cost against your balance, so a run is never started that your balance can't finish. The reservation is released as the run is charged and when it ends. A run that your balance, or your monthly spending limit, can't cover is held and starts when it can, while its clocks keep running.

A batch that names a workflow becomes a run set, with one run per line. GET /v1/run-sets lists them with the batch each came from.

• GET /v1/run-sets/{id}: counts by status, the outcome, and holds.
• GET /v1/run-sets/{id}/runs: the runs, filterable by status and custom_id.
• GET /v1/runs/{id}: every step of one item, with its status, reason, output, cost and time on a worker.
• GET /v1/runs/{id}/nodes/{key}/attempts: every attempt at a step, including the outputs a validator rejected.

otiumctl run-set get <run-set-id>        # counts, outcome, what is held and why
otiumctl run get <run-id>                # every step of one item
otiumctl run attempts <run-id> summarize # every attempt, with what it produced
otiumctl run-set output <run-set-id> --out results.jsonl
otiumctl run-set node-output <run-set-id> summarize --out summarize.jsonl

Held is not stuck. A batch can read running for hours while its work waits, by design, for cheaper compute (price) or for enough work to share a machine's start-up (gathering). holds says which steps are waiting, why, and latest_start, the latest the wait is allowed to run before capacity starts regardless.

When the batch finishes, its output file has one line per item, with the item's status, its outcome and every step's result:

{"custom_id":"doc-1","run_id":"run_…","status":"succeeded","outcome":"completed",
 "nodes":{"summarize":{"status":"succeeded","disposition":"ok","body":{"headline":"Batch inference on spare GPUs"}}}}

A partial, expired or cancelled run still delivers the steps that completed. GET /v1/run-sets/{id}/output serves the same lines for any run set.

If you already have code that reads OpenAI batch output, fetch one step's results in exactly that format:

GET /v1/run-sets/{id}/nodes/{key}/output.jsonl

One line per run, with the model's own response, and an error line carrying the typed reason for a run where that step didn't succeed. Moving a multi-batch pipeline onto a workflow then changes how you submit, and none of the code that collects.

Cancel a whole batch with the Batch API's POST /v1/batches/{id}/cancel, which stops its runs, parked ones included, or one run with POST /v1/runs/{id}/cancel.

A run that stopped can be continued rather than started again: POST /v1/run-sets/{id}/continue creates a new run for every item that didn't succeed, with the same custom_id. Steps that already succeeded are carried over with their outputs and not paid for again; the rest run. A continuation is a new purchase, with its own window and charge, and asking twice returns the same continuation rather than buying it twice. Read its results from GET /v1/run-sets/{id}/output, since there is no batch file behind it.

Continue after a run was cancelled, ran out of time, or hit a passing infrastructure failure. It won't help a step whose output was rejected on every attempt: that would buy the same failure again. Fix the workflow, publish it, and submit a new batch; a continuation only runs the version the original runs used.