Workflows

Otium prices waiting: you buy a completion window and we spend it finding cheap capacity. That works for one call and fails for a chain, because a pipeline pays the full window per stage. Seven stages on a seven-day window is seven weeks.

A workflow moves the window from the stage to the run. One budget, one document, and the scheduler sees the whole shape before it starts, which is also what lets it keep a model loaded across steps and retry a malformed output in place instead of costing you another window.

The document is a graph, a DAG. Each node is an ordinary chat request; the edges are derived from which node reads which output, never declared; and the compiler plans the run from the graph: what runs side by side, where it must wait, and what the whole thing should cost. A node can also be a call to your own system (a tool step), with the answer flowing into the next node. OpenAI's and Anthropic's batch services take one request at a time and stop at a tool call, so the graph is otherwise yours to build and operate. This is graph engineering, as against prompt or loop engineering: the three, compared.

A node's body is an OpenAI chat-completions request, with ${…} bindings filled in from the item's input or an earlier step's output. If you can write a batch line, you can write a node.

name: summarize
description: One step, to see the shape.

input:
  schema:
    type: object
    required: [text]
    properties:
      text: { type: string }

schemas:
  summary:
    type: object
    required: [headline]
    properties:
      headline: { type: string }

nodes:
  summarize:
    target: { product: otium-small }
    estimate: { tokens_in: 800, tokens_out: 120 }
    body:
      messages:
        - role: system
          content: "Summarize in one line. Reply as JSON."
        - role: user
          content: "${input.text}"
    output:
      validators:
        - type: json_schema
          schema: { $ref: "#/schemas/summary" }
      on_invalid: { retry: 2, then: fallback }
    fallback:
      - { target: { product: otium-medium } }

Its tests run the real engine over canned answers, so you can check the logic before spending anything:

cases:
  - name: summarizes
    input: { text: "Otium runs batch inference on spare GPUs." }
    nodes:
      summarize: { output: { headline: "Batch inference on spare GPUs" } }
    expect:
      status: succeeded
      prompts:
        summarize: { contains: ["Otium runs batch inference"] }

  - name: a bad answer retries, then falls back
    input: { text: "x" }
    nodes:
      summarize: # one response per attempt
        - output: "Sure! Here you go:" # not JSON
        - output: { title: "wrong" } # parses, misses headline
        - output: "{"
        - output: { headline: "x" }
    expect:
      products:
        summarize: [otium-small, otium-small, otium-small, otium-medium]

Every complete workflow in these docs compiles without a warning and its tests pass: we check them on every change, against the same compiler you run with otiumctl workflow validate. A few short excerpts show a single feature, or a warning on purpose, and say so.

• The workflow document Every field of a workflow, how a step reads another step’s output, and how the order is worked out from that.
• Conditions Run a step only when the data calls for it, and include part of a prompt only when it applies.
• Validation and fallback Check every step’s output against a schema, retry it, hand it to a bigger model, and see what happened.
• Assets Reference data every item in a batch shares, published once, pinned by each batch, and kept out of your requests.
• Tools and external steps Call your own HTTP endpoint from a step, pause a run while your system or a person does the work, and get notified when a batch finishes.
• Running workflows Publish, submit a batch, choose a window, follow runs, collect outputs, and know what it costs.
• Testing Unit tests with canned step outputs, prompt snapshots, and the compiler’s checks, run offline and in CI.

Your window is a deadline, not a start time. Otium may hold work back to gather it onto one machine or to wait for cheaper compute, so plan on results arriving within the window rather than as soon as possible. With nothing already running, allow up to an hour for the first step to start: finding capacity, starting a machine and loading the model. Tested at 20,000 items through a three-step workflow, with no failures.