Migrating from the OpenAI Batch API

How the batch flow works

An OpenAI batch takes three steps, and a compatible service offers the same three:

1. Upload a JSONL file: a text file with one request per line. Each line carries a custom_id you choose and the request itself.
2. Create a batch that points at the file and says how long you can wait (the completion window).
3. Check on the batch until it is done, then download the results: one line per request, matched to your input by custom_id.

Each line is the same chat request you already send one at a time. So the code that builds your requests does not change. Only the address and the key do.

The base-URL swap

The official OpenAI SDKs let you set the address (the base URL) and the API key when you create the client. Point them at the compatible service and the same calls go there instead:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.getotium.ai/v1",
    api_key=OTIUM_API_KEY,
)

# The file-then-batch calls are unchanged:
f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="7d",  # wider window, lower price
)

What to check when you move

• The model name. Each service has its own list. On Otium you name a size (otium-small, otium-medium or otium-large) rather than a model, and the size runs a current open model.
• The completion window. Pick the longest your work can wait; it is the cheapest.
• Reading the results. They are still matched to your requests by custom_id, so the code that joins them up does not change.
• Your pacing. If your code drip-feeds requests to stay under a rate limit, or splits a backlog into small batches through the day, you can drop it. Upload the whole backlog as one file (up to 200 MB) with a deadline, and Otium holds it, retries what fails and delivers by the deadline.
• Quality. Send a representative sample through the new model and score it before you move the whole job.
• Cost. Check that the tokens and cost reported for each job match what you expected.

Move a slice first

You do not have to move everything at once. Because the switch is a change of address, you can send part of your batch work to the new service, compare cost and quality on real jobs, and move more as you gain confidence. If it does not work out, switching back is the same one-line change.

Once a job has moved, send all of it at once rather than in pieces. Otium waits until there is enough work to fill a machine before starting one, so a whole backlog starts sooner than a trickle. Requests that share instructions are served from cache once the first has run, and on Otium cached input is free, so the more arrives together, the less you pay.

If your batches run in steps

Many batch jobs are one stage of a longer chain: extract, then classify, then write. Sent as separate batches, each stage waits for the one before it, so a 24-hour window per stage becomes days for the chain. You can move each stage as it is and change nothing else.

Or you can send the whole chain as one workflow, with one deadline for every step. Each step is still an ordinary chat request, so the prompts you already have carry over. See how a workflow is written in the workflow docs, and the per-step output that comes back in the same OpenAI batch format.

Reference

Two documents are worth having open while you move: the API you are leaving, so you can check how it behaves rather than remember, and the file format both sides read.