Migrating from the OpenAI Batch API

How the batch flow works

The OpenAI batch pattern has three steps, and a compatible platform mirrors all three:

1. Upload a JSONL file where each line is one request (with a custom_id you choose and the request body).
2. Create a batch that references the uploaded file and a completion window.
3. Poll the batch until it completes, then download the output file — one result line per input line, matched by custom_id.

Because the request bodies on each line are the same chat-completion shape you already send, the part of your code that builds requests does not change. Only the endpoint and credentials do.

The base-URL swap

Most official OpenAI SDKs let you set the base URL and API key when you construct the client. Point them at the compatible platform and the same calls flow there instead:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.getotium.ai/v1",
    api_key=OTIUM_API_KEY,
)

# The file-then-batch calls are unchanged:
f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=f.id,
    endpoint="/v1/chat/completions",
    completion_window="7d",  # wider window, lower price
)

What to verify when you migrate

• Model names: the model field maps to the platform catalog. You can name a concrete model, or declare intent and let the platform choose.
• Completion window: pick the widest your workload tolerates to get the lowest price.
• Output parsing: results are matched to inputs by custom_id, so your join logic is unchanged.
• Quality: run a representative sample through the new model and score it before cutting over the whole pipeline.
• Cost reporting: confirm the per-job token and cost metrics the platform surfaces line up with your expectations.

Migrate incrementally

You do not have to move everything at once. Because the integration is a base-URL switch, you can route a slice of your batch traffic to the new platform, compare cost and quality against your incumbent on real work, and widen the share as confidence grows. Compatibility is what makes the migration low-risk and easy to reverse.

Reference

Two specifications are worth having open during a migration: the API you are leaving, so you can confirm behaviour rather than remember it, and the file format both sides speak.