Data residency for AI inference: pick a boundary, not a region

Your obligation is jurisdictional. A region is not.

Whatever your data-protection commitment says, whether to a regulator, in a customer contract or in your own policy, it names a jurisdiction. Personal data stays in the EU. Records stay in Germany. Nothing leaves North America.

A cloud region is not that. It is one cloud provider’s data center in a place, and it means nothing to a second provider. Pinning to one region is too strict, because it shuts out other regions that would satisfy the same rule. It is also too vague, because a region name is not a legal commitment about where processing happens.

The better unit is a boundary. You declare the jurisdiction, and the platform works out which locations satisfy it. That way the same declaration keeps working when the platform adds another cloud, without you changing anything.

How boundaries actually work

One region can satisfy several boundaries at once, from broad to narrow. Giving each location a set of tags covers both broad jurisdictions and strict single-country rules with one mechanism:

A job declares one boundary, or none. With none, it runs wherever capacity is cheapest, which for work that is not sensitive is the point. With one, it runs only where that tag is present.

The question that separates a real lock from a label

The test is this: your job is locked to the EU, and right now there is no capacity in the EU. What happens?

There are only two honest answers. Either the job waits until its deadline and, if no capacity appears, expires without running. Or it runs somewhere else. A system that quietly does the second when it is busy has no residency guarantee at all. It has a preference, which is a different thing.

This is the question to ask in an evaluation. Ask what happens when the locked location has no capacity, and treat any answer that includes "falls back" as disqualifying for regulated work.

Where the lock has to be enforced

A residency guarantee is only as strong as the place it is checked, and one place counts: the moment work is handed to a machine.

• When work is handed out, by the system handing it out. A machine asks for work, and the central system decides whether this job may go to that location. The check belongs on the side the machine cannot tamper with.
• Not on the machine’s word. A machine that reports its own location and picks its own eligible work is asserting exactly what you are trying to verify.
• When machines are started, too. A system that starts machines in a location and then finds it cannot legally serve the waiting work wastes money, and creates pressure to relax the rule.

Keeping the map from region to jurisdiction in the central system, not on the machines, has a practical benefit too: short-lived machines need to know nothing about compliance, and the map can be corrected in one place when a rule changes.

What a lock costs you

Residency is not free, and a provider who says otherwise is not being straight with you. Narrowing where a job may run shrinks the pool of capacity it can use. A smaller pool means fewer cheap moments to catch and a higher chance of waiting.

The tighter the boundary, the sharper the trade: eu has many locations, eu-de has one. For work with a flexible deadline that is often an acceptable price, because a long deadline makes up for a thin pool. Decide deliberately rather than locking everything by default: most workloads are a mix, and only part of it is regulated.

What to ask a provider

1. Do I declare a jurisdiction or a region, and what does that declaration bind?
2. If capacity inside my boundary is unavailable, does my job wait, expire, or run elsewhere?
3. Where is the rule enforced: when work is handed out, or by trusting the machine?
4. Does the boundary cover everything, including where inputs and results are stored, not just where the GPU is?
5. How do stricter single-country boundaries work, and what do they cost me in price or waiting?
6. What happens to my locked jobs when you add a new cloud provider or region?

The answers tell you whether residency was designed in or added later.

Setting a boundary on Otium

On Otium you declare a boundary, never a region, when you create a batch. It locks every job in the batch, and every step of each workflow run in it:

{"input_file_id": "file-…", "endpoint": "/v1/chat/completions",
 "completion_window": "7d", "metadata": {"boundary": "eu"}}

The lock fails safe: if no capacity inside the boundary appears before the deadline, the job expires rather than running elsewhere. Eligibility is checked by Otium’s central system each time work is handed to a machine, not reported by the machine. A batch with no boundary runs wherever capacity is cheapest.

Sources

This guide argues from the shape of the obligation rather than from any one regime, because the shape survives a change of jurisdiction. If you want the underlying text, the rules on transfers are what make "which jurisdiction processes this" a legal question rather than a preference.