Who is actually running your inference?
Every AI provider will tell you your data is encrypted in transit and at rest. Both are true, and neither covers the one moment that matters: to produce an answer, the model has to read your prompt as plain, readable text. So the real question is who controls the machine doing the reading.
The plaintext window
Encryption in transit protects your prompt while it travels. Encryption at rest protects it while it is stored. In between, there is a window when the prompt, the model, and the answer all sit unencrypted in the memory of one specific machine, because that is the only form in which a model can work.
Anyone with full administrative or physical control of that machine can read all three straight out of memory. No amount of encryption on the network changes this. It is not a flaw in any particular product; it is how running a model works.
A gradient, not a binary
Machines are not simply trusted or untrusted. They sit on a scale set by who controls the machine, which is not the same as where it physically is.
| Who controls the machine | What that means for your data | Reasonable use |
|---|---|---|
| The provider, in their own account, from their own software image | Exposure is limited by that provider’s own controls | Business data, by default |
| A vetted partner with a commercial relationship | Exposure extends to a named third party you can hold accountable | Business data, by explicit opt-in |
| An anonymous operator on a public marketplace | Assume the operator can read your prompts, the model and the results | Public or synthetic data only |
Most of the market sits in the middle rows, and that is where the labelling is weakest. A provider renting anonymous marketplace GPUs and a provider running its own machines can describe themselves in identical words. The difference only shows up if you ask.
Location is a distraction here. A machine in a rack the provider owns and a machine in the provider’s cloud account are in the same class, because the same organisation controls the software, the credentials and the hardware. A machine in a professional data center run by a stranger is a different class entirely.
Two things that must be declared separately
A sound design keeps two things apart, because different parties know them and neither can be worked out from the other.
- Data sensitivity
- A property of the job. Only the customer knows it, so the customer declares it, and an undeclared job is treated as sensitive.
- Machine trust
- A property of the machine. Only the platform can judge it, so the platform assigns it. A machine asking for work must never report its own trust level.
The two meet at one point: deciding where a job runs. The rule that follows is simple, and anyone running a mix of machines should state it plainly:
Never send a job to a machine less trusted than its data requires. The rule sets a floor, not a preference: a trusted machine may still run low-sensitivity work when it is the cheapest one available.
Why the default decides everything
The dangerous case is not a broken rule but a rule added later. Imagine a platform that runs only its own machines today and adds cheaper third-party machines next year. What happens to every job sent before trust levels existed?
If the default is permissive, those jobs quietly become eligible for the new, cheaper, less trusted machines. Nobody is told, because as far as the system is concerned nothing changed. If the default is conservative, they stay where they were, and the cheaper machines are something you opt into.
What the cheapest capacity really costs
Anonymous GPU marketplaces really are cheaper, and for the right work they are a fair choice. Public datasets, generating synthetic data, benchmarking and testing open models need no trusted machine, and paying for one is waste.
The mistake is treating that discount as open to everything. For business data, the question is not whether the operator seems reputable but whether you would be comfortable explaining the arrangement after an incident. There is also a risk beyond reading your data: an unaccountable machine can return plausible but invented results, so checking the output matters as much as choosing the machine.
What to ask a provider
- Whose hardware runs my job: your own account, a named partner, or an anonymous marketplace?
- Can that change without telling me, and what happens to work I already sent?
- How is a machine’s trust level set? By you, or by the machine itself?
- How is a job treated if it does not declare its sensitivity?
- What stops a less trusted machine from taking a more sensitive job, and where is that checked?
- Which of these work today, and which are plans?
The last question is the one most worth asking, and the one most likely to get a vague answer. A provider that separates clearly what it runs from what it plans is telling you something useful about everything else it says.
Where Otium stands, precisely
Every job today runs on machines in our own cloud accounts, started from software images we build. There are no anonymous marketplace machines in the fleet, so your data never reaches a machine someone else controls.
Before we add any capacity we do not control, the rules described here are the bar it has to meet: declared sensitivity, trust assigned by us, a conservative default, and a hard floor checked when work is handed out. If we ever add such capacity, we will say so first.
Two protections apply today. Work goes only to machines inside a job’s declared residency boundary, checked by our central system rather than reported by the machine. And each job is encrypted with a key that belongs to your account, using encryption code we publish, so you can read the claim rather than believe it.
Sources
The plaintext window follows from how a model runs, so there is no single paper to point to. Two references are still worth having: the legal frame that may oblige you to answer "whose machine?", and the evidence that data a model has seen can come back out.
Frequently asked questions
Does confidential computing solve this?
It is the most promising direction: hardware-protected memory that even the machine’s operator cannot read, with a way to prove it. Support across GPUs and providers is still uneven, so today it narrows the problem on specific hardware rather than removing it.
Is a homelab or self-hosted machine less safe than a cloud region?
Not in itself. What matters is who controls the software, the credentials and the hardware, not whether the building is a data center. A machine the provider owns and runs is in the same class wherever it sits; a machine run by an anonymous third party is not, however professional the facility.
If my data is encrypted per job, does the machine still see it?
Yes, at the moment the model runs. Per-job encryption limits the damage: a compromised machine exposes only the job it holds, not the rest of your data or other customers’. But the model still reads that one job as plain text. Encryption limits the damage; choosing the machine decides who is in a position to do it.
Related guides
Is your data retained, trained on, or readable by someone else? The privacy questions that matter in AI inference, and how open models change the answers.
Compliance obligations are jurisdictional, not regional. Why a residency boundary is the right primitive for AI inference, and how a lock must behave.
A model file can run code when it loads, a repository can change under you, and weights can carry a trigger. What stops each attack, and what nothing stops yet.
Open-weight models are competitive on many batch tasks. The real trade-offs in cost, control, privacy and quality, and how to measure them on your workload.