Who is actually running your inference?

The plaintext window

Encryption in transit protects your prompt on the wire. Encryption at rest protects it in storage. Between those two, there is a window in which the prompt, the model weights, and the generated result all exist in plaintext in the memory of a specific machine, because that is the only form in which inference can happen.

Anyone with root or physical control of that machine can read all three out of process memory or GPU memory. No amount of TLS changes this. It is not a flaw in a particular product; it is a property of running a model at all.

A gradient, not a binary

Compute is not simply trusted or untrusted. It sits on a gradient defined by who has control of the box — which, notably, is not the same thing as where the box physically is.

The middle rows are where most of the market lives and where the labelling is weakest. A provider renting anonymous marketplace GPUs and a provider running its own fleet can describe themselves in identical language, and the difference only shows up if you ask.

Location is a red herring here. A machine in a rack the provider owns and a machine in their cloud account are the same trust class, because the same organisation controls the image, the operator credentials, and the hardware. A machine in a professional datacenter operated by a stranger is a different class entirely.

Two things that must be declared separately

A sound design keeps two attributes apart, because they come from different parties and neither can be inferred from the other.

Data sensitivity

A property of the job. Only the customer knows it, so the customer declares it — with a conservative default.

Machine trust

A property of the capacity. Only the platform can assess it, so the platform assigns it. It must never be self-reported by the machine asking for work.

They meet at one point: placement. The rule that follows is simple to state and should be stated plainly by anyone who operates mixed capacity.

Never send a job to capacity less trusted than its data sensitivity requires. The rule sets a floor, not a preference — trusted capacity may still run low-sensitivity work when it is the cheapest thing available.

Why the default decides everything

The interesting failure mode is not a broken rule; it is a rule introduced later. A platform that runs entirely on its own hardware today and adds cheaper third-party capacity next year has to answer a hard question: what happens to all the jobs submitted before the concept of a trust tier existed?

If the default sensitivity is permissive, those jobs silently become eligible for the new, cheaper, less trusted machines. Nobody is notified, because from the system's point of view nothing changed. If the default is conservative, they stay where they were and the cheaper capacity is opt-in.

What the cheapest capacity really costs

Anonymous GPU marketplaces are genuinely cheaper, and for the right workload they are a legitimate choice. Public datasets, synthetic data generation, benchmarking, open-source model evaluation — none of that needs a trusted host, and paying for one is waste.

The mistake is treating that discount as available to everything. For business data the relevant question is not whether the operator seems reputable but whether you would be comfortable explaining the arrangement after an incident. There are also failure modes beyond confidentiality: an untrusted host can return plausible but fabricated results, which is why verification matters as much as placement when the host is not accountable.

Where Otium stands, precisely

Every job today runs on capacity in our own cloud accounts or on hardware we own, launched from images we build. There is no anonymous marketplace capacity in the fleet, so there is currently no situation in which your data reaches a machine controlled by someone else.

The tiered placement gate described here — declared sensitivity, assigned machine trust, a conservative default, and a hard floor enforced when work is handed out — is a published design rather than a shipped feature, because we do not yet operate capacity that needs it. We would rather describe it accurately in advance than discover the policy after the capacity.

Two things are enforced today: work only goes to machines inside a job's declared residency boundary, and payloads are encrypted per job under a per-customer key, with the encryption code published so the claim can be read rather than believed.

What to ask a provider

1. Whose hardware runs my job — your own account, a named partner, or an anonymous marketplace?
2. Can that change without notifying me, and what is the default for work I already submitted?
3. How is a machine's trust level established? Is it assigned by you, or reported by the machine?
4. What is the default sensitivity for a job that does not declare one?
5. What stops a lower-trust machine from leasing a higher-sensitivity job, and where is that enforced?
6. Which of these are shipped today, and which are designs?

That last one is the question most worth asking, and the one most likely to be answered vaguely. A provider that distinguishes clearly between what it runs and what it plans is telling you something useful about everything else it says.

Sources

The plaintext window is a property of how inference works rather than a finding anyone had to publish, so there is no single paper to point at. Two references are still worth having: the legal frame that makes "whose machine" a question you may be obliged to answer, and the demonstration that data reaching a model is not always recoverable-in-theory but gone-in-practice.