Self-hosted workers

Before you start

An Otium account. First 30 days free, no credit card needed. Enrolling a machine needs an owner or admin.

A machine with a GPU, Linux (amd64 or arm64) or Windows (amd64), that can reach https://worker-api.getotium.ai. The worker only calls out: nothing has to listen on the internet.

An OpenAI-compatible inference server on the same machine: vLLM, llama.cpp's llama-server, or anything that serves /v1/models and /v1/chat/completions.

In the portal, open Self-hosted workers and choose Enroll worker. Pick the model the machine will serve and, if you like, a weekly capacity window: the hours it takes work, such as nights and weekends. You can change the window later.

The portal then shows the worker's token and its environment, once. Save them to a file on the machine, worker.env:

worker.env

OTIUM_WORKER_STANDING=true
OTIUM_WORKER_RUNTIME=vllm
OTIUM_RUNTIME_ENDPOINT=http://127.0.0.1:8000/v1
OTIUM_WORKER_API_URL=https://worker-api.getotium.ai
OTIUM_WORKER_TOKEN=owt_…
OTIUM_WORKER_MODEL=<the model you enrolled for>
OTIUM_WORKER_REGION=self-hosted

Keep the token secret: it is the machine's identity. The endpoint is vLLM's default port; change it if your server listens elsewhere.

2. Serve the model

The Workers page lists each model's weights: the repository, the revision, the quantization and the context length. Serve those weights, under the model's name: Otium sends every request with that name, and a server that does not know it answers 404. Listen on 127.0.0.1 so only the worker can reach it.

With vLLM:

vLLM

vllm serve <repository> \
  --revision <revision> \
  --served-model-name <model name> \
  --max-model-len <context length> \
  --max-num-seqs 16 \
  --host 127.0.0.1 --port 8000

With llama.cpp, for weights published as a GGUF file, and set OTIUM_RUNTIME_ENDPOINT=http://127.0.0.1:8080/v1:

llama-server

llama-server -m <model>.gguf \
  --alias <model name> \
  --parallel 8 --ctx-size <context length × 8> \
  --host 127.0.0.1 --port 8080

The worker keeps as many jobs in flight as your server can run at once. It reads llama-server's slot count by itself. With vLLM, set OTIUM_WORKER_CONCURRENCY to your --max-num-seqs; unset, the worker sends one job at a time.

3. Start the worker

On Linux, the container image (amd64 and arm64). --network host lets it reach a server on 127.0.0.1:

Docker

docker run -d --name otium-worker --restart unless-stopped \
  --network host --env-file worker.env \
  ghcr.io/getotium/otium-worker:latest

Or a binary from the releases page. Check it against the release's SHA256SUMS before you run it:

Linux

curl -fsSLO https://github.com/getotium/otium-worker/releases/latest/download/otium-worker_linux_amd64
curl -fsSLO https://github.com/getotium/otium-worker/releases/latest/download/SHA256SUMS
sha256sum -c SHA256SUMS --ignore-missing
chmod +x otium-worker_linux_amd64
set -a; . ./worker.env; set +a
./otium-worker_linux_amd64

Windows (PowerShell)

Get-Content worker.env | ForEach-Object {
  $k, $v = $_ -split '=', 2
  Set-Item "env:$k" $v
}
.\otium-worker_windows_amd64.exe

Run it under whatever keeps a service up on your machine (systemd, Docker's restart policy, a scheduled task), so it comes back after a reboot. Within a minute the Workers page shows the machine as Online.

Where your work runs

With Run work only on our own hardware off, the default, your machines take your work when they can, and Otium's cloud takes what they will not finish by its deadline: it learns how fast your machines go and launches only for the rest. Turned on, it runs on your machines only, and waits for them. A change applies to work submitted after it.

A worker takes your account's work and no one else's. Outside its capacity window it takes nothing and keeps checking in, and starts again when the window opens.

What it costs

$49 for 30 days, including 2 billion tokens, then $0.02 per million tokens, input and output alike, whatever the model. The 30 days start when one of your workers takes its first job, so time they sit idle between them costs nothing. Jobs your own workers run cost nothing on top.

Your first 30 days are free and need no balance until their tokens reach the allowance. After that, a new 30 days needs the fee in your balance and is charged as it starts, and tokens past the allowance are charged each night. While the balance is empty, your workers wait. Tokens are counted as the larger of what your worker reports and Otium's own estimate from the prompt and the result.

Stopping and removing a machine

Stopping the worker (Ctrl+C, docker stop, a shutdown) lets it finish what it holds, for up to five minutes, then exit. Give your service manager at least that long. The machine stays enrolled, and picks up where it left off when it starts again.

To remove a machine, choose Unenroll on the Workers page. Its token stops working at once, and a job it was running goes back to the queue.

If something is wrong

What you see	What to do
Exits at start	It says OTIUM_WORKER_STANDING is set without OTIUM_WORKER_RUNTIME=vllm. Add that line: without it the worker would answer each job with its own prompt, so it refuses to start.
Stuck waiting for the endpoint	It logs “waiting for vLLM endpoint” and goes no further: OTIUM_RUNTIME_ENDPOINT does not reach your server. Check the port, that it ends in /v1, and that a container runs with --network host.
Never Online	The machine cannot reach worker-api.getotium.ai over HTTPS, or the token is wrong. A token is shown once: if it is lost, unenroll and enroll again.
Online, no work	Outside its capacity window; nothing queued for its model; or the account’s balance is empty (past the free pass’s token allowance, or in a paid pass). With “Run work only on our own hardware” off, our cloud may also have taken the work.
Jobs fail with 404	Your server serves the model under another name. Set --served-model-name (vLLM) or --alias (llama-server) to the model name you enrolled for.

What leaves your network

Prompts and results pass through Otium's control plane, encrypted at rest, as they do for work on our cloud: your workers fetch each job and return its result over HTTPS. If your data must never leave your network, that is on-prem.