Renting vs owning a GPU for open-weight models

Start with memory, not speed

A model runs on a GPU only if its weights fit in the GPU memory (VRAM), with some room left for the requests it is working on. So before comparing prices, work out how much memory your model needs. A good rough rule: the number of parameters, times the bytes each weight takes, times about 1.2 for the working space.

Memory is the fair way to pair a home card with a rented one, because it decides what you can run at all. Speed is the other half, and the market report gives it as memory bandwidth: how fast each side reads its memory, which for a model that fits sets how fast text comes out. An RTX 4090 reads its memory about 3 times as fast as the 1× L4 24 GB a cloud rents with the same memory. It cuts the other way too: the ASUS Ascent GX10, a small computer with 128 GB shared by its processor and GPU, holds models no single card can, but the 4× L40S 48 GB a cloud rents to match it read memory about 13 times as fast, so they finish the same work in far fewer hours. Where the two sides differ that much, the hours to break even are not the whole story.

The comparison: hours of renting per $1,000

Card prices move every week and differ by seller, so instead of guessing yours, turn the question around: how long does $1,000 rent the same GPU memory? Owning still costs electricity, so take off what the card would have used in that hour.

hours per $1,000 = 1,000 / (rental price per hour - card watts / 1,000 x electricity price per kWh)

A worked example from the September 2026 market report. RTX 3090: 350 W at most, and at the US average home rate of 18.31¢ per kWh that is $0.064 an hour. The machine matched to it, with the fewest GPUs holding 24 GB of memory, was 1× L4 24 GB at $0.990 an hour on-demand ($0.476 on spot). So $1,000 rented it on-demand for about 1,100 hours, which is about 50 days around the clock.

Electricity matters less than people expect. Even a 450 W card costs only cents an hour at the average rate. But rates vary a lot: in July 2026 they ran from 12.72¢ per kWh in Louisiana to 48.00¢ in Hawaii. Use the rate on your own bill.

The number that decides it: how much you will use the card

Multiply the hours per $1,000 by the card's price in thousands, and you have how many hours of use it takes before owning comes out ahead. Say the card would cost you $1,500 (put in your own price). Against the example above, that is about 1,600 hours: about 2 months running around the clock, or about 1.1 years at four hours a day. The market report's table does this sum for you: put in what each card would cost you and how many hours a day you would use it.

So the question is less "which is cheaper?" and more "how busy will the card be?". A card that runs batch jobs every night pays for itself much sooner than one used for an hour of chat in the evening.

The formula leaves things out on both sides. Write them down before deciding:

Owning costs more than the card

The rest of the computer, a bigger power supply, cooling, and the power it draws while idle. A card also keeps some resale value, which lowers its real cost.

Renting costs more than the hourly price

Storage for the model weights, time spent downloading them each time a machine starts, and data transfer. Small, but not zero.

Owning gives you things money does not measure

No queue, no sign-up, your data never leaves the room, and it works offline.

Renting gives you sizes you cannot buy easily

An 80 GB card, or several at once, for an afternoon. You can also try a big model before deciding whether to buy the card for it.

Spot or on-demand?

Clouds sell spare machines cheaply as spot capacity, often at half the standard (on-demand) price or less. The catch is that the cloud can take a spot machine back at short notice. That is fine for work that can stop and resume: batch jobs, evaluations, processing a pile of documents. It is a poor fit for a chat assistant you need to stay up.

So compare like with like. If you would use the home card for always-on, interactive work, the on-demand column is the fair comparison. If the work can wait and be retried, spot is.

Renting through Otium

If your large-model work is batch work that can wait, Otium does the renting for you. You send requests through an OpenAI-compatible batch API and choose a deadline; Otium waits for low-cost capacity, runs the open-weight model, and returns the results. You pay for the tokens processed, not for machine hours, so time spent starting machines or waiting is not on your bill.

Sources