Renting a GPU in the cloud — and why you probably want serverless instead

There are three ways to buy someone else's compute, and most people only know the first and the last. The middle one is the interesting part.

Per-token endpoints are what most of this site prices: you send a prompt, pay for the tokens, and never think about hardware. The economics work because the provider packs thousands of users onto each card — their utilisation problem, your low price. For almost everyone, almost always, this is the right answer, and the catalogue exists to find the model and the price.

Renting a GPU by the hour is the other thing people have heard of, and it is the expensive, technical option — usually for reasons that only show up after you've committed. The bill runs whether the card is working or idle, and an inference workload is idle most of the time, so your effective price per token can be many times the per-token rate for the same model. And the software stack — drivers, a serving framework, model updates, keeping it alive — is now your job. A rented GPU is a good deal in one situation: when you can keep it genuinely busy, or when you need what it uniquely offers.

What it uniquely offers is control. Your own fine-tuned model that nobody hosts. A retention story you can state in one sentence: single tenant, my logs, nobody else's policy. Latency you tune yourself. If one of those is your actual requirement, the cost and the ops burden are the price of it — just make sure the requirement is real before paying it.

Serverless GPU is the middle rung most people don't know exists. Platforms in this space give you a GPU that spins up when a request arrives, bills by the second, and disappears after — custom model, no idle cost. The catch is the cold start: the first request after a quiet spell waits for the model to load, which can be seconds to minutes for large models. For bursty workloads on a model nobody hosts, it beats both neighbours; for steady traffic, per-token or a busy rented card wins.

The honest decision order. Is the model you want already hosted? Use the per-token price — that fight is won. Custom model, bursty use? Serverless GPU. Custom model, heavy steady use, or a privacy requirement you can articulate? Rent the card, and budget for the person who runs it — even when that person is you.

Where next: Hosted models and prices · Who can see your prompts · What inference is