ToolScore
Local GPU AI 5 min read

Rented GPU vs API: $0.27 Per Million Tokens — When Each One Actually Wins

~$0.27 per 1M output tokens the number that matters here

By Stan K.

A rented 2x RTX 4090 pod serving 985 t/s costs about $0.27 per 1M output tokens at vast.ai median rates — the per-token math against OpenRouter pricing nobody has published, plus the honest privacy picture of timeshared GPUs.

A rented pair of RTX 4090s serving 985 tokens/sec under load costs about $0.27 per million output tokens — just under the $0.28 output price of a cheap 32B API, and far under $1.40 frontier-adjacent output. But that number only holds while the batch queue is full. Empty the queue and the same pod costs $1.48+. Here is the full math, and the privacy fine print.

Privacy: where your tokens physically go

A rented GPU is a VM on someone else’s machine, frequently timeshared with strangers. The forum consensus for sensitive workloads is concrete: demand bare-metal or MIG-isolated instances, a private VPC with no public IP, disk encryption under your own keys, and GPU passthrough rather than timeshared vGPU; for genuinely sensitive data there are confidential-compute inference options such as privatemode.ai. The one thing we cannot print is an incident report: no documented cross-tenant leak from a rented GPU box surfaced in our recon — treat the caution as structural, not as a track record of breaches.

An API or subscription is the opposite trade: your data leaves your control entirely, governed by a terms page. For code and documents that must not move, neither API nor a timeshared pod qualifies — that work belongs on local hardware (see our budget GPU guide under $800).

Throughput: what a rented box actually serves

Under real concurrent load with vLLM/sglang, community-reported aggregates:

  • 2x RTX 3090 (27B INT4, vLLM): ~200 t/s at 3–4 concurrent users — it saturates there — with 10k+ t/s prefill.
  • 2x RTX 4090 (35B-A3B AWQ, sglang): 985 t/s at 8 concurrent, ~180 t/s single-stream, 9–10k t/s prefill.
  • 4x RTX 5060 Ti (27B FP8): 80–90 t/s decode plus 3,000 t/s prefill.

Prefill is the hidden weapon: these pods ingest prompts at thousands of tokens per second — roughly an order of magnitude beyond what API tiers deliver as input throughput. If your jobs are context-heavy, the rented GPU reads the book while the API is still opening it.

The math, concretely

Nobody has published a direct per-token cost for a rented vLLM pod, so here it is. A 2x 4090 pod rents at about $0.96/hour (2 × the $0.48 vast.ai median, checked 2026-11-10). The 985 t/s figure is aggregate output at 8 concurrent users:

  • Saturated: 1M output tokens take 1,000,000 ÷ 985 ≈ 1,015 s ≈ 0.282 h → 0.282 × $0.96 ≈ $0.27 per 1M output tokens
  • Single-stream: at ~180 t/s, the same million tokens take ~1.54 h → 1.54 × $0.96 ≈ $1.48 per 1M
  • API comparators (OpenRouter, per 1M): qwen3-32b $0.08 in / $0.28 out; glm-5.3 $0.07 in / $1.40 out with cached reads at $0.07
Cost of 1M output tokens — rented pod vs API list prices
2x4090, 8 users — $0.96/h ÷ 985 t/s $0.27
API qwen3-32b — output list $0.28
API glm-5.3 — output list $1.40
2x4090, 1 user — $0.96/h ÷ 180 t/s $1.48

Reading it honestly: a saturated vast-median pod lands a cent under the cheap API’s output price ($0.27 vs $0.28) and undercuts glm-5.3 output by about 5x. But the pod bills by the hour whether tokens flow or not, while API input is nearly free ($0.07–0.08 per million) and cached reads at $0.07 make repeated context almost nothing. The model classes differ too — the pod figure is a 35B MoE quant, the API figures are a 32B dense and a frontier-family MoE — treat this as same-weight-class economics, not a controlled benchmark.

~$0.27per 1M output tokens, pod saturated

Two rented 4090s at $0.96/hour (vast.ai median) pushing 985 t/s in aggregate. The same million tokens on one stream cost $1.48 — utilization, not hardware, decides whether the pod wins.

Plans: the third leg

Subscriptions compress agent workloads even further. GLM’s Coding Plan Lite starts at $18/month, and a measured community bench (November 2026) found a $10/month plan did 3–5x more work per dollar than $10 of API credits under daily agent usage — the trade-off being roughly a second of extra latency before each response starts. Claude ($17–20 Pro, $100 Max) and Cursor ($8–200 tiers) sell the same shape: predictable monthly cost, no queue to keep full. The forum volume rule still holds: under ~1B tokens/month, API or a plan wins outright; the rent/own crossover sits between 40% and 60% sustained utilization.

The takeaway

One asymmetry worth naming before the geometry: buying leaves residual value, renting leaves none. A card you own can be resold after a few years — used GPUs hold meaningful resale value (the used market is literally where our budget guides shop), so the true cost of ownership is the purchase price minus the resale price, not the sticker. Rented hours, like API tokens, are gone when spent. The counterweight: an owner carries the hardware risk — a card that dies out of warranty is your loss, while a renter just spawns a fresh instance. Net: resale value quietly narrows the own-vs-rent gap in favor of owning; failure risk quietly widens it back.

Prices will drift; the geometry won’t. Privacy-constrained work goes local or to isolated bare metal; cache-heavy and spiky work goes to APIs and plans; only sustained, batched, high-utilization throughput justifies a rented GPU — and then it wins by a lot. If you are weighing the pod against owning the cards instead of renting them, that math lives in GPU rent vs buy; for the cards themselves, the over-$800 used-GPU guide and the 2x 3060 vs 3090 duel map the hardware side. All of it is collected in the Local GPU AI hub.

Rental rates from vast.ai median on-demand pricing, API prices from OpenRouter’s model API, throughput from community vLLM/sglang reports — all checked 2026-11-10.

Numbers change — the math doesn't
Token prices and plan limits shift monthly; the structural arguments in this guide age slowly. For current per-tool numbers see the live rankings.