Used GPUs Over $800 for Local AI: The Cost-per-GB Collapse and the $3,500 Mining-Card Gamble
~$99 per GB of VRAM the number that matters here
By Stan K.
Used-US sold medians, human-verified 2026-11-10 (online ask/BIN runs higher): above $800 each rung roughly doubles the cost per GB — ~$50 (3090), ~$99 (4090), ~$216 (5090). What each tier buys, why the value collapses, and the CMP 170HX unlock story with its 2-in-5 failure rate.
Above $800, every rung of the used-GPU ladder roughly doubles what a gigabyte of VRAM costs. The budget tier bottoms out near $27 per GB (two used 3060s); a used 3090 is ~$50; a used 4090 is ~$99; the new-flagship tier stops pretending. Above $800 you are no longer buying capacity — you are buying speed, slots and single-card simplicity. Usually that is worth it. Usually.
The ladder, rung by rung
The 16GB rung ($780–1,400). RTX 5060 Ti at ~$780 ($617–910 — it straddles the $800 line from below), RTX 4070 Ti Super at ~$1,150 ($950–1,250), RTX 5070 Ti at ~$1,400 ($995–1,650). Verified speed data is sparse here: a 5060 Ti reports 55 t/s on a 27B-class model at iq1_m, and four 5060 Tis in sglang do 80–90 t/s FP8 with ~3000-token prefill. For the 4070 Ti Super and the 5070 Ti there is no verified tokens-per-second report yet — treat seller claims accordingly.
The 24GB rung ($1,199–2,387). The used RTX 3090 at $1,199 (sold median, $908–1,343) is the tier’s anchor, and the AI premium is visible: the same card traded at $700–900 not long ago. It delivers ~80 t/s on the 27B class. The RTX 4090 at $2,387 (sold median, $2,144–3,000) delivers ~92 t/s on 27B GGUF and ~130 with NInfer + MTP3. Multi-GPU batching is where 24GB pairs pay off: 2× 3090 in vLLM INT4 reports ~200 t/s at 3–4 concurrent users with 10k-token prefill (the full pair-vs-single duel).
The why-not rung. RTX 5090: $6,919 new at the scalped street median (retail range $6,549–9,490) for 32GB — ~$216 per GB. RTX PRO 6000: $16,399 — ~$171 per GB. The PRO 6000 is a capacity purchase (96GB in one slot, no dual-card plumbing), and as value it is not one: cost per GB runs 2–4× the 3090 rung. These cards exist for people whose constraint is slots, not dollars.
The math, concretely
Sold medians ÷ usable GB, human-verified 2026-11-10 (new retail for the last two), with the budget-tier baseline for scale:
- 2× 3060 — $638 ÷ 24GB = $26.58/GB (the baseline)
- 3090 — $1,199 ÷ 24GB = $49.96/GB
- 4090 — $2,387 ÷ 24GB = $99.46/GB
- RTX PRO 6000 — $16,399 ÷ 96GB = $170.82/GB (new)
- 5090 — $6,919 ÷ 32GB = $216.22/GB (new, scalped)
Even the cheapest verified 4090 sold ($2,144) lands at $89 per GB — still more than 3× the 3060 pair’s $27. That multiple is the “4090 tax”: you pay it for ~92–130 t/s in one slot, and only a verified workload can tell you whether it is worth it.
~$99per GB on a used RTX 4090 (sold median)
Nearly four times the 3060 pair’s cost per gigabyte for the same 24GB — the price of ~92 t/s on the 27B class in a single slot, before the market’s AI premium moves again.
CMP 170HX: 64GB of datacenter silicon, with a body count
The strange one. The CMP 170HX is a GA100/A100 die with HBM2e that shipped as a mining card: 8 or 10GB visible, with memory-controller regions disabled in software because of defective VRAM. CMP Unlocker (a GitHub project shipping a patched 610.43.03 driver) re-enables the hidden VRAM, compute and ECC: ~40GB stable, with mass reports stable at 64GB (80GB remains unstable). The performance is real: an unlocked 64GB card runs the 27B class at 97 t/s in vLLM W8A8 + MTP3 — 97–138 t/s across reports, faster than a 3090 — overclocked reports reach 202 t/s, and a pair runs GLM-5.3-Flash at 384K context at 90 t/s.
The market noticed: ~$500 on Alibaba in August 2026, $1.5–2.3k in September, roughly double again by November. Today the card trades at $2,500 with the 40GB unlock and $3,500 with 64GB (sold medians, human-verified 2026-11-10).
In one operator batch, 2 of 5 cards died inside two weeks — the defective-VRAM bins the unlock re-activates are defective for a reason. Add PCIe Gen2 x4 only (slow model loads), no display outputs (a server/AI box, not a desktop) and no standard cooler mounting — it expects its own turbo fan. This is a rack experiment, not a workstation card.
If you need 40–64GB for a few hours a month, renting beats gambling — the break-even math lives in GPU rent vs buy.
The rule that survives price changes: step up one rung only when a verified workload needs that rung. The doubling pattern holds even while any particular price is already stale. Occasional heavy hours belong to rental; always-on token volume belongs to the rental-vs-API math; the sub-$800 tier starts here, and every card sits in the Local GPU AI hub.
Prices are used-US sold medians, human-verified 2026-11-10 (new retail for the 5090 and PRO 6000); online ask/BIN runs higher. They move weekly — recompute before you buy.