ToolScore
Data & Files 4 min read

SQLite Carried Our Data Platforms for Years

0 database servers, 3 platforms the number that matters here

By Stan K.

Three production data platforms, one database file each, zero database servers. What years of SQLite under real load taught us about the single-writer limit, busy_timeout, and the moment you actually need Postgres.

We run several production data platforms — market research scanners processing tens of millions of records, a public-records collector, the data layer behind this site’s research. Their databases share one property: each is a single SQLite file. No database server, no connection pooler, no HA cluster, no database line on any invoice. Years in, this is what the experience actually teaches.

What the single-file model buys you

The backup story is trivial. A database that is one file is copied like a file. Our snapshots are cron jobs, not pg_dump choreography (see backups you already have — SQLite is the star of that inventory).

Zero-latency reads. SQLite is in-process: no network round-trip, no wire protocol, no pool. On an NVMe drive a single node routinely serves tens of thousands of simple reads per second — a number that would embarrass a networked database doing the same query over localhost.

No ops surface. No server to patch, no auth to rotate, no memory tuning, no 3am connection exhaustion. The failure modes of SQLite are the failure modes of your disk, which you’re monitoring anyway.

Portability. The whole platform state moves with one file — between machines, into backups, into a laptop for offline work.

The single-writer limit, in practice

The famous constraint: SQLite allows one write transaction at a time (officially documented — WAL mode permits concurrent readers alongside the single writer, but not concurrent writers). On paper that sounds fatal for anything serious. In practice, for our workloads:

  • Writes are ingest-shaped: batches appended by a collector process. One writer, no contention.
  • PRAGMA busy_timeout = 5000 (a convention — 5 seconds of patient retrying) absorbs the rare collision between the writer and a maintenance job.
  • We have never hit a wall. Our read-heavy dashboards and single-process ingest pipelines simply never produce the contention the limit forbids.
When the wall is real

The honest thresholds — community heuristics, not official SQLite doctrine: sustained hundreds of short writes per second are fine with busy_timeout; ~1,000+ concurrent write transactions per second, or multiple app servers that must write to the same database, are Postgres territory. So are Row-Level Security, PostGIS and pgvector when you need them.

The decision we re-derive every time

Every new service starts SQLite and stays there until a specific requirement arrives:

Requirement arrivesAnswer
A second process must write concurrentlyone writer process + queue, or move
Multiple servers, same dataPostgres — at that point it’s not a debate
Vector search, GIS, RLSPostgres (pgvector/PostGIS/RLS)
Serverless functions needing the DBTurso-style embedded replicas, or Postgres
“Wouldn’t a cluster be more professional?”No. Boring is professional

The last row is the important one. The industry default — “real product, therefore managed Postgres” — spends $25+/month and real ops attention on problems a single file solves for free at the scale most products actually live at for years. Modern frameworks agree: Rails 8 and Laravel now ship SQLite defaults.

The escape hatch we like

When a SQLite platform genuinely outgrows the file, the cleanest path is Turso-style embedded replicas (libSQL): local SQLite file for reads, writes synced to a primary — microsecond local reads with networked durability. One honest caveat from the current landscape: LiteFS, the other project in this space, had its Cloud service retired in 2024 and sits outside Fly.io’s support scope — we wouldn’t start new production on it. And Turso itself joined Supabase in October 2026, so watch where that consolidation lands before betting on it.

The full managed-versus-self-hosted comparison — including the Hetzner-CX23-for-€6 Postgres route and the real cost of “free” database admin time — is in Postgres on a VPS vs $25 Managed, and the per-provider pricing at a fixed workload lives in the Databases hub.

Numbers change — the math doesn't
Token prices and plan limits shift monthly; the structural arguments in this guide age slowly. For current per-tool numbers see the live rankings.