Case Study 1.2: How ChatGPT Scales PostgreSQL
Concept. ChatGPT runs on PostgreSQL with one primary and nearly 50 read replicas. Every write goes to the primary. Reads, which are most of the traffic, go to the replicas.
Intuition. Think of the Listens table at a much larger scale. A row is written once, then read again and again. Across roughly 800 million users, reads pile up much faster than writes.
The previous case study showed how governed definitions and tests guide an agent's SQL. Underneath that stack sits an operational database. ChatGPT's is PostgreSQL.
One Chat Request, Two Paths
OpenAI describes this setup in "Scaling PostgreSQL to power 800 million ChatGPT users". The app server sits between you, the database, and the model. Every write goes to one primary, which keeps PostgreSQL's ACID guarantees. The primary sends an update stream (its write-ahead log) to nearly 50 read replicas spread across regions, and reads go to those replicas. OpenAI keeps the replication lag, how far a replica trails the primary, near zero. The workload is mostly reads, so adding replicas adds capacity where the traffic is.
Figure: One chat request, two paths through PostgresA user sends a message or opens a chat. The ChatGPT app server calls the database and the model; the model runs on GPUs, gets the context, and returns a reply in seconds. Writes, orange, go to one Postgres primary with ACID guarantees. The primary sends an update stream, its write-ahead log, to every read replica. Reads, blue, go to the replicas, nearly 50 of them with several in each region, and take milliseconds each.
Figure 1. The post does not say which ChatGPT data lives in PostgreSQL, so the figure labels the traffic only as reads and writes. The model box is how chat apps generally work, not a detail from the post. Some reads stay on the primary because they are part of a write transaction.
Module 5 covers sharding and replication. Module 4 covers transaction guarantees.
Takeaway: ChatGPT scales a read-heavy PostgreSQL workload by sending writes to one primary and distributing reads across nearly 50 replicas.