Paging & Storage: How Does One Machine Process 128 GB with Only 16 GB of RAM?

Concept. Disks and SSDs read in fixed-size pages (typically 64 MB in big-data systems). One IO costs the same whether you read one byte or a full page, so algorithms minimise page count, not byte count.

Intuition. How do you process 128 GB on a machine with 16 GB of RAM? You read it 64 MB at a time. The disk hands you a full page on every IO whether you want one byte or all of it, so algorithms count pages, not bytes, and the page that holds Mickey's listens costs the same whether you're after one row or fifty.

Paging: From CPU to RAM

The core computes out of its registers and L1/L2/L3 cache, which the hardware fills from RAM in 64-byte cache lines. Between RAM and the SSD the unit is bigger: data moves one fixed-size page at a time, and one page moved is one IO. Classic Unix and Postgres use 8 KB pages; big-data systems use 64 MB. From here on page means this unit, the database's page, and DB-page when it has to be told apart from the smaller one the operating system moves.

Three tiers left to right: a CPU working out of registers and L1/L2/L3 cache (filled from RAM in 64-byte cache lines, in nanoseconds), RAM (16 GB, 256 page slots) in the middle, and an SSD (128 GB, 2048 pages) on the right as a grid of same-size cells with one highlighted orange as it is paged into a free RAM slot, labeled one page, 64 MB, one IO, transfer-bound at about 5 GB/s so about 13 ms.

Figure 1. SSD-to-RAM transfers move one page per IO. One 64 MB IO costs the same whether the query needs one row or the whole block, and transfer time dominates, so moving 64 MB at about 5 GB/s takes ~13 ms. RAM-to-CPU transfers move 64-byte cache lines into L1/L2/L3 in nanoseconds, and the core executes out of cache and registers. RAM holds 256 pages (16 GB) and the SSD holds 2048 pages (128 GB), so the dataset passes through RAM one page at a time.

Why Paging Matters: The Speed Gap

Nanoseconds and milliseconds are hard to feel, so the cost of falling to disk stays abstract. Rescale every access as if a cache hit took one second, and the hierarchy turns visceral.

Four storage tiers rescaled as if an L1 cache hit took one second: cache 1 second, RAM 1.5 minutes (100x slower), SSD 3 hours (another 100x), hard disk 4 months (another 1000x). Paging keeps the working set in the fast tiers, so a read is a short pause, not a months-long wait.

Figure 2. Rescale latency so an L1 cache hit takes one second. RAM becomes 1.5 minutes, an SSD read becomes 3 hours, and a hard disk read becomes 4 months. Paging keeps the working set in RAM. A query that reads from disk on every page pays the hard disk or SSD cost once per page.

Storage Hierarchy: Speed, Cost, and Capacity

Faster storage costs more and holds less. RAM is about 100x faster than SSD and 100,000x faster than HDD, and price per terabyte runs the opposite way.

Four storage tiers as a descending staircase: cache and registers at ~1 ns, RAM at 100 ns, SSD 100x slower at 10 µs, HDD another 1,000x slower at 10 ms, with a left axis from faster/costlier/smaller down to slower/cheaper/bigger.

Figure 3. Cache, RAM, SSD, and HDD form a latency ladder. Each step down runs roughly 100x to 1,000x slower and costs less per terabyte while providing more capacity: RAM at 100 ns and $3,500/TB, SSD at 10 µs and $75/TB, HDD at 10 ms and $25/TB. An algorithm chooses a working set size that fits within a tier.

IO Cost Definitions

IO cost is two numbers: a fixed startup overhead, then a sustained transfer rate.

Access Latency: time to initiate an IO before any data moves

  • A fixed overhead paid once per IO, independent of size (the per-tier values are in the hierarchy above)

Throughput: the sustained transfer rate once data starts moving

  • RAM 100 GB/s, SSD 5 GB/s, HDD 100 MB/s (not shown above; this is the other half of the cost)

Key Insight: For large pages (64MB), transfer time dominates access time. For small pages, access time dominates.

Refresher: Read your OS materials on how the OS' IO controllers work. DBs rely on OS for those details.

Modern Reality: CPUs/GPUs Can't Escape the Disk Bottleneck

The 10,000,000× Gap

The gap between storage and compute is not 10x, it is 10,000,000x: one HDD seek (10 ms) is the time a CPU needs for ten million operations. Even with the data on a fast SSD, the slow part is the link that feeds the processor, not the math.

Two single-machine setups, a laptop and a server, each a vertical data path: grey compute (CPU plus GPU) at the top, a red-strip RAM box (volatile) in the middle, a green-strip NVMe SSD box (durable) at the bottom, joined by orange links labeled with bandwidth (PCIe ~32 GB/s, SSD ~5 GB/s). The server has more cores, RAM, and SSDs but the same per-link speed.

Figure 4. Link bandwidth caps end-to-end throughput inside one machine. The GPU does ~3x1014 ops/s, while PCIe moves ~32 GB/s and an SSD supplies ~5 GB/s. Reading a 1 TB dataset across the PCIe link to the GPU takes ~30 s, while computation over it takes ~0. RAM (red) is volatile and the SSD (green) is durable, and both connect through the same links. A server adds cores, RAM, and SSDs as additional devices and capacity, while link speed stays similar.

GPUs Are Data Hungry

The bottleneck chain:

  1. Data lives on disk - Your 1TB dataset won't fit in 80GB GPU memory (e.g. A100 has 80GB in 2025)

  2. PCI interfaces between hardware components are narrow - 32GB/s seems fast until you have 1TB to move

  3. Compute is free - 312 TFLOPS means compute takes ~0 time

Recommended: How Cache, Memory and Storage Work

The hardware behind this page, from cache and RAM down to the SSD.