What Is a System?
Concept. A system is components chosen to work with each other. Its speed is set by its weakest part and its cost by all of them. Systems design is picking the right trade-off for every subcomponent: fast enough for the work, and costing the least.
Intuition. Say you want a car that will do 150 mph on a road. A jet engine is plenty, but bolt it to a chassis made of Lego and the chassis fails. The Lego is the bottleneck, and you paid for the jet.
Figure: Pick one from each binThree bins of parts, then three cars built from them. Engines: E1 a motorbike good for 130 mph at five thousand dollars, E2 a turbo four good for 150 at twelve thousand, E3 a jet good for 400 at two million. Chassis: C1 Lego good for 2 mph at fifty dollars, C2 steel good for 150 at ten thousand, C3 carbon good for 250 at five million. Wheels: W1 road tyres good for 150 at six hundred, W2 sport at 200 for two thousand, W3 slicks at 250 for fifty thousand. The builds: E3 plus C1 plus W3 is mismatched and does 2 mph for two million and fifty thousand dollars; E3 plus C3 plus W3 is oversized and does 250 mph for seven million; E2 plus C2 plus W1 is right-sized and does 150 mph for twenty-two thousand six hundred.
| Engine | |||
|---|---|---|---|
| E1 | Motorbike | 130 mph | $5,000 |
| E2 | Turbo four | 150 mph | $12,000 |
| E3 | Jet | 400 mph | $2M |
| Chassis | |||
|---|---|---|---|
| C1 | Lego | 2 mph | $50 |
| C2 | Steel | 150 mph | $10,000 |
| C3 | Carbon | 250 mph | $5M |
| Wheels | |||
|---|---|---|---|
| W1 | Road | 150 mph | $600 |
| W2 | Sport | 200 mph | $2,000 |
| W3 | Slick | 250 mph | $50,000 |
| Mismatched | Oversized | Right-sized |
|---|---|---|
| E3 + C1 + W3 | E3 + C3 + W3 | E2 + C2 + W1 |
| 2 mph | 250 mph | 150 mph |
| $2.05M · C1 is the bottleneck | $7.05M · 100 mph past the goal | $22,600 · hits the goal exactly |
Figure 1. The goal is a car that will do 150 mph on a road. Pick one row from each bin; the car then runs at its slowest part and costs every part in it. E3+C1+W3 spends $2.05M and travels at walking pace: C1 is the bottleneck. E3+C3+W3 has no bottleneck, but it is oversized: it runs 100 mph past the goal, at $7.05M. E2+C2+W1 hits 150 on all three parts at once for $22,600.
Systems design is picking the trade-offs
Fast parts cost more than slow parts. Don't overpay for capability the weakest part will never let you use.
So the work is one decision per subcomponent, made against the work and against the other parts: how fast does this one have to be?
The same problem, for data
Data is stored in RAM, SSDs, disks, databases, and more machines: the fast parts costs orders of magnitude more per byte.
Figure: Same idea, for a data systemThe goal is to serve ten thousand requests a second. Three bins of parts. Storage: S1 disk at one thousand requests a second for two hundred dollars a month, S2 SSD at ten thousand for six hundred, S3 memory at a hundred thousand for four thousand. Database: D1 a local database at one thousand for two hundred, D2 a cloud database at ten thousand for two thousand, D3 a NoSQL cluster at a hundred thousand for fifteen thousand. Servers: P1 two nodes at one thousand for four hundred, P2 twenty nodes at ten thousand for three thousand, P3 two hundred nodes at a hundred thousand for thirty thousand. The builds: S3 plus D1 plus P3 puts memory and two hundred nodes behind a single local database, so the local database is the bottleneck and it serves one thousand a second for thirty-four thousand two hundred; S3 plus D3 plus P3 serves a hundred thousand, ten times past the goal, for forty-nine thousand; S2 plus D2 plus P2 hits the goal exactly for five thousand six hundred.
| Storage | |||
|---|---|---|---|
| S1 | Disk | 1,000 r/s | $200 |
| S2 | SSD | 10,000 r/s | $600 |
| S3 | Memory | 100,000 r/s | $4,000 |
| Database | |||
|---|---|---|---|
| D1 | Local | 1,000 r/s | $200 |
| D2 | Cloud | 10,000 r/s | $2,000 |
| D3 | NoSQL | 100,000 r/s | $15,000 |
| Servers | |||
|---|---|---|---|
| P1 | 2 nodes | 1,000 r/s | $400 |
| P2 | 20 nodes | 10,000 r/s | $3,000 |
| P3 | 200 nodes | 100,000 r/s | $30,000 |
| Mismatched | Oversized | Right-sized |
|---|---|---|
| S3 + D1 + P3 | S3 + D3 + P3 | S2 + D2 + P2 |
| 1,000 r/s | 100,000 r/s | 10,000 r/s |
| $34,200 · D1 is the bottleneck | $49,000 · 10x past the goal | $5,600 · hits the goal exactly |
Figure 2. Same bins, same arithmetic, and a goal of ten thousand requests a second. S3+D1+P3 puts everything in memory and rents two hundred nodes, then serves it all through one local database: D1 is the bottleneck, so the system does a thousand a second and pays $34,200 a month to miss the goal. S3+D3+P3 has no bottleneck and is still wrong, at ten times the goal for $49,000. S2+D2+P2 hits it exactly for $5,600. Storage, database and servers do the job the engine, chassis and wheels did.
Every system in this course, from a laptop running Pandas to the one serving eight hundred million people, makes this same choice at a different scale.
Takeaway
A system is not its best part. It is how the parts fit each other and the job. For the rest of this course those parts hold data, and the question never changes: how fast, and at what cost.