EP 01 · THE PARALLEL MACHINE
Your screen has 8,294,400 pixels.
It can redraw every one of them 60 times a second.
In a modern game, each pixel needs hundreds to thousands of calculations.
That's about 0 calculations. Every second.
The chip doing it is made of sand.
ACT 1
One brilliant worker
A CPU core is a brilliant, lightning-fast worker. It follows a recipe, one step at a time. Here's the recipe for one pixel of a tiny 32×18 screen. Press play and watch it paint.
Click a pixel to inspect it. Hover (or tap) a line of the recipe to see what it looks at.
waiting…
For every pixel, the CPU works through these steps:
SUN = (20, 8) SUN_R = 4.2 SEA = 11That was 576 pixels × 8 steps = 4,608 steps. Now a real game, on your screen:
A CPU core running at 4 GHz does about 4 billion steps a second. That's roughly 0.5 frames per second. Your screen wants 60.
So just make the CPU faster? Try it.
A normal desktop CPU. Drag the slider →
ACT 2 · LEVEL 1
Build a chip
Click or drag to place. Click a part to remove it.
A CPU core is huge because it's smart. Most of it is brain: parts that guess which way your code will branch, reorder instructions, and juggle a dozen tasks. Then there's cache, a private stash of memory.
The part that actually does the math, the ALU, is one tile out of sixteen.
But every pixel follows the same recipe.
Why does every worker need its own brain?
An ALU (Arithmetic Logic Unit) is the circuit that actually does math: add, multiply, compare. Beside it sit a few registers, its own tiny notepad. Each ALU starts with its own pixel's x and y written in it.
The control unit shouts what to do. Every ALU does it to its own numbers. Same instruction, different data.
get ready…step 0 / 8Notice the if steps. When the condition is false for some ALUs, they just sit that step out. Nobody runs ahead, everyone stays in lockstep.
What can renderPixel depend on? Only its inputs: its own x, y, values that are the same for every pixel (here SUN and SEA; in a real game the time, the camera, the lights), and data it reads from memory (like textures). It outputs one color. It never peeks at another pixel's answer, so any pixel can run on any ALU, in any order. That's what makes it parallel.
ACT 3
You just invented the GPU
One control unit shouting the same step to a whole row of simple math units. Engineers call it SIMD: Single Instruction, Multiple Data. It's the core idea of every graphics card. Now scale it up.
- Each core: ~5–6 GHz
- Great at twisty, one-thing-after-another work
- ~2.5 GHz, in lockstep groups of 32
- ~80 trillion calculations per second
ACT 4
AI is a giant grid of multiply-adds
Inside a neural network, almost all the work is one operation: matrix multiplication. Each answer cell is a row times a column, multiplied and summed. Hover over (or tap) an answer cell.
Every answer cell needs only its own row and column. No cell waits for any other cell. Sound familiar? It's pixels all over again.
A tiny layer of 24 neurons, each with 16 weights (the columns of B), looking at 24 inputs of 16 numbers each (the rows of A). Every answer cell in C is one neuron's response to one input: 576 answers.
Watch the gold markers: that's step k, one multiply-add. The CPU walks one row and one column at a time. Your chip lights up many rows and columns at once, and all their gold markers move together, in lockstep.
function answerCell(i, j) → number sum = 0 repeat k = 1 … 16: ← one step each sum = sum + A[i][k] × B[k][j] return sum
1 CPU CORE
YOUR CHIP · 48 WORKERS
Training a GPT-4-class AI model takes roughly 20,000,000,000,000,000,000,000,000 operations (2×10²⁵). How long would that take?
Log scale. Rough estimates: CPU core ≈ 10¹⁰ ops/s; AI GPU ≈ 4×10¹⁴ ops/s in practice. 60 million years is roughly how long ago the dinosaurs vanished.
ACT 5
CPU or GPU?
16,384 workers are useless if they have to wait for each other. For each job, pick the better chip.
ACT 6
Why a GPU costs as much as a car
A flagship AI GPU like the H100 sells for around $25,000–$40,000. Tap each card.
We took sand.
Purified it until only one atom in a billion was wrong.
Grew it into a perfect crystal. Sliced it. Printed 80 billion switches on it with light.
Then we ran electricity through it, and it did math. Trillions of times a second.
We tricked rocks
into thinking.