Source: https://oliviercouthaud.com/experiments · Status: Prototype · 2026 · Tags: Python, Simulation, Compilers, Machine learning

# Small experiments

Twelve from-scratch builds: a programming language, a database, a chess engine, a renderer, a tiny operating system and a few more.

Figure: The cloth: a 20 by 20 grid of particles joined by constraints, top two corners pinned, falling under gravity. The clip ends before it settles.

## Why I built them

I wanted to see what could get built in a single quick session, how much creativity would come out of it, and whether anything along the way would inspire a project of its own. To give it some shape I picked twelve things I use all the time, a programming language, a database, a chess engine, a renderer, a tiny operating system and a few others, and built the core of each myself instead of importing a library that already does it. They are not products. Each is a few thousand lines written to reach the interesting part and stop. They were made in April 2026 and come to about 45,000 lines of Python (blank lines and comments included), plus about 11,500 lines of C, assembly and Lune.

The rules were loose. I still used numpy, scipy, matplotlib, pygame and Pillow for arrays, plots and windows, and the compiler and OS experiments use gcc, nasm and ld to assemble and link. The two learning experiments train on public data, the MNIST digits and the Tiny Shakespeare text, which the scripts download. Neither is mine.

## How they fit together

Several of them ended up depending on each other. Click a box to see what it is. On a phone, use the buttons under the diagram.

**1. Lune.** A small scripting language with closures, pattern matching and a pipe operator, implemented four ways: a tree-walking interpreter, a bytecode VM, a transpiler to C, and a lexer, parser and compiler written in Lune itself. 8,595 lines of Python, plus the Lune and C programs around it. Its output feeds the C compiler (6) and the WebAssembly compiler (12).

**2. Rendering.** A ray tracer with diffuse, metal and glass materials, a BVH, triangle meshes, Perlin-noise terrain and a Cornell box, plus a software rasterizer and a progressive real-time path tracer in pygame. 4,404 lines of Python. Stands alone.

**3. Neural nets.** A framework on numpy with hand-written backward passes, and on top of it a CNN, a VAE, a character-level GPT, a DQN agent and an AlphaZero attempt for chess. 6,810 lines of Python. The AlphaZero part imports the chess board from 8.

**4. 2D physics.** Circles, walls and distance constraints with Euler or Verlet integration. Runs the cloth, a 30-link chain, a billiards break and a Newton's cradle, and there is an Asteroids clone on top of it. 2,223 lines of Python. Stands alone.

**5. N-body and sound.** Gravity with a fourth-order Runge-Kutta integrator, and a from-scratch audio side: oscillators, envelopes, filters, a 16-step sequencer and Markov-chain melodies. The two meet in a piece that plays an orbit as sound. 2,348 lines of Python. Stands alone.

**6. C compiler.** A subset of C (int, char, pointers, arrays, loops, printf) to x86-64 assembly, with a type checker, constant folding, dead-code removal and a peephole pass, plus a JIT experiment. 3,004 lines of Python. It takes the C that Lune transpiles to, after a small post-processor strips it down.

**7. Database.** A B+ tree on 4 KB pages with a write-ahead log, a SQL parser and executor with indexes and joins, snapshot isolation, and a Raft implementation over localhost sockets. 6,588 lines of Python. It stores the chess app's game history in 9.

**8. Chess engine.** Move generation, iterative-deepening alpha-beta with a transposition table, and a UCI mode for chess programs. 2,449 lines of Python, including its own AlphaZero attempt. It is the opponent in 3 and the engine behind 9.

**9. Web server.** HTTP/1.1 on raw sockets, a router, a small template engine, and HTTP/2 with frames, header compression and multiplexed streams. A chess web app sits on top and uses 7 and 8. 2,448 lines of Python. The crypto code in 12 wraps its sockets.

**10. MiniOS.** A 512-byte bootloader in assembly and a 32-bit kernel in C: screen output, interrupts, keyboard, a shell, a memory manager, paging, a heap and a read-only FAT12 driver. About 1,700 lines of C and assembly, plus build scripts. Stands alone.

**11. WebAssembly runtime.** A binary parser, a stack-machine interpreter and a minimal WASI layer, enough to run small modules like Fibonacci. 2,955 lines of Python. It runs what the compiler in 12 produces.

**12. Crypto and Lune to Wasm.** AES-128, SHA-256, HMAC, RSA and ECDH written from the specs, with a TLS-style handshake that wraps the web server, and a compiler from Lune to WebAssembly that handles integer code only. 2,294 lines of Python.

Figure: Diagram. An arrow points from the experiment that supplies something to the one that uses it. Boxes without an arrow stand alone.

Experiment 8's own AlphaZero script imports Experiment 3's neural-net layers, and 3's AlphaZero imports 8's chess board, so those two lean on each other. A small integer-only Lune program has gone all the way from Lune source to a native executable through the C compiler. The full language needs more than that compiler covers, because it only knows int, char and pointers.

## Rendering

Experiment 2 is a ray tracer in pure Python. Step through its features below. After the sky come surface normals, diffuse shading, shadows, a metal ball, a glass ball, and finally triangle meshes with a teapot.

### Ray tracer, one feature at a time

[Image: A smooth pale blue sky gradient and nothing else.]

Figure: Step 1. A camera and a sky gradient. No objects yet.

[Image: A sphere coloured by its surface direction, cyan to pink, against the sky.]

Figure: Step 2. Ray against sphere. Each pixel is coloured by the direction the surface faces, which is the quickest way to see that the maths is right.

[Image: A blue matte sphere on a green ground, slightly grainy.]

Figure: Step 3. Diffuse material: light bounces off in random directions.

[Image: An orange sphere and a small yellow sphere on a green ground with soft dark shadows under them.]

Figure: Step 4. Two spheres. Shadows fall out of the bouncing without any extra code.

[Image: A gold sphere, a blue sphere and a mirror sphere in a row on a grey floor.]

Figure: Step 5. Metal, with a mirror and a slightly fuzzy gold.

[Image: A glass sphere, a blue sphere and a mirror sphere in a row on a grey floor.]

Figure: Step 6. Glass, which bends the ray as it enters and leaves.

[Image: A gold Utah teapot and a glass ball on a grey ground under a pale sky.]

Figure: Step 7. Triangles. The Utah teapot is a standard test mesh that the script downloads. A BVH keeps each ray from being tested against all 6,320 of its triangles. 800 by 450 at 64 samples per pixel.

Each frame is a render from that stage; the last one is larger.

**What does "samples per pixel" mean?**

A path tracer fires random light paths backwards from the camera through each pixel and lets them bounce around the scene. One path is a noisy guess at the pixel's colour. Averaging many of them settles the guess, and the grain shrinks roughly with the square root of the count, so four times the samples halves the noise.

[Image: A path-traced scene of many coloured matte, metal and glass spheres on a grey ground under a pale sky, with grain from low sample counts.]

Figure: The ray tracer's "final scene", 800 by 450 at 64 samples per pixel.

[Image: A rendered square patch of procedural terrain with green grass patches, grey rock and a strip of blue water at the front edge.]

Figure: Perlin-noise terrain, with the material picked by height. The speckle is sampling noise, at 32 samples per pixel.

The Cornell box is the classic test scene: a lit room with a red wall and a green wall, where you can see colour bleeding onto the white surfaces. Mine is plain path tracing with no direct light sampling, so even at 256 samples per pixel it is noisy, and in pure Python it takes a while. I also wrote a software rasterizer, which is the other way to draw 3D: edge functions, a z-buffer and flat, Gouraud or Blinn-Phong shading. The teapot in the second image is its output.

[Image: A path-traced Cornell box: a white room with a green left-hand wall, red right-hand wall, a ceiling light and two white boxes, heavily speckled.]

Figure: Cornell box, 400 by 400 at 256 samples per pixel.

[Image: A bronze-coloured teapot, shaded smoothly, on a dark grey background.]

Figure: The rasterizer, Phong shading, 320 by 240.

## Languages and compilers

Experiment 1 is Lune, a small scripting language with closures, pattern matching and a pipe operator. I wrote it four times over, each version built on the last. First a tree-walking interpreter, which is clean and slow. Then a bytecode VM, a transpiler that turns Lune into C, and finally a lexer and parser written in Lune itself. The C output is hundreds of times faster than the tree-walker on a 1,000-element sort, which is about what you would expect for leaving Python. That is for sorting. On the ray tracer the C version was no faster than plain Python, because every vector is a table and every access a hash lookup.

**What is a bytecode VM, and what is a transpiler?**

A tree-walking interpreter reads the program as a tree and evaluates it node by node. A bytecode VM first compiles the program into a flat list of simple instructions and runs those in a loop, which is faster. A transpiler translates the program into another language, here C, so that a C compiler does the hard part.

Writing a language in itself found bugs I had not hit any other way. Lune processes string interpolation when it lexes, so you cannot write the two characters ${ inside a string, and I use chr(36) + "{" instead. Newlines are ignored inside braces, so a list literal on the line after a call was being read as an index on that call, and the fix was an explicit return. The lexer and parser in Lune handle most of Lune (match expressions are not handled yet), and a third piece, a compiler written in Lune, prints bytecode as text. Producing executable bytes and writing a VM in Lune are the remaining steps to full self-hosting.

[Image: A tiny render of a red, a green and a blue sphere on a flat ground under a sky, blocky because it is only 100 by 100 pixels.]

Figure: A 168-line ray tracer written in Lune, run through the language I built. It is 100 by 100 pixels, shown at four times size.

Experiment 6 is a compiler for a subset of C: int, char, pointers, arrays, if, while and for, with printf, write and exit. It goes through a lexer, parser, type checker, an optimizer (constant folding, dead-code removal and a peephole pass) and a code generator that writes x86-64 assembly. gcc then assembles and links it, under WSL on my machine. There is also a JIT experiment that writes machine code straight into executable memory and calls it through ctypes.

Experiment 11 is a WebAssembly interpreter, with a binary parser, a stack machine and a minimal WASI layer. Experiment 12 has a Lune to WebAssembly compiler that targets it. It compiles 32-bit integer code only: arithmetic, comparisons, if/else, while loops and functions that call each other. Strings, lists, tables and closures are outside what it handles.

## Databases, servers and an operating system

Experiment 7 is a key-value database that grew into a SQL one. The base is a B+ tree with up to 100 keys per node, stored on 4 KB pages with a 64-page cache. A write-ahead log makes it survive a crash: on startup it redoes committed transactions and undoes the rest. On top of that I put a SQL parser and executor with secondary indexes, hash and nested-loop joins and EXPLAIN, then snapshot isolation, then a three-node Raft implementation over localhost sockets.

Experiment 8 is a chess engine. It is the opponent in the AlphaZero experiments and the engine behind the web app. It does iterative-deepening alpha-beta with a transposition table and quiescence search, and its evaluation is built from material, piece-square tables and pawn structure. It can also speak UCI. Experiment 9 is the web server: HTTP/1.1 on raw sockets, a router with path parameters, a tiny template engine and HTTP/2 with frames, header compression and multiplexed streams. The chess web app on top of it keeps its game history in the database from 7.

Experiment 10 is MiniOS. A 512-byte bootloader loads a 32-bit kernel in C that handles interrupts and the keyboard, has a shell, a bitmap memory manager, paging, a first-fit heap and a read-only FAT12 driver. This is its own screen, captured after boot. One line in the middle is the kernel testing its own heap: a thousand allocations, freed in shuffled order, with nothing leaked.

### MiniOS booting, as captured from its screen

[OK] Entered 32-bit protected mode
  [OK] Kernel at 0x00001000  |  Stack at 0x00090000
  [OK] IDT installed (#DE, #GP, IRQ1/keyboard)
  [OK] PIC remapped  (IRQ0-7 -> INT 0x20-0x27)
  [OK] PMM ready  (7936 free frames / 32 MB)
  [OK] Paging enabled  (identity map 0-4 MB, heap 4-8 MB)
  [OK] heap: 1000 allocs, shuffled frees, no leak
  [OK] Heap ready  (kmalloc/kfree, 4 MB virtual)
  [OK] Interrupts enabled
  [OK] FAT12 filesystem ready (ls, cat, hex)

  Type 'help' for available commands.

$

This is MiniOS booting in an emulator, minus the boxed title banner. The shell has help, clear, mem, halt, ls, cat and hex.

## Physics and sound

Experiment 4 is a 2D physics engine. It has circles, walls and distance constraints that I solve by sweeping over them a handful of times a step (4 to 25 depending on the scene), and you can pick Euler or Verlet integration. It runs the cloth in the clip at the top, a 30-link chain, a billiards break and a Newton's cradle, and I made an Asteroids clone on top of it.

Figure: Experiment 4: a billiards break, a cue ball into a rack of 15.

Experiment 5 starts with gravity. It is an N-body simulator with a fourth-order Runge-Kutta integrator, a standard accurate way to step a simulation forward, and I tested it on the figure-eight orbit, where three equal masses chase each other around one figure-eight track. The starting values are the published figure-eight solution from 2000. Total energy should stay constant in an orbit, so any drift in it is the integrator's error, and over 20 laps that drift stays within 7 × 10-14 of the starting energy.

[Image: Three coloured dots on a green figure-eight track on a black background.]

Figure: The figure-eight orbit, 20 periods drawn on top of each other.

[Image: Line chart of relative energy error over time for the figure-eight run, wandering between about plus 1e-14 and minus 6e-14.]

Figure: Relative energy error over the same run.

The same experiment has an audio side I wrote from scratch: oscillators, envelopes, filters, a 16-step sequencer and Markov-chain melodies, including a three-minute generative jazz piece. The two meet in one program that plays the figure-eight as sound. Each body gets a sine tone. Its distance from the centre sets the pitch, snapped to a pentatonic scale, so closer is higher. Its speed sets the loudness, its left-right position sets the pan, and the gravitational energy opens and closes a low-pass filter. Press play for 30 seconds of it.

Figure: The figure-eight orbit, three periods, played as three sine tones. The orbit repeats, so the sound does too.

[Image: Four stacked charts over 30 seconds for three bodies: pitch in hertz, amplitude, stereo pan and low-pass cutoff. Pitch, amplitude and pan each rise and fall repeatedly; the filter cutoff dips slowly and recovers.]

Figure: What the sound is made from: pitch, amplitude, pan and filter cutoff over the 30 seconds.

## Learning from scratch

Experiment 3 is a neural network framework on numpy. It has dense and convolution layers (the convolution is done as one big matrix multiply), batch norm, dropout and the Adam optimizer, with every backward pass written by hand and the convolution and batch-norm ones checked against numerical gradients. On top of it I trained three digit classifiers, a VAE, a small GPT, a DQN agent and an AlphaZero attempt.

**Handwritten digits (MNIST), test accuracy on the 10,000 held-out images. One run each.**

| Model | Accuracy | Errors |
|---|---|---|
| Dense net, 784-512-256-10, dropout | 97.99% | 201 |
| CNN, two conv layers and a dense layer | 99.22% | 78 |
| Deeper CNN, batch norm and augmentation | 98.85% | 115 |

The deeper one aimed for 99.3% or better and landed at 98.85%, a little under the plain CNN, so the plain CNN is the best of the three here. This is the plain CNN's confusion matrix, which shows which digits it mixes up.

[Image: Confusion matrix for the CNN, test accuracy 99.22 percent, with a strong diagonal and a few small off-diagonal counts.]

Figure: The CNN's confusion matrix on the test set. Rows are the true digit and columns what it guessed.

The VAE has a two-number latent space and trained for 40 epochs. Each cell in the grid below is what its decoder draws for one point on a 20 by 20 grid across that space, and the digits blend into each other as you move.

[Image: A square grid of 400 handwritten-looking digits that slowly change from 6 to 1, 3, 5, 2, 8, 9, 4, 7 and 0 across the grid.]

Figure: The VAE's 20 by 20 latent grid. Horizontal and vertical are its two latent numbers, and every digit here was drawn by the model.

The GPT is a character-level transformer with 128-wide embeddings, 4 layers, 4 heads and a 128-character context, trained for 5,000 steps on the Tiny Shakespeare text. It ends at roughly 1.35 training and 1.58 validation cross-entropy, single run, and its output reads like Shakespeare with invented words:

HENRY BOLINGBROKE:
Who, what I afeary small well not book the death,
And be my rebel'd Romeo's Duke of Glounty breath.

[Image: Line chart of cross-entropy loss against training step for a tiny character-level GPT, train and validation curves falling from about 4.2 to about 1.35 and 1.58.]

Figure: The GPT's training curves. Validation flattens above training.

[Image: A four by four grid of attention maps, one per layer and head. In the first layer attention is spread thinly over earlier characters, and in later layers it concentrates near the diagonal, with one head in layer three fixed on a single column.]

Figure: The GPT's attention, one panel per layer and head. In layer 1 each character looks thinly at everything before it. Further up, the heads mostly look at the last few characters, and one head in layer 3 stares at a single character.

## What I ran into

The DQN agent is the first open end. I wrote a headless Asteroids game for it, where the agent sees 25 numbers: the ship's position, speed and angle, and the five nearest asteroids. Over 20,000 episodes the reward hovers around zero and does not clearly climb.

**What is a DQN?**

A Deep Q-Network is a reinforcement-learning method. The agent plays by trial and error, and a neural network learns to predict how much future score each action is worth in the current situation. It then picks the action with the best prediction, apart from a small share of random moves so that it keeps exploring.

Figure: The DQN agent flying in the headless Asteroids environment. It ends with the ship hitting an asteroid.

[Image: Three stacked charts over 20,000 episodes: episode reward oscillating around zero, episode length around 900 steps, and exploration rate dropping to 0.05 within about the first ten episodes, which looks instant on a 20,000-episode axis.]

Figure: DQN training curves. The smoothed reward stays near zero. Exploration drops to 5% almost immediately, so nearly every episode ran with 5% random actions.

The AlphaZero attempt is at an earlier stage still. Against my alpha-beta engine the network scored no wins and no draws in the 6 games of each of ten iterations. Those games were capped at 80 half-moves (40 moves each side) and decided on material, the engine got 0.05 seconds a move and the network 40 search simulations a move. Each iteration was 20 self-play games at 25 simulations a move, and the whole run took about 63 minutes. The network has 560,206 parameters. A second, separate attempt with its own network also scored 0% against the alpha-beta engine.

**What is AlphaZero?**

AlphaZero is a method where a neural network guesses which moves look good and how well a position is going, and a tree search uses those guesses to look ahead. The network plays itself, trains on the results, and the next version replaces the champion only if it beats it clearly. Mine used a 55% win-rate threshold.

Almost every self-play game ended in a draw, so there was very little signal about who was winning. Drag through the ten iterations and see it.

### AlphaZero attempt, iteration by iteration

**Results of each iteration.**

| Iteration | Self-play draws, of 20 | New against old, 10 games | Against alpha-beta, 6 games | Training loss | New champion |
|---|---|---|---|---|---|
| 1 | 20 | 5 wins, 5 draws | 0 wins, 6 losses | 3.367 | yes |
| 2 | 18 | 10 draws | 0 wins, 6 losses | 3.733 | no |
| 3 | 19 | 5 wins, 5 draws | 0 wins, 6 losses | 4.299 | yes |
| 4 | 20 | 10 draws | 0 wins, 6 losses | 3.732 | no |
| 5 | 20 | 5 wins, 5 losses | 0 wins, 6 losses | 3.852 | no |
| 6 | 19 | 10 draws | 0 wins, 6 losses | 4.032 | no |
| 7 | 20 | 10 draws | 0 wins, 6 losses | 4.038 | no |
| 8 | 20 | 10 draws | 0 wins, 6 losses | 4.174 | no |
| 9 | 20 | 10 draws | 0 wins, 6 losses | 4.185 | no |
| 10 | 20 | 10 draws | 0 wins, 6 losses | 4.251 | no |

Ten-game pits are noisy. The network took over as champion twice, at iterations 1 (against the untrained starting network) and 3, both on 5 wins and 5 draws. Every other pit scored 50%.

[Image: Line chart over ten iterations: win rate against the previous champion at 0.75, 0.5, 0.75 then flat at 0.5, and win rate against the alpha-beta engine flat at zero.]

Figure: The same run as a chart. A draw counts as half a win. Blue is against the previous network, red against the alpha-beta engine. The dashed green line is the 55% promotion threshold and the dotted line is 50%.

Two more results came out differently from what I set out to show. My chaos demo was meant to show two nearly identical starting points drifting apart. I nudged one body by 10-10, and the two runs separate to about 3 × 10-5 by the end, because the three bodies get flung apart early on (that is the jump at about time 0.6) and then coast off in straight lines. It shows sensitivity to the starting point, not chaos.

[Image: Log-scale chart of the separation between two nearly identical three-body runs over time: a sharp rise at about time 0.6, then a slow climb to about 3e-5 at time 25, far below the line marking a separation of 1.]

Figure: Separation between the two runs, on a log scale. It never gets near 1.

My Newton's cradle behaves differently from a real one. The balls collide, but after the first hit they swing together and jostle into each other, with some riding up over their neighbours, instead of passing the motion down the row.

Figure: Experiment 4: five balls on strings, restitution 0.995.

The cryptography is for learning, not for use. Keys, primes and padding come from Python's general-purpose random module, RSA defaults to 512 bits "for speed", and the handshake is TLS-style, not TLS. It is a way to learn the pieces.

## Where it stands

These are quick explorations at prototype stage. The route to a fully self-hosted Lune is described above. I also wrote a second version of the DQN script with Double DQN, reward clipping and best-checkpoint saving, and there is no result from it yet. The HTTP/2 tests use the third-party h2 library as the client.

- **Stack**: Python (numpy, scipy, pygame, matplotlib, Pillow), some C and NASM
- **Status**: Prototype
- **Code**: Not public