AI Research · Open Source
Red Queen
Genetic programming, neuroevolution, NEAT, Q-learning, DQN, PPO and self-play — each implemented by hand, competing on purpose-built games under one honest leaderboard. And you can watch every one of them think, live in the browser.
Overview
A Research Playground and a Textbook That Runs
Most explanations of reinforcement learning stop at the equations, and most implementations hide the algorithm behind a framework. Red Queen does neither. Every algorithm is written from first principles, and every one of them plays on a game chosen to make a particular idea visible: why representation matters more than compute, why lookahead beats evaluation, why exploring costs something.
The site has three games — Snake, Checkers and a multi-armed bandit — each with its own leaderboard, plus a 15-chapter Learn section that runs from genetic algorithms to self-play. Each chapter trains its algorithm live in your browser and cites real results, including the ones that didn't work the first time.
The name comes from Through the Looking-Glass and the co-evolutionary arms race: every improvement in one player is a new problem for another.
Algorithms
No PyTorch. No Gym. No NEAT-Python.
Genetic programming
Linear (register machine) and tree GP; tournament, lexicase and Pareto selection
Neuroevolution
Fixed-shape networks evolved as weight vectors with Gaussian mutation
NEAT
Innovation numbers, structural mutation and speciation
Bandits
Greedy, ε-greedy, optimistic, UCB, Thompson sampling, gradient bandits
Tabular RL
Q-learning and SARSA with n-step returns
Deep RL
DQN (replay, target network, double, dueling, prioritized), REINFORCE, A2C, PPO
Self-play
TD(λ) and TD-Leaf(λ) position evaluators for Checkers, plus an AlphaZero-style learner
Gradients
Reverse-mode autodiff and a character-level transformer built on it
The Games
Each Game Teaches One Idea
The games aren't there for their own sake. Each one is chosen so that the difference between algorithms becomes something you can see on screen.
Snake
Representation matters more than compute
Each algorithm learns to steer a snake toward food without trapping itself. The biggest improvements came from changing what the snake can see — adding "how much room does this move leave?" — rather than from switching algorithms.
Checkers
Lookahead beats evaluation
Two-player strategy. Some entrants search moves ahead; others learn by playing a million games against themselves. The strongest one combines both: a small network that learned to judge positions, steering a search.
Bandit
Exploring costs something
A row of slot machines with hidden payouts. A strategy has to balance trying new machines against playing the best one it knows. You can watch its beliefs update with every pull, then race it yourself.
How It Works
From Training Run to Your Browser
Train
A training job runs an algorithm on a game and streams its progress live: watch a population evolve, or a network's play improve, as it happens.
Evaluate
The finished champion plays games it has never seen and is ranked beside simple baselines, so a high place means it learned something general rather than memorised its training games.
Package
The champion is exported as a small model file and checked to make sure it still plays exactly the same moves after export.
Watch
Your browser loads the game and the model and runs them locally. Every move it makes, you can see which inputs it weighed and how strongly.
Architecture
Under the Hood: One Core, Three Places It Runs
The monorepo separates algorithms from everything that observes them. Training is a job that wires an algorithm to a telemetry library; the algorithm libraries never import telemetry. Evaluation is a separate job that ranks every finished run's champion. The FastAPI backend serves run history — with a live SSE stream while training — and model packages. The browser does the rest.
libs/games, libs/rl Rust core: rules, observers, bandits → PPO, TD(λ)
├─ PyO3 ──► jobs/trainer train, log metrics, save champions
│ └─► libs/modelpack ONNX export + measured parity
└─ WASM ──► browser same game, same seed, same moves
└─► ONNX Runtime Web champion runs client-sideDecisions
Four Decisions Worth Explaining
Every algorithm built by hand
Why: The point is to understand and compare the algorithms, not to call them. The evolution loops, autodiff engine, batched MLP with hand-written backprop, Adam, replay buffers, GAE and PPO's clipped objective all live in the repo — and each is tested against an independent reference implementation.
Trade-off: No GPU frameworks to lean on. The hot paths moved into dependency-free Rust to keep training practical on a CPU.
Game rules live once, in Rust
Why: Each game's core is bound to Python (PyO3) for training and compiled to WebAssembly for the browser, so a seed is the same game everywhere. CI checks determinism digests: a given seed trains bit-identically natively and in WASM.
Trade-off: Two bindings to maintain. The original Python implementations are kept as test oracles so the Rust port can't silently drift.
Rank on unseen games, never training performance
Why: Training performance rewards memorising the games an agent has already seen. Every entrant is scored on games it never trained on, under a versioned protocol, next to fixed baselines — with the uncertainty and the measured cost to train and to run shown alongside.
Trade-off: Evaluation is its own job and takes real compute — but a leaderboard is only worth reading if it's honest.
Inference runs in the visitor's browser
Why: Champions are exported as content-addressed ONNX model packages and run in ONNX Runtime Web (WebGPU or WASM) beside the WASM game. Watching or playing costs the server nothing per move, and the site can be served as a static snapshot.
Trade-off: Export can change behaviour, so parity between the trained model and the exported one is measured and shown, not assumed.
Process
Negative Results Are Results
Every major decision gets a numbered design doc — twenty so far — written before the work, with the hypotheses, and updated with what the results actually said. Some of the most useful ones record what didn't work: an AlphaZero-style learner fine-tuned from the self-play champion never beat the TD-Leaf recipe it started from, and the doc says so, with the runs to back it.
The same discipline shows up in the product: Learn chapters cite real results, and leaderboards show training and inference cost beside the score, so a stronger agent that costs ten times more to run doesn't look like a free win.
The Learn section: 15 chapters, foundations first, each with a live demo.
Live
Watch the models think
Play Snake against the leaderboard, put two Checkers engines on the board, or race a bandit strategy — all running in your browser.
Open redqueen.mackinations.ca ↗Interested in working together?
Get in touch