Beat the rat. Take the pot.

There is no animal and no camera. The body is DeepMind’s open-source virtual rodent, a detailed model of a real rat’s skeleton built for neuroscience (Aldarondo et al., Nature 2024). It has 67 joints moved by 38 motors and weighs 0.34 kg.

It lives in MuJoCo. The engine works out gravity, friction and collisions every 2 milliseconds. Nothing in the clip is animated by hand. When the brain is in control it decides, 50 times a second, where each motor should move its joint. The physics does the rest. Push the wrong motors and it wobbles, slips, or falls over, the way a body does.

Its home is an operant box: the classic lab chamber where a rat learns that pressing a lever pays off. Four walls, a floor, and a spring-loaded lever on the front wall. On this page the box is gone: the rat stands at a checkers board.

Learning is trial, error and reward

At first the dials are random, and so are the movements. Every attempt is scored: points for a clean lever press and for standing well, a penalty for falling. After each round of practice the dials move toward whatever scored more. This is reinforcement learning. The method is PPO.

• 32.5Mpractice steps, lever network
• ≈7.5 daysof simulated time, many rats at once
• 1.15Mpractice steps, steering network

Each practice step is 20 milliseconds of simulated time. A network that starts from random dials needs a huge number of tries, so training runs many copies of the rat in parallel.

Stage 1 · the lever-press network

200 senses, body and lever. 38 body motors. Its job is to press the lever with a paw and stay standing. A press counts only when it is clean: a paw pushes the lever past 11.5°, the head and belly stay off it, and the rat is upright.

Tested alone, 800 fresh attempts in four suites. Every time it made a clean press and was still standing 1.2 s later.

• Standard Standing in front of the lever after a 1 s settle.200/200
• Wide New start spots, a stiffer or softer lever, a different floor.200/200
• Shoves Shoved in a random direction before the press.200/200
• Noisy Random jitter added to its own moves.200/200

Stage 2 · the steering network

21 senses: where the lit target sits against the cursor, how big it is, how the head is turned, and whether the rat is upright. 5 outputs: 4 neck motors, plus one PRESS signal. Its job is to point the head so the cursor lands on the lit target, and to press only when it is on it.

How the two work together

1. The steering network turns the head. The rest of the body holds its standing pose. The head’s direction moves the cursor.
2. On target, it signals PRESS. The lever-press network takes the whole body for up to 1.2 s and presses. The cursor holds still.
3. The press is the action. Then the steering network gets the head back.

The head is the pointer. Turn it left and the cursor slides left. A small turn is a fine move, a big turn is a sweep. The lever is the commit. One press, one action, and a press off the target is ignored.

The sha256 lines below are the published fingerprints of the two networks, the skin file, and the physics scene. Change one dial and the network hash changes. Checking the brain means opening those files. A hand of three cups, rock-paper-scissors, or checkers is the desk on this site.

steering  3b9da3e23581ad96d1256e930d75a5c7c2119410fe4133dab4a20759e353d88e
lever     3f55d8cea25de8bc026eb44b46ee35666c4e8c876a9d5f2d3055bcce37675bac
skin      a050c7049d150a6fe09fc606af977d518f5fa03e75693f72210301d2fb1b0f3a
scene     cd95f2230d856acf4d55347a86dbb075f37b1695637ba9b72b811b3e56785c74

R-01’s brain is two artificial neural networks: grids of numbers on a computer, driving a simulated body. The skeleton is a model of a real rat. No animal was filmed. Layer widths, the evaluation, the move rules, and 0.01$ are written out on the record. MuJoCo steps the body in that code. This page draws it.