GBuddy’s Escape: Machine Learning on an orignal Game Boy (DMG-01)
Poor GBuddy walks into a wall. He stands there for a moment, tries again, and walks into it a second time. Then he turns, trots down a corridor he has already visited twice, and somewhere along the way, slightly to his own surprise, finds the exit.
What looks like a clumsy little robot is a learning experiment. GBuddy, the mascot of this museum, is small, square and endlessly patient. He collects experience, one bump at a time, and changes his decisions based on it. And all of that learning happens inside a ROM for the original Game Boy.
The DMG-01 was never meant for anything of the kind. Its processor runs at a little over four megahertz, it has eight kilobytes of working memory, and it cannot even multiply two numbers without being taught how in software.
GBuddy's Escape joins three other cartridges built for this collection. One is a reference work in the manner of the InfoGenius titles of 1991, one tests the console it sits in, and one prints labels. This one hands the screen to a robot and lets him work out a maze for himself.
A robot that knows nothing
The maze is ten fields wide and eight high, each field sixteen pixels square, and all of it fits on the screen at once. GBuddy starts in one corner, the exit is marked G, and he has four possible moves: up, down, left and right.
What he does not have is any idea where the exit is. There is no map in his head, no recorded sequence of moves and no table worked out on a computer beforehand. On a fresh cartridge, every one of his opinions is exactly zero.
So he guesses, cheerfully but blindly. He bumps badly into every wall and visits the same dead end as if it might have changed since last time. A training run ends when he reaches the exit or after 256 moves, whichever comes first, and on the early runs it is usually the second. Then the next run begins, and what he learned stays with him.
Each move earns him a small score. An ordinary step costs one point, bumping into a wall costs five, and reaching the exit earns fifty. Those three numbers are the only thing anyone tells him about the maze. Everything else he has to find out for himself, antenna first.
Game Boy machine learning, one move at a time
The method is called Q-learning, a form of learning by trial and feedback. It helps to imagine GBuddy keeping a small notebook for every field of the maze. It holds four entries: how good it has turned out to be to go up, down, left or right from there.
After every step he adjusts the entry for the move he just made. He takes into account the score of the step and how promising the field he landed on already looks. Good experience near the exit therefore spreads backwards along the corridors, field by field, until the moves that lead towards it are rated higher than the moves that lead away.
The arithmetic is done in whole numbers, taking a quarter of each surprise on board at a time. The DMG-01 has no hardware for fractions, and a learning rule that relied on them would not fit.
Most of the time GBuddy picks the move he rates highest. Now and then he deliberately tries a random direction instead, at first on one move in four. With training these tries become rarer, but they never disappear, because without them he would settle on the first route that worked and never find a shorter one.
It is worth being precise about what this is. GBuddy does not understand the maze the way a person does. He learns ratings for decisions at particular positions. This is tabular Q-learning, not a neural network, a chatbot or a language model, and it is still genuine machine learning: experience changes what he does next.
Fun fact: Christopher Watkins described this machine learning approach in his doctoral thesis at Cambridge in 1989, the year the Game Boy came out.
Luck is not knowledge
Reaching the exit proves surprisingly little. A robot that tries things at random will stumble out of a small maze sooner or later, and to anyone watching, that looks exactly like success.
The cartridge is a fond but strict teacher. Every time GBuddy reaches the exit in training, it immediately runs a second, silent check. It sets him back at the start and lets him walk on what he has learned alone: no random tries, no learning, always the move he rates highest.
If that walk does not reach the exit, something other than his learned decisions helped him in training. The screen then says so: EXIT REACHED, LUCK INVOLVED.
The cartridge does not measure how much of a success was luck. The line is a playful way of saying that the route could not yet be confirmed.
Two counters, and why they differ
That distinction runs through the whole cartridge, and it is easiest to see in its two counters. SUCCESS counts every time GBuddy reaches the exit in training on the current maze, however he got there. IMPROVEMENT counts only progress that has been checked: the first confirmed route, and after that every confirmed route that is shorter than the best one so far.
An example shows the difference. GBuddy reaches the exit five times in training, which makes SUCCESS #5. Only then does he manage it in a check with no random tries, his first confirmed route and IMPROVEMENT #1. Later he learns a shorter way, and that is IMPROVEMENT #2. Testing the same route again adds nothing.
The test that proves it
The same check is open to the player. Once at least one training run is complete, TEST puts GBuddy at the start and lets him walk on what he has learned. Nothing he does during a test changes his ratings, the statistics or the random sequence of the training.
A test ends at the exit, at a safety limit, or when he steps onto a field he has already visited on the same walk. That last case matters. Without randomness, the same ratings on the same field produce the same choice every time, so a robot that comes back to a field would go round in circles forever. The cartridge stops him kindly and explains that he needs a little more practice.
Solved, and then the walls move
How does the cartridge know a route is the shortest? A separate function works out the shortest possible length through the current maze. It never hands GBuddy a single move and never trains him; it only measures his route against that number. The cartridge can judge a solution without giving it away.
When his confirmed route matches that length, training pauses and the screen reads MAZE SOLVED. In the run that produced the pictures for this article, that took sixteen training runs on the standard maze, with a shortest route of twelve moves. Another run may take more or fewer, because the random tries differ.
The cartridge then suggests the obvious next step, which is to change the maze. EDIT ARENA moves a cursor over the fields and turns floor into wall and back. Start, exit and the outer walls are protected, and the editor refuses any change that would cut the exit off. The maze never changes by itself; only the player moves the walls.
GBuddy keeps what he has learned, and that is the interesting part. On the changed maze he sets off confidently along his old favourite route and meets a wall that was not there yesterday. Now he has to adjust the corridors that no longer lead anywhere. It is hard not to feel for him. His willingness to explore is raised again, and the counters for the maze start from zero. On the edited maze from these pictures he went on to find the new shortest route, fourteen moves long.
For a clean start there is RESET AI. It erases his ratings, statistics and counters but keeps the maze, so the same task can be watched again from nothing.
A wiki for a robot
Fifteen short pages on the cartridge explain what is happening, in plain language and without a single formula. They cover the scores, the difference between practice and test, luck against learned decisions, every status message and both counters.
The last page says what the whole experiment shows. The learning happens in the ROM: no internet connection, no server, no external AI and no calculation on a connected PC. What is on the screen is what the program itself has worked out, through a loop anyone can follow: try, receive feedback, adjust, check, and face a new challenge.
Try it
The cartridge is written for an original Game Boy DMG-01 on a flash cartridge with battery backed memory. Testing took place on an EverDrive X7 Pro.
Try it yourself: GBuddy's Escape runs in your browser, and the ROM is free to download for a real Game Boy, on its player page.
There is something fitting about it. The machine that made handheld gaming ordinary was built from cheap, proven parts, and it has no business learning anything. Thirty-seven years later a small robot learns a maze on it anyway, one quarter of a surprise and one bumped antenna at a time. And when he gets lucky, the cartridge says so, plainly and without holding it against him.
Comments