Source: https://oliviercouthaud.com/wordle · Status: Prototype · 2025 · Tags: Python, PyTorch, PPO

# Wordle Solver

A neural network that taught itself to play Wordle, in a simulator I built.

Figure: A game I picked to show how it works, not a recorded run of the agent. Wordle colours are shown here as black, orange and outline.

Figure: A game I picked to show how it works, not a recorded run of the agent. Wordle colours are shown here as black, orange and outline.

### Run it on today's Wordle

One button. It fetches today's puzzle, the trained network plays it here in your browser, and the board fills in live. Nothing is recorded or chosen beforehand.

This is the first version of the network, with the same word list, observation and mask as in training, rewritten in JavaScript. The puzzle comes from the Wordle feed through a small function on this site. A word outside its 16,132-word list can't be played.

## What I wanted

I got a bit obsessed with Wordle and started wondering how good cheating could be: how strong I could make the system, and how many guesses it would need on average. I wanted an agent that picks its guesses from the colour feedback alone, trained entirely in a simulator, and then I wanted to see whether the same agent could play a real site. The training runs are from 26 and 27 November 2025.

## How it works

A Gymnasium environment picks a random target from a list of 2,344 answers. The agent picks one of 16,132 allowed guesses, and the environment returns the green, yellow and gray feedback. What the agent sees is a grid of 26 letters by 5 positions by 3 states (unknown, not here, confirmed), plus how many guesses it has used and what fraction of the word list is still possible. v2 adds the previous guesses as 30 more numbers: 6 guesses of 5 letters each.

Training uses MaskablePPO from sb3-contrib. The mask removes every word that contradicts the feedback so far, so the agent never has to learn the rules, only which allowed word is a good pick.

**What is PPO?**

Proximal Policy Optimization, a standard way to train a policy by trial and error. The agent plays a batch of games, works out which guesses led to better scores than expected, and nudges the network towards them. A clip stops any single update from changing the network too much, so one bad batch can't undo everything it has learned.

**What is an action mask?**

A list that tells the agent which of its 16,132 options are allowed right now. Mine switches off every word that contradicts the feedback so far. It also means the agent always plays a stricter version of Wordle's hard mode: every guess has to be a word that could still be the answer, so it can never spend a guess just to test some letters.

Figure: Diagram. The orange box is the only learned part. On a real site the word list is the same, but the site may reject a word it does not know, and then the script masks it and asks again.

Figure: Diagram. The orange box is the only learned part. On a real site the word list is the same, but the site may reject a word it does not know, and then the script masks it and asks again.

The reward is made of a few parts. Each guess earns 2 points per bit of information it gains, meaning how far it shrinks the list of words that still fit, plus 0.5 per green and 0.2 per yellow. A win pays 50 plus 5 for each guess left unused. A loss costs 500. That penalty is deliberately huge: the mode in my environment is called safety first, and the idea was that the agent should never lose. v2 keeps the same numbers and adds an extra 3 times the information gain on the first guess.

**What is a bit of information?**

One bit halves the number of possibilities. A guess that takes 16,132 candidate words down to 44 gains about 8.5 bits, because 16,132 divided by 44 is about 367, which is roughly 2 multiplied by itself 8.5 times. The environment counts these bits over the 16,132 allowed guesses, not over the 2,344 answers.

### Score one game, guess by guess

Played out with the v1 rewards, the game in the picture at the top is worth 18.24, 12.32 and 67.50 points for its three guesses, 98.06 in total. Almost all of the first guess's points come from how many words it ruled out.

A game I chose for illustration, not a recorded run. The word counts are the real ones for my 16,132-word list. The rewards use the formula from my environment.

v2 swaps the network for one that starts with a state encoder (three small branches for the letter grid, the guess count and the guess history, joined into 256 numbers) followed by 256 and 128 unit layers, keeps the same mask and reward shape, and adds four reward modes and code for ensembles and for looking at attention. v1 is a plain 3-layer network of 256, 256 and 128 units for both policy and value.

Around the agent there are a few small tools. One is a script that opens an unlimited-Wordle site in a browser with Playwright, lets the model pick a word, types it in, and reads each tile's state from the page. If the site doesn't know the word it masks it and asks the model again, which matters because my guess list is much looser than a real game's: it has obscure words and names in it. Another is a helper where I type in the colours from a real game and the model suggests a word. The last two are plain solvers with no learning. The first is an interactive candidate filter with a letter-frequency suggestion (December 2025). The second (February 2026) is built around the colour rows of someone else's guesses, where I can see the colours and not the letters, and keeps the answers that some allowed guess could have produced.

## How well it plays

While training, each run evaluated itself every so often: 12 evaluations of 50 games, with the model playing its best guess every time and no exploration. A game counts as won when its reward ends up positive, and that is unambiguous here: every win in these evaluations scored between about 82 and 144, and every loss between about minus 471 and minus 435.

Each point is one evaluation of 50 games. v1 was evaluated every 40k steps up to 480k and v2 every 80k up to 960k. The two runs differ in more than the architecture.

| Run | Wins, all evals | Mean guesses | Last eval |
|---|---|---|---|
| v1, MLP | 532/600 | 4.79 | 42/50, 5.08 |
| v2, 1M-step run | 576/600 | 4.33 | 50/50, 3.98 |

Mean guesses count a loss as 6. v2 was best late in training, but it won 50 of 50 twice and 45 of 50 once, so a single point at 50 games is noise. The win rate during training agrees: measured over the last 100 games with exploration still on, it averaged about 83% for v1 and about 95% for v2 across the last ten readings. Training took about 50 minutes for v1 and about an hour and three quarters for v2.

### Look inside one evaluation

The last evaluation of v2, at 960k steps, won 50 of 50 games with 3.98 guesses on average: 2 games on guess 2, 11 on guess 3, 25 on guess 4, 10 on guess 5 and 2 on guess 6.

Real numbers from the 50 games of each evaluation. Bars show how many games ended on each guess, and the orange bar is games lost. Both runs use the same vertical scale.

There is no baseline in the same masked environment, for example a random consistent word or the letter-frequency suggester, so these numbers don't show how much the learning itself adds on top of the mask.

I ran the browser script on an unlimited-Wordle site quite a bit. From memory it averaged about 3 to 4 guesses a game, now and then got the word in 2, and I think it never lost. I didn't record those games, so treat that as an impression and not a result.

**Live in the browser.** The box at the top runs the v1 network in JavaScript: the same 392-number observation, the same action mask and the same feedback rule as my environment, with the output layer stored as 8-bit integers so the whole model is about 3 MB. I checked it against the Python version on 42 games and it picked the same word at every step. Over all 2,344 answers, the words it trained on, the best v1 checkpoint wins 2,130. Four of those answers are not in its guess list, so that is 2,130 of 2,340, or 91%. It takes 4.55 guesses on a win, and 4.68 if a loss counts as six. The wins split as 1 in one guess, 32 in two, 237 in three, 778 in four, 692 in five and 390 in six. Because these are training answers, it is not a held-out result.

## What I ran into

The comparison between v1 and v2 is not clean. v2 differs in features, reward (the opener bonus), network shape, learning rate (0.001 against v1's 0.0003) and training length (about a million steps against about half a million), and it ran 16 games in parallel. The two runs changed several things at once. At 480k steps, where v1 stopped, v2 was already ahead in these evaluations, 48 of 50 against 42. v1 peaked at 48 of 50 at 320k and slid back from there, so more training did not obviously help it.

v2 was set up with a learning-rate schedule, but the rate stayed at 0.001 for the whole run, so the schedule had no effect on these numbers.

The attention model is also not attention over words in that run. The attention version was meant to match a query from the game state against embeddings of all 16,132 words, with the mask left off so that it learns the rules itself. The run I'm showing numbers for only uses the state encoder, and it does use the mask. The real word attention lives in a separate training loop, and that loop won no games in about 4,000 steps: every game ran to six guesses, and the average reward sat near minus 547 throughout.

**What is attention here?**

In a neural network, attention scores how relevant each item in a set is to a question, then mixes the items using those scores. Here the set was meant to be all 16,132 words, each a learned list of 128 numbers, and the question was the current state of the game. That would let the network see which words are similar before choosing. The run behind the numbers above skips it.

The word lists have one mismatch. Four of the 2,344 answers (admin, cyber, inbox and ramen) are not in the 16,132-word guess list, so the agent cannot win those four. That is a tiny share of the answers.

The browser script has no logged results, only my recollection of the games I watched it play. The helper where I type in colours is set to load a model from a folder that no longer exists, so it needs a path passed in.

## Next

The next step is a fair test: answers the agent never trained on, a simple baseline in the same masked environment, v1 and v2 compared one change at a time, and a logged run of the browser script over a fixed number of games on the real site, to replace my memory of it. The word-attention loop would need another try after that.

- **Stack**: Python, PyTorch, Gymnasium, Stable-Baselines3 with sb3-contrib (MaskablePPO), Playwright for the site script
- **Status**: Prototype
- **Made**: November 2025 for the agent, December 2025 and February 2026 for the two plain solvers
- **Code**: Not public