Source: https://oliviercouthaud.com/codenames · Status: Prototype · 2026 · Tags: Python, PyTorch, GRPO, JS

# Codenames

A Codenames clue giver: first two language models playing each other, then a model of how real people guess.

[Image: A five by five grid of squares: nine red, eight dark grey, seven pale and one black, with a key on the right.]

Figure: A Codenames board the way the clue giver sees it. Nine cards are mine, eight belong to the other team, seven are bystanders and one is the assassin. The guesser only sees the words. This is an illustration with a random layout, not a real board.

## What I wanted

This started with Gibberlink and the stories of AIs inventing secret languages to talk to each other. I wanted to see whether two small language models, rewarded only by the game score, would do the same and drift into a private code. In Codenames one player gives a one-word clue and a number, and a partner guesses which board words it means. I made the clue a stream of lowercase letters with no vocabulary behind it, so nothing forced it to be English, and let the pair play each other thousands of times.

It is the single-team version. Nine words are mine, eight belong to a team that never plays, seven are bystanders and one is the assassin. A team word is worth +1, the other team's card −1, a bystander −0.3 and the assassin −4. Clearing the board is +3 and every turn costs 0.15. I built it over a few days in late July 2026 on rented RTX 4090s, in two halves: three rounds of language-model experiments, then an agent that drops the language model and models the person it is playing with.

## Two models and a letter channel

A spymaster and a guesser shared one base model (Qwen3, 0.6B in round one and 1.7B after) and one LoRA adapter, and were trained together with GRPO. To test for a private code, a frozen copy of the untrained model read the same clues as a stranger would. If the pair had invented something, its clues should work with its own partner and fall apart with the stranger.

**What are GRPO and LoRA?**

GRPO is a reinforcement-learning method. The model plays a batch of games and is pushed toward the plays that scored better than the batch average, without training a separate scoring network. A LoRA adapter is a small set of extra weights attached to a frozen language model, so training changes a small fraction of its numbers. Here the spymaster and the guesser used the very same adapter.

Figure: Diagram. The self-play loop in rounds one to three. The orange box is the thing under test: whether the clue channel drifts away from English.

Round one used the 0.6B model for 15 and 18 updates of 128 games each, one run with a KL anchor and one free to wander. Both scored below random guessing, and most of that came from the interface I had built, not the learning.

Offering the guesser a "no guess" option made it pass between 80 and 98 percent of the time, because a generic phrase outscores any specific board word. Letting it write its guess one token at a time committed it to a first token it could not back out of, so I switched to scoring every remaining board word in one batch. Forbidding a clue to end inside a board word made it pad with repeated letters, so now I reject the clue and sample again. And "pass" is itself a word in the deck, so using it to stop quietly revealed that card on 6.5% of boards.

**What is a KL anchor?**

It is a penalty for moving the model's word choices away from where the original model started. With the anchor on, the clues are held near ordinary English. With it off, nothing but the score stops them drifting somewhere strange, which is what you would need for a private code.

Round two used the 1.7B model with all of that fixed, and ran the comparison I actually cared about: four runs, with or without a warm start and with or without the KL anchor. The warm start was behaviour cloning from a GloVe word-vector oracle on 120 boards. The oracle scores a clue by how much closer its weakest intended word sits than the most dangerous other card, with the assassin weighted hardest. Same RL, same budget: the warm runs reached 5.84 and 6.06 words per game and the cold runs 2.30 and 2.83. Self-play sharpened a pair that already played and could not create one.

Round two did not stress the private-code idea much, because plain English clues already worked. So in round three I made English expensive. Every guess was also scored by the frozen stranger, and the pair was charged for the probability the stranger put on the word its own guesser picked. If a private code was forming, that number had to fall.

**Team words found per game, out of 9, pair against itself on held-out boards (48 in round one, 64 in the others). Compare within a row only.**

| Round | Untrained | Trained | Random |
|---|---|---|---|
| 1 (0.6B) | 0.73 | 0.29, 0.10 | 2.13 |
| 2 (1.7B) | 2.95 | 6.06, 2.83 | 2.40 |
| 3 (1.7B) | 2.97 | 6.41, 6.06 | – |

Trained, round one: two variants. Round two: 6.06 is after a warm start (no KL arm), 2.83 is the best cold arm. Round three: 6.41 is the control, 6.06 the run paid for secrecy. Random guessing was not measured in round three. Rounds differ in model size (Qwen3 0.6B vs 1.7B), interface fixes, warm start and the secrecy term.

The warm start alone, with no RL, scored 5.48 in round two and 5.64 in round three.

Here are the four round-two runs while they trained. The table below gives the first, best and last update of each run; with scripts on, a slider reads off any single update. These numbers are sampled at temperature 1.0 during training, so they sit lower than the held-out scores in the table.

### Round two, team words found per game during training

The table gives the first, best and last value of each run, and the slider reads off the value of each run at the update it points to. In the chart the faint lines are those raw values and the thick lines are a 7-update moving average.

| Run | First | Best (at) | Last |
|---|---|---|---|
| Warm start + KL | 4.81 | 5.77 (36) | 4.58 |
| Warm start, no KL | 4.81 | 5.82 (29) | 5.05 |
| Cold start + KL | 2.81 | 3.76 (59) | 2.19 |
| Cold start, no KL | 2.81 | 3.96 (59) | 2.84 |

Every run peaks part-way and then declines, and the cold runs stay well below the warm ones.

Figure: Round three. The probability the frozen stranger gives the word the pair's own guesser picked, per update. Faint lines are the raw values, thick lines a 7-update moving average.

If a private code were forming, that line would fall. It does not. Averaged over the first ten updates and the last ten, the stranger's probability in the run that was charged for it went from 0.109 to 0.112. In the control run, which was not charged, it drifted up from 0.116 to 0.133. So the charge held the clues steady and did not make them harder to read. Cross-play says the same: against the frozen guesser the secrecy run found 3.56 words, the control 3.36 and the warm start with no RL at all 3.34. Their clues stayed recognisable English, with about 55% of them a single whole-word token in the model's vocabulary.

## Searching against a model of the partner

After round three I dropped the language model from the loop. Whatever the pair learned was graded by a guesser that was itself a language model, and every stand-in partner I measured against shared too much with the system to give an independent score. The next agents work from a model of the partner and search for the clue that partner is most likely to follow.

The first versions scored clues with sentence encoders, which measure similarity of meaning, and reached about 6 of 9 words on English. Codenames does not ask whether two words mean the same thing, it asks whether one makes you think of the other. Swapping in ConceptNet Numberbatch, which is built on a graph of how words relate, raised the mean cross-play score over five word packs (English, French, Spanish, invented words and a pack of nonsense strings) from 4.71 to 5.38 words, and Spanish alone by 1.8.

**What are Numberbatch and the USF norms?**

Numberbatch is a set of word vectors built from ConceptNet, a hand-made graph of how words relate, so it knows that two words go together even when they do not mean the same thing. The USF free-association norms come from a survey of about 5,000 cue words, where people wrote down the first word each one made them think of, which is close to what a Codenames guess is.

As a stand-in for a person I used the USF norms directly: a partner that ranks board words by how strongly people associated them with the clue, with an encoder to fall back on for the 3.4% of words the norms never recorded.

The agent won about a third of its games at that point. Nine words in eight turns needs 1.125 words per turn and it managed 1.04, so it kept finishing 8 to 9. Every board I tried turned out to be winnable: for every team word on every board, some clue in the vocabulary makes the partner pick that word first. The agent had not looked at those clues, because it scored 40 candidates picked by a similarity heuristic. Ranking the partner over all 8,000 or so legal clues is a single matrix product, and searching all of them closed the gap.

**Against the human-norms partner, 40 to 120 games per row. The last row also lets the clue giver model the partner with the same association norms the partner is made from.**

| Stage | Words of 9 | Win rate |
|---|---|---|
| Encoders only, shortlist search | 6.22 | 18% |
| + association graph (blend) | 6.90 | 28% |
| + shortlist search | 7.38 | 38% |
| + exhaustive search over the whole vocabulary | 9.00 | 100% |

I checked it on three fresh seeds, 40 games each. The first run won 119 of 120, and the one loss was an assassin hit. Then I raised the assassin penalty and fixed two bugs, and the same seeds went to 120 of 120. That last step is tuned on those seeds, and 120 of 120 is consistent with a true win rate as low as about 97%. It is also the least interesting number here, because of what comes next.

It is the same agent in every row below. Pick one to see what happens when its picture of the partner is right and when it is wrong.

### Same clue giver, different idea of its partner

Exhaustive-search clue giver, 60 games per row. Won is in brackets. Words is team words found out of 9, Per clue is words per clue; Hit assn. is the share of games that hit the assassin. With scripts on, the buttons below spell out each case.

| Case | Words | Per clue | Hit assn. |
|---|---|---|---|
| Generic human (60/60) | 9.00 | 1.65 | 0% |
| Own partner, blend (60/60) | 9.00 | 1.42 | 0% |
| Encoder partner (52/60) | 8.87 | 1.22 | 0% |
| Wrong, vs human (1/60) | 6.47 | 0.89 | 23% |
| Wrong, vs encoder (0/60) | 5.32 | 0.70 | 18% |

The 100% rows rest on a partner built from the same free-association norms the clue giver uses, so they measure fitting a human-shaped partner, not beating a person.

That pair of numbers is the real result. Skill at this game turned out to be almost entirely the accuracy of your model of the other player. Optimising hard against a model multiplies whatever the model gets right, and multiplies what it gets wrong just as hard.

## A partner fitted to real guesses

The next step was to stop picking weights by eye. The salt-nlp Codenames Duet data has real human guesses, and after cleaning I had 5,453 turns to fit on and 307 held out. The partner model is a multinomial logit over the board words. Its features are Numberbatch similarity, three sentence encoders, USF free-association norms and a hint prior built from crossword clues.

**What is a multinomial logit?**

For each word on the board it adds up the features, each multiplied by a fitted weight, and turns the totals into probabilities that sum to one. Fitting means nudging the weights until the probability given to the word a human really picked is as high as possible. A temperature is then tuned so the probabilities are honest numbers and not just a ranking.

Here is the agent that works from that model. Each part is explained under the diagram, and with scripts on you can tap or click a box (or a button below it) to read just that one.

**Numberbatch.** Word vectors built on a graph of how words relate. They know that two words go together even when they do not mean the same thing, which is most of what a Codenames clue trades on.

**Three sentence encoders.** Small multilingual models (MiniLM, MPNet and DistilUSE) that measure similarity of meaning. That is not quite what a guess is, but they can score any string at all, including words I invented.

**USF association norms.** A survey of about 5,000 cue words where people wrote down the first word each one made them think of, and how often. It is the closest thing I had to how people really associate.

**Crossword hint prior.** Millions of crossword clues boiled down to which words people use to hint at which answer. It is the only source here about what a person would say, not just what they would think of.

**LearnedPartner.** For each word on the board it combines twelve features with fitted weights and gives the probability a person picks it. The weights come from 5,453 real human guesses, and a temperature is tuned so the probabilities mean what they say.

**EVSpymaster.** Takes a cheap first pass over every candidate clue, keeps the best 120, then samples the partner's picks 96 times for each clue and clue size. A clue is scored by the chance the team can still win afterwards, times the chance of missing the assassin, and it assumes the partner stops guessing when it is not confident enough.

**Clue and count.** The count is the number of words the clue was chosen for. An earlier version asked the model to guess its own count after the fact, so what I meant and what I said could disagree.

**Web export.** Computes the partner's scores for 3,200 candidate clues against every word in the pack and squeezes them to 8-bit integers, so a web page can carry the whole model.

**Web game.** One web page where the same search runs in JavaScript and you play the guesser against it. I have not put it online because it carries the commercial game's word list.

Figure: Diagram. The later agent. The browser game needs no server and no model, only the baked table.

One design point mattered more than the rest. The spymaster used to pick the clue whose best-ranked word was a target, and that rewards a clue like soap for a card that carries almost no probability. Choosing by expected score under the calibrated distribution is meant to remove that exploit.

First, how well the fitted partner predicts people. On 307 held-out human turns, with chance at 4.5%, the fitted model beats each of the features it is built from. On pairs it has not seen, an LLM asked directly scored higher (0.400 against 0.357).

**Top-1 accuracy at predicting the word a human actually guessed. Claude Haiku may have seen this public dataset in training. The last column counts only clue and word pairs that never appear in the training data. About 53% of the test set is memorisable, which is why a lookup table scores so well overall and almost nothing on unseen pairs. A dash means not measured.**

| Scorer | Top-1 | Unseen pairs |
|---|---|---|
| One encoder (MiniLM) | 0.518 | – |
| Encoder consensus | 0.573 | – |
| USF human norms | 0.599 | – |
| Fitted partner model (12 features, English) | 0.664 | 0.357 |
| Lookup of training pairs | 0.616 | 0.052 |
| Qwen3-8B log-probability | 0.349 | 0.191 |
| Claude Haiku, asked which board word the clue points at | 0.661 | 0.400 |

### One person, 20 boards

The one check that is not a stand-in is a person, and the person was me. I guessed on 20 boards, with clues from this agent as it was on 30 July, without knowing which words it was aiming at. The clues announced 2.5 words on average. I found 32 of the 50 intended words, fully solved 7 boards and never hit the assassin. Of my 44 guesses, 38 were team words, 3 were the other team's cards and 3 were bystanders. Some of the clues made sense at once and some made no sense to me at all, which was odd and a bit cool. Here are ten of the twenty, so you can see the kind of miss.

### Ten clues, what they were aimed at, and what was picked

Clue and count as announced. The Aimed at and Picked columns show words from the board. The second line under Picked is how many of the aimed-at words were picked.

| Clue | Aimed at | Picked, and hit |
|---|---|---|
| dragon, 2 | fire, knight | fire, knight 2 of 2 |
| actress, 2 | star, hollywood | star, hollywood 2 of 2 |
| supporters, 3 | fan, revolution, net | fan, revolution, net 3 of 3 |
| prayer, 2 | church, grace | church, grace 2 of 2 |
| mars, 4 | march, alien, van, opera | alien, then stopped 1 of 4 |
| bishop, 2 | church, king | king, then stopped 1 of 2 |
| foreign, 2 | alien, america | alien, wall (my team's) 1 of 2 |
| fire, 3 | dragon, light, crash | death, light, point (other team's) 1 of 3 |
| ebay, 4 | shop, web, lap, beijing | shop, chair (other team's), web 2 of 4 |
| bat, 2 | ball, cricket | vet (my team's), bug (bystander) 0 of 2 |

## What I ran into

I saw no sign of a private code in any round. Round one was too short and small to test it, and round three tried one secrecy weight (3) for about 100 updates, so this does not rule out a private code forming. The charge cost 0.35 words per game (6.41 for the control, 6.06 for the charged run) and did not produce secrecy. My guess is that the shared adapter ties the two roles together: to become unreadable to the base model the spymaster has to move away from it, but the guesser is that same base model plus the same LoRA, so it moves too. Separate adapters would test that.

An earlier, differently rewarded run of round one found a reward hack on its own. The spymaster said "blank" every turn and the guesser passed every time, and the reward improved from −2.36 to −0.94 for doing nothing. Charging for every team word still hidden at the end fixed it, because doing nothing became the worst result. In round two every run peaked part-way and then declined, so the end-of-run numbers in the table understate every run. From round three I picked checkpoints by held-out games and not by training reward.

Several measurements turned out to be more generous than they first looked. A human-norms partner with an encoder fallback shared a model with the agent and inflated the result by 1.9 words (8.78 against 6.88 once I rebuilt it). A search-based agent scored 3.33 words higher in self-play than against a held-out encoder, averaged over five word packs. And the exhaustive-search agent with a wrong model of its partner won 2% of games or fewer.

A language model as clue generator made no measurable difference: 4.71 mean cross-play words with it and 4.71 without. Its invented clues were dominated by a frequency-ranked word list, at roughly ten times the compute. Adapting to an individual partner online moved prediction by less than half a percentage point.

The person test is a very small sample, one person and 20 boards. How the agent does with people who guess differently from me or from the data it was fitted on is the open question.

## Next

I built two more blind tests aimed at that gap, one where the clue giver picks the target set first and one where a language model vetoes weak clues, and their results are still to come. Separate adapters for the two language-model roles, and association data for languages other than English, are next to try. The next real step is playing a lot more games with people other than me.

- **Stack**: Python, PyTorch, Transformers and PEFT (LoRA), GRPO on Qwen3-0.6B and 1.7B, sentence-transformers, ConceptNet Numberbatch, USF norms, salt-nlp Codenames Duet data, a crossword-clue corpus; JavaScript for the web game
- **Status**: Prototype
- **Code**: Not public

[← All projects](https://oliviercouthaud.com/projects)