Source: https://oliviercouthaud.com/monopolyai · Status: Prototype · 2026 · Tags: Python, PyTorch, PPO

# Monopoly AI

A Monopoly simulator I wrote from scratch, and an experiment in whether a program that works out how its opponent plays can win more because of it. All of it runs in simulation, against scripted bots.

[Image: Bar chart of win rate with the opponent read on and off. Diverse league, 12 bots the model trained against: 83 percent on, 73 off. Held-out league bots: 69 on, 56 off. GNOME-style bot: 56 on, 59 off. Fixed-policy bot: 79 on, 68 off.]

Figure: The headline result. Two-player games, one checkpoint. The "read off" bar still runs the same search; it just assumes opponents move randomly. The opponents are scripted bots I wrote, not people.

## The problem

I picked Monopoly because I have played it, and I could see room for an AI to do something interesting. I was curious whether it could beat my friends, my family or me.

I wanted a Monopoly agent that works out who it is playing and exploits it, instead of playing one fixed style. Apart from dice and card order the game is perfect information, and auctions and trades give a careless opponent plenty of ways to lose. The reward comes only from the final result (placement, later blended with final net worth). Nothing rewards a strategy directly.

First I needed something to train against. The engine is about 1,150 lines of plain Python with the standard US rules: house and hotel shortage, even building, auctions, mortgages, bankruptcy, trades as structured objects. It has 37 rule tests, and 1,000 random games ran with no exceptions. I checked edge cases against two public implementations, [GNOME-p3](https://github.com/mayankkejriwal/GNOME-p3) and [fj317/Monopoly-RL](https://github.com/fj317/Monopoly-RL), as a reference only. I did not copy code from either. It is an unofficial simulator, Monopoly is a Hasbro trademark, and I use no board art or card artwork.

## How it works

The first agent was self-play PPO: a small network over a 691-number observation and 135 masked actions. The later one adds a transformer with a per-opponent memory. Each opponent's recent moves become a "style" vector, and a side head is trained to predict that opponent's next move. The head only shapes the representation and never touches the reward. The idea follows published opponent-modelling work (Hernandez-Leal et al., [arXiv 1907.09597](https://arxiv.org/abs/1907.09597); LIAM, [arXiv 2006.09447](https://arxiv.org/abs/2006.09447)).

**What is PPO?**

Proximal policy optimisation is a reinforcement-learning method. The agent plays, finds out whether it won, and nudges its policy toward the moves that went better, with a cap on how big each nudge can be so training does not blow up. In self-play the opponents are copies of the agent itself.

**What is a side head?**

A network has a main job, here choosing a move, and can be given a second job on the side. Mine guesses the opponent's next move from the style vector. That forces the vector to carry real information about that opponent, without tying it to the reward.

The version that finally used the read is a search, not a network. At each buy, auction or post-roll decision it copies the game, tries a few candidate moves, plays 30 decisions forward and scores the result by net-worth margin. My scripted HeuristicAgent plays my side in that forward play, and the opponents follow the learned prediction. The search only leaves the heuristic's move for an alternative that scores more than 20 better.

Figure: Diagram. The final agent. After a roll the candidate set also includes ending the turn, up to two builds and a couple of lopsided trade offers. In an auction it is pass plus a few bids.

### One decision, step by step

- **Decision.** The search only runs on three kinds of decision: whether to buy a property I landed on, what to bid in an auction, and what to do after a roll (build, trade or end the turn). Everything else is left to HeuristicAgent.
- **Candidates.** A short list, not everything that is legal. The heuristic's own move is always on it. After a roll I add ending the turn, up to two builds and up to two lopsided trade offers. In an auction it is pass plus a few bids.
- **Rollout.** Each candidate is tried on a copy of the game, which is then played forward 30 decisions. I play the heuristic's moves and the opponents play what the model predicts they would do. With the read switched off they move randomly instead. That is the only difference between "read on" and "read off".
- **Score.** Each end state is scored by net-worth margin, with a very large bonus if I won the game outright inside the rollout.
- **Pick.** I keep the heuristic's move unless another candidate scores more than 20 better. That keeps the search close to solid play instead of letting it chase every small gain.

## Results

The read itself works. The more of an opponent's moves the model has seen in a game, the better it predicts the next one. In one run that goes from 47% with one to three moves seen to 87% with 23. Drag the slider to walk along it, or switch to the second run, which has a point for every move count and is less tidy.

### How often it guesses the next move right

The vertical axis starts at 40%. Accuracy is dominated by common moves such as roll, buy and end turn. It is lower on the rare ones that matter, which comes up below.

**What do "read on" and "read off" mean?**

Both versions run exactly the same search. In "read on" the opponents in the forward play do what the model predicts. In "read off" they move at random. So the gap between them is how much the guess about the opponent is worth. The same games are played both ways, starting from the same seeds, to keep the comparison fair.

**Win rate against scripted bots, two players, paired games. The read-off column is the same search with opponents assumed random.**

| Opponents | Games | Read on | Read off |
|---|---|---|---|
| Diverse league, 12 bots the model trained on | 240 | 83% | 73% |
| League, 4 held-out bots | 240 | 69% | 56% |
| GNOME-style bot | about 112 | 56% | 59% |
| Fixed-policy bot | about 112 | 79% | 68% |

### Read on against read off, one set of opponents at a time

**What is a standard error?**

If you replayed the same number of games, the win rate would wobble a little just from the luck of the dice. A standard error is the typical size of that wobble. A gain of three or four of them is very unlikely to be luck. A gain of one is something luck could easily produce.

The two league gains are roughly three to four standard errors, and the fixed-policy gain about two. Against the GNOME-style bot, a tight and competent player, the read adds nothing I can measure. That bot is a reimplementation of the shape of the public GNOME baseline, not its code. The fixed-policy bot is just that: buy above a flat cash buffer, build when able, turn down every trade. The model never trained against either. The league bots are 16 draws from one generator of six style numbers, mainly trade willingness, bidding and jail habits, so "held-out" means unseen draws, not a different kind of player.

Reading opponents beat assuming they are random by 10 to 13 points against a diverse bot league. The league results are all against 16 draws from one bot generator, and everything here is two players and one checkpoint.

## What I ran into

Most of the work was in the steps before the last one. Plain self-play won about 100% against a random player but 0 to 4% against the heuristic: an outcome-only reward gives almost no signal when the agent rarely wins. A curriculum addressed that in two players, starting from a near-random heuristic and sharpening it, and the agent reached about 51% against the full heuristic (42 to 58% across evaluations). That is parity, and more training did not move it. In four players the same curriculum gave about 0% against full heuristics. A heuristic-heavy league reached an average placement of 0.74, third to fourth, with almost no outright wins.

The opponent-reading network was the next step. With its memory on or off it won the same share of games on the hard bot panel (26.0% and 26.7%, 150 games). Its policy sat near-uniform whatever I changed, including batches ten times larger on a rented 36-core GPU machine. The read itself worked, and the search described above turned out to be the way to put it to use.

**What does "drifted to near-uniform" mean?**

A policy is a spread of probability over the legal moves. One that has learned something puts most of it on a few moves. One that has learned nothing spreads it evenly. Entropy measures that spread, and mine sat close to its maximum, so the agent was in effect choosing moves at random and had nothing to adapt.

**Search against opponents that accept any trade, 2 players, small samples. Win rate by what the search believes about the opponent.**

| Opponent model in the search | Before fix | After fix |
|---|---|---|
| Learned (aux) | 0.75 | 0.60 |
| Random (null) | 1.00 | 1.00 |
| Perfect (oracle) | 0.75 | 1.00 |

The first version of the search scored 21 points below the random-opponent baseline (0.73 against 0.94 over 100 games), and a perfect opponent model did no better. The cause was the scoring. One-step net worth made grabbing a property always beat building, and in a move-by-move comparison against a competent opponent the agent went for a trade on 95% of turns. Rolling the game forward fixed the oracle, which now matches the baseline. The learned model's own row went from 0.75 to 0.60, which is within noise on 10 games. All three after-fix cells are 10-game runs, so only the direction counts. A 60-game run had shown +28 points on held-out bots, and that came down to +13 at 240 games, which is the figure I use.

The learned read is most accurate on the common moves and less so on the rare ones that matter. In an earlier diagnosis, before the league-trained model, it predicted "reject" 68 times out of 68 against a bot that accepts any trade.

## Where it stands

Every result for the search and the opponent read is two players against scripted bots. The only four-player numbers are the earlier PPO runs above. The design covers two to six players, and the search is evaluated at two so far. My evaluation caps games at 35 rounds and ranks survivors by net worth, so a "win" can be a lead at the cap. Human play data and games against a person are the next things to add.

A larger run is scoped (about 40 bots, GNOME-style and fixed-policy bots inside training, two to six players) and has no results yet. Trades currently come from a handful of candidate offers rather than being built freely, and a faster engine in a compiled language would help at league scale.

The aim is a super-human player, and this is the first stage. Once it gets there, I will try it on the family at Christmas, which is a good excuse for a few Monopoly games.

- **Stack**: Python, PyTorch, PPO, a rule-based Monopoly engine, a model-based search
- **Status**: Prototype
- **Built**: August 2026
- **Code**: Not public