TwentyOne

I built this because it's a game I've played, and it looked like something an AI could get properly good at. I wanted to find out whether one could beat my friends, my family, or me. The one time I played it, it beat me badly, and I'm hoping that was luck.

Twenty One is a card game inside Roblox, based on the 21 minigame in Resident Evil 7. Two players draw cards numbered 1 to 11, and whoever gets closer to 21 without going over wins the round. The loser pays for it: a saw on a track slides toward them by the current bet, which starts at 1 and goes up by 1 every round, and when it reaches you the match is over. There are also 25 kinds of special cards, called trumps, that change the target, the bet, your total or the deck. It's unofficial: I'm not connected to the game's makers, and none of their art or code is in my project.

Besides playing well, I wanted it to be able to tell me what to do mid-round. The game has no complete rulebook, the wiki and the in-game card text disagree in places, and a few rules I'm still unsure of, so the program has a switch for each uncertain rule and can play under either reading.

It has three parts. A rules engine plays the game exactly as I've written it down. A neural network taught itself to play by playing an enormous number of matches against itself. And a search, when there's time, takes each move I could make, plays out the rest of the match in its head about a thousand times, and checks whether the network's first instinct holds up. That's the difference between a chess player's gut feeling and stopping to calculate: the gut answer takes a few thousandths of a second, the calculation about one. I type in what I can see on screen, and it tells me what to do and how likely I am to win.

Against other computer players it does well. The newest network beats my first one 53 times in 100, a real edge in a game with this much luck. Even two perfect players only win half each, and in a cut-down version with no special cards, where perfect play can be calculated, a player who could see the opponent's hidden card would win only about two matches in three against a perfect one. The newest network is about a tenth of a percentage point of win rate away from perfect play in that version.

What I don't know is how good it is against a person. Every number here is computer against computer. I played one match against an earlier version of it, with no hints, and it beat me in four rounds. I hope that was luck, and in a game this lucky one match can't really say either way. Training also stalled: learning by self-play stopped helping, so I taught the network from the search's answers, which worked, but only a bit. And the search adds less on top of the newest network than it did on top of my first one.

What's next

The part I haven't done is put it up against people properly, which is what I was curious about to begin with: friends, family, and me again. I also want to use its advice in a real game and see whether it holds up. Some of the rules I modelled are still guesses, so those games are also how I find out which ones are wrong.

What I wanted

I picked this one because it's a game I've played, and I could see how an AI might get good at it. What I really wanted to know was whether it could beat my friends, my family, or me. The one time I played it myself, it took me apart, and I'm hoping that was luck.

Twenty One is a Roblox card game, a take on the 21 minigame in Resident Evil 7. Two players draw cards numbered 1 to 11 toward 21, and whoever is closer without going over wins the round. The loser of each round takes damage: a saw moves toward them by the current bet, which starts at 1 and rises by 1 each round by default, and the match ends when it reaches them. There are also 25 kinds of trump card that change the target, the bet, the totals or the deck, and some of them raise or lower the bet. This is unofficial. I'm not connected to the game's makers, and none of their art or code is in it.

Beyond playing well, I wanted it to be able to advise me in the middle of a round. The rules aren't written down in one place, so the first job was a spec: 45 settings, where every rule I'm not sure about is a flag. A wrong flag makes the advice wrong, so the networks are trained over both readings of the contested rules. The spec starts from the rules as I know them from playing, and I checked them against a community wiki.

How it works

I implemented the spec twice. One is a Python engine written to be easy to read, at about 10,000 steps a second. The other is in Rust, for speed: around 11 million steps a second for the engine step alone, batched on 24 threads with a random policy (the whole loop with a Python policy runs at 2 to 6 million). A differential fuzzer plays random games on both in lockstep and compares the full state after every step. Its first run covered 210,000 games and 15.6 million steps, and every difference it found was fixed and kept as a regression test.

The solver gives the exact equilibrium of the game with trumps switched off. That is my yardstick for how far a network is from perfect play. I can only compute it without trumps.

What is an equilibrium here?

In a game with hidden cards, an equilibrium is a strategy that nobody can beat on average by switching tactics. If both players use it, neither gains anything. So a network's distance from it is how many percentage points of match win rate it gives away to a perfect player. It can be computed exactly only for the simpler game, which is why the yardstick stops at the no-trump version.

The network has about 2.4 million parameters, with a GRU, a small memory layer that carries what it has seen through a match. v1 is recurrent PPO: it plays itself, its own older snapshots and some simple bots.

What is PPO?

Proximal Policy Optimization. The network plays, and after each batch of games it is nudged toward the moves that went well and away from the ones that didn't, with a cap on how far one update can push it, so a lucky batch can't wreck it. Here the only reward is winning or losing the match.

v2 adds the solver's answers as extra training targets. v3 is expert iteration: seven cycles of an hour of search labels, half an hour of training and a check, 8.3 hours on my RTX 5070 Ti.

What is expert iteration?

A loop between a quick player and a slow one. The slow one, here the search, looks hard at a position and finds a better move than the network's instinct. The network is then trained to produce that move directly, which makes the quick player better and the next round of search better. Training one model to copy a stronger one is called distillation.

The advisor is what I'd use at the table. I type what I see in a short shorthand, a small web app that only runs on my own machine turns that into game events, and the network answers in about 5 ms on the CPU. In the background the search takes a copy of the match, samples hidden-card worlds that fit what I've seen, and plays the rest of the match out with the network, assuming the opponent plays like it. There are three budgets: fast, about 1,000 rollouts and 1 s; normal, 5,000 and 5 s; deep, 16,000 and 30 s. I used to require a lead of two standard errors before the search overrode the network and lowered that to one after measuring what each setting gained per decision. The advisor also warns me when a loss would kill me, or when one of my Bless cards is buried under another trump and isn't protecting me, and it keeps an UNVERIFIED RULES line on every answer until I confirm each unsure rule in the real game. There's a second screen in the same app where I can play against the network at three levels, which is where my one game happened.

The same numbers: the judging search plays the opponent as listed. Gain per decision at margin z = 1, with z = 2 in brackets underneath.

Opponent	Fast	Normal	Deep
The network itself	0.63 (0.50)	0.80 (0.68)	0.87 (0.80)
Looser (T = 2)	0.57 (0.46)	0.69 (0.63)	0.77 (0.75)
Sharper (T = 0.5)	0.64 (0.49)	0.75 (0.65)	0.78 (0.74)
The exploiter network	0.56 (0.46)	0.80 (0.65)	0.82 (0.74)
v1	0.55 (0.46)	0.75 (0.63)	0.79 (0.71)

The recommendations hold up against opponents that play differently from the network: 85 to 100% of the gain survives in every world, it never gets near zero, and the recommendation is worse than the network's own move in only 5 to 8% of states. A person might play patterns none of them contain.

Results

Simulation only. Opponents are networks and the exact trump-free solver, never people. The search row is a per-decision gain in win-probability points, not a match win rate.

Question and conditions	Result
v3 network vs v1 network Best move for both, 15,000 matches over five rule sets	53.07% [52.27, 53.86]
Gap to perfect play, no trumps Points of match win rate, mean of 6 cells, 60,000 matches each	v3 0.11, v1 1.59
v3 + search vs v1 + search Search player vs raw v2 and exploiter networks, 2,500 matches each	58.2 vs 56.8
What search adds per decision 2,000 states, margin rule at 2, judged by an independent search, in the model's own world. Gain in points, for the fast / normal / deep budgets	+0.67 / +0.88 / +0.96
A network trained only to beat it Copy fine-tuned for 3,285 updates against the frozen target, 60,000 matches each	wins 52.48% vs v1, 47.75% vs v3

The network on its own got better. With the search switched on, v3 and v1 come out level within noise: the difference is +1.4 points with a standard error of 1.35.

What does level within noise mean?

A standard error says how far a measured number could drift if I ran the whole test again with fresh random games. A gap of 1.4 points with a standard error of 1.35 is about one standard error, so the true gap could easily be zero or negative. As a rough rule, anything within two standard errors of zero can't be told apart from luck.

v3 is ahead of v1 on every rule set I tested, by 1.9 to 4.6 points. The exploiter is a copy of the network fine-tuned only to beat one frozen opponent. Against v1 it found 2.5 points. Against v3 the same recipe found nothing, but that only says this recipe couldn't exploit v3, not that nothing can. On v1, the fast search beat the plain network in 61.8% of 322 duplicate matches [56.4, 66.9], with the margin at 2, which is why v3 was trained on search labels.

Read the 53.07% with care. The opponent is v1, which v3 descends from, and the search labels were made against v1-like play, so part of the gain may be exploiting v1's habits. A player that can see the hidden cards beat v1 83.7% of 182 matches (±2.8). That player isn't perfect, so at most about 16% of matches at this level are unwinnable even with that knowledge. In the no-trump game the numbers are cleaner: a perfect player beats the best fixed rule, drawing while under 16, by winning only 53 to 55 matches in a hundred, and a player who sees the opponent's hidden card reaches 68.

What didn't work

PPO plateaued. v1 beats its own older snapshots by less and less, and it is level with the snapshot from update 20,000. One win or loss after about 80 decisions is a noisy teacher, so v3 moved to search labels.

The search adds less on top of v3. In match play against v1-like opponents the search is worth about +4 points on top of v3 and +7.5 to +9.8 on top of v1, even though per decision, in the search's own model, the gain is the same (+0.67 against +0.69). A network is a better model of opponents that play like itself, and every opponent here descends from v1. After seven cycles the network also hasn't moved toward the search at the states where the two disagree. Two further variants and weight-averaged checkpoints came out level with v3 (48.5 to 50.4%), and the v4 run was stopped.

Hand-written rules backfired. Forcing "Hush before Stay" onto v1 dropped it to 38.3% in 6,000 matches. Overriding a recurrent network with a rule can wreck its own plans. Blanket rules for the card game fail the same way: in the no-trump game, never drawing from 16, 17 or 18 wins 47.1, 49.2 and 49.5% against plain v1.

v1 draws too much. Without trumps the solver knows the right answer, so v1's mistakes are easy to count: about 19 decisions in 1,000 are clear mistakes, where the other action was at least 3 points better, and 86% of those are draws that should have been stays. The error grows with the hand total.

With trumps it is blurrier. A search flags about one decision in ten, but the flags are noisy: where the exact answer is known, only 44% of them were real. The flags with a claimed gap of 8 to 15 points held up when I replayed both options from the exact state against the real opponent (+11.4 points on average, 14 cases), and the three above 15 did even better. The 3 to 5 point flags were noise. The weak spot is card-moving trumps (Exchange, Remove, Return, Destroy, Reincarnation) played on the wrong card or at the wrong time. That study covers 4,648 decisions, about a quarter of what I planned, because it was stopped early. v1 beat three hand-written stand-ins for human players 88 to 95% of the time (79 to 83% with 2 trumps per round). Those are guesses at how people play, not people, and without trumps they tie it.

What's unfinished

I don't know how strong it is against a good person. I've played one game against it, on 13 October, against v1 with search (not v3), with no hints, and it beat me in four rounds. I hope that was luck, since one game of something this lucky can't settle it. Every other number here is against networks, hand-written bots or the exact solver, and the advisor hasn't been used in a real game against a real opponent. The contested rules (Hush, the scope of Shield, Bless) are modelled but not confirmed in the real game, and a wrong guess there makes the advice wrong in those positions. The advisor's win percentage can also differ from the search's by several points in trump-heavy positions (39% against 48% in one test), and I don't know which is better calibrated. Nothing is running right now.