Catan is deceptively hard for computers. Unlike Chess or Go, where the board state tells you everything, Catan throws dice, hides cards, and forces you to negotiate with opponents who have their own agendas. We wanted to build a neural network AI for ColonyRush — our digital Catan implementation — that could challenge experienced players. We started with a model that won 5% of its games. Seventeen days and sixty experiments later, it wins 78%.
The final model has 129,000 parameters, fits in 0.5 MB, runs in your browser at 5-10ms per evaluation, and plays on your phone. This is the story of how we got there — the breakthroughs, the dead ends, and what we learned about teaching a neural network a game it was never designed to play.
The Problem: Teaching a Computer to Play Catan
Chess has perfect information. Go has no randomness. Poker has no board. Catan has all of the above: a spatial board where position matters, dice rolls that inject randomness every turn, hidden hands of resource cards, and a negotiation layer where players trade with each other. Oh, and there are only ~100 turns per game, which means the model needs to learn from relatively short episodes.
The standard approach for board game AI — train a model to evaluate positions, then pair it with tree search — sounds straightforward. The devil is in the training signal.
The label problem
Our first model used binary labels: 1.0 for the winner, 0.0 for everyone else. This is how most game AIs start. For Catan, it was a disaster. A player who’s dominating the game with 9 victory points but loses to a lucky dice streak gets labeled identically to a player who was never in contention. The model couldn’t learn anything useful from this sparse, noisy signal. It won 5% of its games — worse than picking moves at random.
The fix was embarrassingly simple: instead of binary win/loss, we gave every turn a score based on the player’s current standing:
label = 0.4 × (playerVP / 10) + 0.3 × (vpDiff / 10) + 0.3 × winBonus × gameProgressVictory points, the gap between you and other players, and a win bonus that ramps up as the game progresses. Every single turn now carried a training signal instead of just the final outcome. Win rate jumped to 22% — a 4.4x improvement from changing nothing but the labels.
Later, we’d discover an even better labeling scheme — rank-based labels that assign scores based on final placement (1st: +0.95, 2nd: +0.317, 3rd: -0.317, 4th: -0.95). This single change would eventually add another 25 percentage points to our win rate. But we’re getting ahead of ourselves.
Architecture Evolution
Our model went through three distinct architectural eras. Each one taught us something about what matters when representing a Catan board for a neural network.
Act 1: The MLP Era (5% → 36%)
We started where everyone starts: a Multi-Layer Perceptron. Flatten the board into a vector, stack some dense layers, predict a value. Our first model had 1.6 million parameters (512→256→128→64) and the spatial awareness of a blindfolded player.
The MLP treated the board as an orderless bag of numbers. A settlement at the intersection of wheat, ore, and sheep was indistinguishable from one at wood, brick, and desert — unless the model memorized every possible arrangement. It couldn’t.
Scaling helped somewhat. Jumping from 1.6M to 3.56M parameters pushed us from 22% to 25%. Going further to 8.5M parameters got us to 36% against Medium AI. But we were buying performance with brute force, and the returns were diminishing fast.
Key learning from this era: data quality beats data volume. We tried training on 327,000 samples from Catanatron (a strong open-source Catan AI). Our own 84,000 samples from self-play against Insane AI outperformed it handily. The Catanatron data came from an alpha-beta search agent whose play style was fundamentally different from what our model needed to learn. Distribution match matters more than dataset size.
Act 2: The CNN Breakthrough (17% → 65%)
Catan’s hex board has spatial structure. Adjacent hexes share vertices. Resources cluster geographically. Settlements near the center access more trading partners. This screams convolution.
We mapped the hex grid onto an 11×21 pixel grid with 12 channels — terrain types (one-hot encoded), dice probabilities, robber position, and building occupancy. A dual-path CNN processed this spatial representation alongside 57 context features (resources in hand, development cards, opponent stats, production rates).
Spatial Path: Board [12×21×11] → Conv(32) → Conv(64) → Conv(64) → Pool → 64-dim
Context Path: Features [57] → Dense(128) → Dense(64)
Fusion: Concat [128] → Dense(256) → Dense(128) → Dense(1) → TanhTotal: 140,000 parameters. 60x fewer than our best MLP, and immediately better.
But the real gains came from regularization and training improvements, not architecture tweaks:
- Spatial dropout (0.15 on conv layers): +11 percentage points. This was transformative — forcing the model to not rely on any single spatial feature.
- Hard AI training data: We’d been training on Insane AI games, but evaluating against Hard AI. Switching to Hard AI training data eliminated a distribution shift that was costing us ~13 percentage points.
- Rank-based labels: The single biggest improvement. Switching from VP-based to rank-based labels (1st place = 0.95, 4th place = -0.95) jumped us from 31% to 52% in one experiment. Rank labels give cleaner gradient signal — a 2nd-place finish with 8 VP in a close game is clearly different from a 4th-place finish with 8 VP in a runaway.
- Data scaling: 2,000 games → 4,000 games → 8,000 games, each step adding ~6.5 percentage points.
With 8,000 Hard AI games (902,000 training samples), rank labels, and PUCT search (more on that below), the CNN hit 65% against Hard AI — verified over 200 games. But it plateaued there. Validation loss stuck at 0.133 no matter what we tried: more data (16K games actually regressed), larger models (128 channels overfitted), auxiliary training targets, self-play fine-tuning. The CNN had hit its representational ceiling.
Act 3: The Graph Neural Network (62.5% → 78%)
The CNN’s grid representation was a hack. Catan’s board isn’t a pixel grid — it’s a graph. Hexes connect to hexes. Settlements sit on vertices where three hexes meet. Roads run along edges between vertices. Squashing this into a rectangular grid destroys the topology.
First attempt: hex-only GNN. We built a 19-node graph where each node is a hex tile, connected to its neighbors. Three layers of Graph Convolutional Network (GCN), residual connections, mean pooling, same fusion head. 95,000 parameters, and it achieved a validation loss of 0.1245 — significantly better than the CNN’s 0.133 floor. But win rate was only 62.5%. Better loss, slightly worse play. The hex-only graph captured terrain relationships but lost track of where buildings actually sit.
Second attempt: heterogeneous GNN. This was the breakthrough. Instead of just hex nodes, we gave the graph three types of nodes:
Hex nodes (19): terrain, dice number, robber, nearby buildings
↕
Vertex nodes (54): my settlement/city, opponent buildings, port access, production value
↕
Edge nodes (72): my road, opponent road145 nodes total, connected by 300 undirected edges across three relationship types:
- Hex ↔ Hex: natural board adjacency
- Hex ↔ Vertex: each hex connects to its 6 surrounding vertices
- Vertex ↔ Edge: each edge connects to its 2 endpoint vertices
Each node type gets its own feature projection (hex features → 64 dims, vertex features → 64 dims, edge features → 64 dims), then three shared GCN layers with LayerNorm and residual connections propagate information across the entire graph. Finally, type-specific mean pooling (one 64-dim vector per node type) concatenates with the 64-dim context path for a 256-dim fusion input.
Type-specific projections:
Hex: 12 features → 64-dim
Vertex: 6 features → 64-dim
Edge: 2 features → 64-dim
Shared GCN (×3 layers, residual + LayerNorm):
H' = LayerNorm(ReLU(A_norm × H × W) + H)
Type-specific pooling:
hex_pool = mean(hex nodes) → 64-dim
vertex_pool = mean(vertex nodes) → 64-dim
edge_pool = mean(edge nodes) → 64-dim
context = Dense(57 → 64) → 64-dim
Fusion: concat [256] → Dense(256) → Dense(128) → Dense(1) → Tanh129,000 parameters. Validation loss: 0.1248. Win rate: 78% against Hard AI.
Why does this work so dramatically better? Because vertices and edges encode exactly what the CNN could only approximate. A vertex node knows precisely whether a settlement sits there, which port it connects to, and its production value. An edge node knows exactly which roads are built. The CNN had to infer all of this from a blurry grid representation where hex boundaries overlap and vertex positions are ambiguous.
The +13 percentage point jump from CNN to HeteroGNN (65% → 78%) was the single largest architecture-driven improvement in the entire project.
Search: MCTS + Neural Evaluation
A neural network that evaluates positions is only half the story. To play well, you need to look ahead. We paired our value network with Monte Carlo Tree Search (MCTS) — the same family of algorithms behind AlphaGo.
The idea: from the current position, simulate many possible futures. At each decision point, use the neural network to evaluate how promising each branch looks, and focus your search on the most promising lines. After enough simulations, play the move you explored most.
PUCT: letting intuition guide the search
Plain MCTS explores randomly. PUCT (Predictor + Upper Confidence Bound for Trees) uses a policy network — a separate model trained to predict what action a strong player would take — to guide exploration toward promising moves before the value network has confirmed them.
The selection formula balances exploitation (moves that look good so far) with exploration (moves the policy network recommends but haven’t been tried enough):
score = (totalValue / visits) + c × prior × √(parentVisits) / (1 + visits)The first term is the average value from simulations. The second term is the policy prior’s influence, which starts large for unvisited moves and shrinks as they accumulate visits. The constant c controls the exploration/exploitation tradeoff.
Our policy network is a CNN classifier (143,000 parameters, 75.4% accuracy) that predicts which of 19 action types to take next. It doesn’t predict the exact move — just the category (build settlement, buy development card, trade, etc.). This coarse-grained intuition is enough to focus the search on productive lines.
How much search is enough?
We tested 50, 100, and 200 simulations. The answer was clear: 100 simulations saturates the benefit. More simulations didn’t help — and at 200, performance actually dipped slightly. With a noisy evaluator (our value network isn’t perfect), more search means more time amplifying evaluation errors in deep branches. For a 100-turn game with ~10 decisions per turn, 100 simulations per move means roughly 1,000 neural network evaluations per turn — about 5-10 seconds of thinking time on a CPU.
The Graveyard: What Didn’t Work
Sixty experiments means a lot of failures. Here are the most instructive ones:
| What We Tried | Result | What We Learned |
|---|---|---|
| 16K training games (2× more data) | 58.5% (↓ from 65%) | Model capacity saturated — more data overfit to noise |
| Insane AI training data | 52% (↓ from 65%) | Distribution shift: train on what you’ll play against |
| Self-play from scratch (2K games) | 47% | Neural self-play is too noisy without a strong base |
| Expert Iteration (MCTS visit policies) | 16% | MCTS visits were ~93% concentrated on one action — near one-hot labels |
| 128-channel CNN (2× wider) | 64% (no gain) | Architecture ceiling was in the encoder, not the model capacity |
| ResNet-style skip connections | No improvement | Catan boards are small; deep networks aren’t needed |
| Catanatron training data | 16% from scratch, 24% fine-tuned | Can’t learn search-based play from imitation |
| Knowledge distillation | Worse than direct training | Teacher’s knowledge is in the search, which the student can’t replicate |
| Feature engineering (+27 hand-crafted features) | 5% | More features = more noise for MLPs to drown in |
| Label normalization | 8% | Removed the signal about absolute position quality |
The benchmark variance trap
Our most expensive lesson wasn’t a failed architecture — it was trusting small sample sizes. A 50-game benchmark has ±15% variance. We spent days celebrating “improvements” and chasing “regressions” that were pure noise.
Example: one model benchmarked at 47% over 50 games, then 31% over 50 different games. Same model, same opponents, 16-point swing. After we standardized on 200-game benchmarks, the phantom improvements and regressions disappeared, and our experiment velocity actually increased because we stopped chasing ghosts.
Our most expensive lesson: always run 200+ games before celebrating.
Running in Your Browser
All of this runs client-side. No server calls, no GPU required.
We export the PyTorch models to ONNX format — an open standard for neural network inference that runs everywhere. The browser loads the models via onnxruntime-web, which supports WebGPU acceleration on capable devices but falls back gracefully to CPU.
The numbers:
- Value model: 0.5 MB download, 5-10ms per evaluation
- Policy model: ~0.5 MB download, <5ms per evaluation
- Total per move: ~100 evaluations × 5-10ms = 0.5-1.0 seconds of thinking time
- Works on phones: same models, same runtime, same performance
One neat engineering detail: the ONNX runtime auto-detects the model’s expected input size. Our value model expects 3,297 features (the heterogeneous encoding), while the policy model expects 2,829 features (the standard encoding). The inference wrapper automatically truncates or pads the feature vector to match, so both models work with the same encoder output.
What’s Next
At 78% against Hard AI, the neural model is our strongest non-search-based player. But it has clear weaknesses. It hoards resources (16.7 discards per game vs. Hard AI’s 8.0), buys too few development cards (2.8 per game vs. 6.0), and never contests Longest Road or Largest Army. These are patterns inherited from its training data — it learned what Hard AI does, not what beats Hard AI.
The natural next step is self-play reinforcement learning: let the model play against copies of itself and learn from its own wins and losses instead of imitating a fixed opponent. Our first attempt at this (Expert Iteration, Experiment 54) was a spectacular failure — but that was early in the project, with a weak base model and crude visit-count targets. With a 78% base model, better training infrastructure, and lessons learned from the graveyard, a second attempt at self-play could break through the imitation learning ceiling.
The dream is a model that develops strategies we didn’t teach it. Maybe it discovers that early development card investment beats greedy expansion. Maybe it finds trading patterns that no human player has tried. The 78% model is a strong foundation — but it’s still playing someone else’s game.
By the Numbers
Experiments run: 60
Development time: 17 days
Training samples: 4.4 million game states (across all experiments)
Best model parameters: 129,217
Model file size: ~0.5 MB
Win rate vs Hard AI: 78%
Win rate vs Insane AI: 55.5%
Inference time: 5-10ms per evaluation
MCTS simulations: 100 per move
Architectures tried: MLP → CNN → GNN → HeteroGNN
Biggest single gain: +25pp (rank-based labels)
Biggest architecture gain: +13pp (CNN → HeteroGNN)
Most expensive mistake: Trusting 50-game benchmarks