Concepts

The move network

The move network sees the information state as 92 tokens, one per occupiable square, each built from that square's slice of the input planes plus a learned positional embedding, and one extra token for the value. Eight pre-layer-norm encoder layers, embedding 384, 8 heads and a feed-forward width of 1,536 give 14.7 million parameters. The size was chosen by trading sample efficiency against iteration speed, which the authors found to be a significant tradeoff.

92 square tokens + value tokenvalue tokeneach token: planes of that square + positionpre-LN blockself-attentionfeed-forward8 layers · 384 wide8 heads · 14.7M paramsOutcome from the value tokenwin58%draw27%loss15%Move logits = key(from) · query(to)rows: source squarecolumns: destinationgrey: illegal, masked outsoftmax over the legal entries is the policy
Square tokens pass through the encoder, the value token predicts the outcome, and a key-query product between source and destination tokens scores every move.

The policy head

A move is a source square and a destination square, so the policy is a matrix. Each output token produces a key and a query; the logit of moving from square to square is the dot product of the query of with the key of , masked to the legal moves and normalised with a softmax. The paper credits this parameterisation, borrowed from AlphaZero's chess-move representation lineage, with faster learning than simpler heads. The 10,000-entry action space the engine exposes (source times 100 plus destination) is the flattened form of that matrix; only the 92 by 92 block that corresponds to occupiable squares is ever scored.

The value head

The value token predicts the categorical outcome (win, draw, loss) rather than a scalar. Its expectation with win 1, draw 0 and loss is the baseline in the advantage and the quantity search averages; its full distribution is the target of the return. The distribution matters for search because draws are common in self-play, much more common than between humans, and a scalar would hide the difference between a certain draw and a coin flip.

Illegal moves get a logit of negative infinity before the softmax, so the policy only ever puts mass on the arbiter's legal list; the engine's mask is a 10,000-entry boolean array per position. The magnet policy that the reverse KL pulls toward is computed from the same mask: uniform over the pieces that have at least one legal move, then uniform over that piece's moves. A Scout with eighteen moves therefore gets the same total mass as a Sergeant with one under the magnet, which keeps the regulariser from favouring mobile pieces simply because they have more actions.

Cost of a forward pass

For a batch of 1,536 positions per GPU the move network is the bottleneck of the whole loop: the paper's time-expenditure figure shows iteration speed limited by move-network data collection and training. At the small sizes used for smoke runs here (embedding 128, four layers) one forward pass over a batch of 1,024 positions is a few milliseconds on the Radeon 8060S; the paper-sized network is roughly ten times that. The training package reports environment steps per second and network positions per second separately so that the two can be balanced.