Concepts

The belief network

Search needs concrete positions to roll out, and the counting prior from the observation planes is a poor guess: it says nothing about how a Marshal is hidden or where Bombs go. Ataraxos trains a belief network to maximise the log-likelihood of the true hidden pieces given the information state, over trajectories of its final self-play policy.

B23F510B23F510B23F510B23F510B23F510B23F510What the observer knows· which squares hold an opponent piece· which of those have moved (not a Bomb or the Flag)· which were revealed by a battle or a long Scout move· how many of each type have been lostSetups per side: 40! / (8! 5! 4!³ 3! 2! 6!) ≈ 1.4 × 10³³Texas hold'em, for comparison: 1,326 hands.Bar length: probability under a uniform prior over the remaining pieces. The belief network learns a sharper one.
The belief network decodes the hidden opponent squares one at a time, each type conditioned on the ones already sampled.

Architecture

An encoder of depth 6 over the 92 square tokens, like the move network, followed by a decoder of four blocks whose keys and values are the encoder outputs restricted to the squares that hold hidden opponent pieces. The decoder emits the type of each hidden piece in row-major order, conditioning on the types already emitted, so a sample is a joint configuration rather than a set of independent per-square guesses and the army composition constraints can be respected. Embedding 512, 8 heads, 57.1 million parameters, trained on 4 H100s for 4 days after the reinforcement learning run finished.

Dropout on purpose

The belief model is trained on self-play, but at test time it faces humans and other bots whose setups and moves are far from the self-play distribution. Dropout of 0.2 during training is there to make the model generalise to those out-of-distribution positions rather than to regularise a small dataset. The paper's appendix on further improvements reports that interleaving temporal attention or recurrence with the spatial attention gave much stronger belief models per unit of compute, which is the first change to try here.

Why self-play beliefs make search opponent-independent

The search averages the move network's values over positions sampled from the belief network, with rollouts played by the move network. Because the belief network approximates the self-play policy's own posterior and the rollouts and values are the self-play policy's, the averaged values estimate self-play action values whatever the actual opponent does. That is the sense in which the search stays safe against opponents it has never seen: it never assumes it knows the opponent's strategy, only its own.

Search needs joint samples, not marginals. With independent per-square guesses the sampled army could contain two Marshals or no Flag; the autoregressive decoder avoids that by conditioning each type on the ones already placed and masking types whose remaining count is zero. At search time the number of samples is about a thousand divided by the number of legal moves, so a position with 40 legal moves gets 25 configurations and each is rolled out once per candidate. A position with few legal moves, typically late in the game, gets hundreds of samples, which is where beliefs matter most because most pieces are known and the few unknowns decide the outcome.

Targets in this program

The environment bindings expose the hidden types of every opponent square in the acting player's frame as belief targets, alongside the observation planes, so belief data comes for free from any self-play run. Until a belief network exists, the search rung's rollouts use the counting prior; the registered comparison of learned against counted beliefs is described on the Ataraxos approach page.