Concepts

The setup network

The setup is a decision made once, in secret, with an enormous action space: more than arrangements. Ataraxos handles it as a sequence. A decoder-only transformer of depth 4, embedding 512, 8 heads and 12.6 million parameters places pieces in row-major order over the forty home squares. Its input is the sequence of piece types placed so far with learned positional embeddings; at each prefix it outputs a distribution over the type to place on the next square, a three-way outcome estimate for the game that will follow, and a prediction of the conditional entropy of the rest of the setup. A whole setup trains in one forward and backward pass, which is why the decoder-only form was chosen.

Learning

The advantage for the placement at prefix combines the game's outcome and an entropy bonus:

with the policy loss of the same clipped form as the move loss and a reverse KL of 0.1 to the policy that generated the setup. The entropy prediction is trained by squared error against the realised conditional entropy divided by ten. Five epochs over the setups of the games finished in the last data-collection period, batches of 1,024 per GPU, Adam at , gradient norm clipped at 0.5, EMA 0.999. Removing setup learning and using uniform random setups flattened the whole Elo curve in the paper's ablation, both because random setups are weak and because self-play against them teaches the move network wrong assumptions about opponents.

What it learned

The paper reports that Ataraxos bombs in its Flag about two thirds of the time, mirrors its setups left to right after sampling so that both orientations are equally likely, and, according to the players who faced it, uses aggressive setups with high-value pieces near the front, high Bombs on the third and fourth rows, and back-corner Flags more often than humans do, with less predictable structure overall.

The heatmap above is illustrative: it encodes those qualitative statements as a plausible distribution and is not the paper's measured figure. The figure will be replaced by measured placement frequencies from this program's own setup network once a run exists.

Entropy as a first-class output

The conditional-entropy head is unusual. Its target is the realised entropy of the setup distribution given the prefix under the parameters that generated it, scaled by a tenth, and its prediction enters the advantage as a bonus for placements that keep future choices open. The effect is a maximum-entropy objective applied through the value function rather than as a loss term: a placement is credited not only for the outcome it led to but for how much randomness it preserved, and the temperature on that credit is the annealed coefficient from the damping schedule. The training logs show setup entropy falling smoothly over the run and the proportion of clipped setup updates falling with it.

The uniform baseline

Every rung below the RL rung uses uniformly random setups on both sides, which keeps the comparison between search methods clean but means their absolute strength is understated. The first thing the RL rung changes is the setup distribution, and the ladder records both the setup source and the move policy for every player.