Reports · 2026-09-19

A small-run belief network loses to the counting prior

Neither measure supports the learned belief at this scale

On the replay library's 64 games (10,588 positions, 270,223 hidden pieces, 64 sampled assignments per position, probabilities floored at 1/128), the counting prior assigns the true hidden types 2.10 negative log-likelihood per piece and 24.2% top-1 accuracy; the network assigns 2.29 and 21.4%, worse in every phase and for every acting player in the library. In the registered match on seeds 0 to 99 from both seats, expectimax with belief-sampled worlds won 90 of 200 (45.25%, interval 38.5% to 52.2%) against the same expectimax with prior-sampled worlds; the per-seed contrast is minus 0.095 with a 95% interval of minus 0.217 to 0.027, which overlaps zero, so the result is inconclusive with a negative point estimate. The decision rule is not met.

Expectimax with belief-sampled worlds against prior-sampled worlds

Expectimax with belief-sampled worlds against prior-sampled worlds
Row	Value	Low	High	n
expectimax+belief	45.2	38.5	52.2	200
expectimax	54.8	47.8	61.5	200

Both sides search 16 worlds at depth 2 under the training rules; the candidate draws its worlds from the belief network, the control from the counting prior, on seeds 0 to 99 from both seats; built by scripts/build_belief_assets.py.

The two measurements agree, and they say the same thing twice: the belief network did not learn a belief better than the exchangeable prior. On its own self-play positions it reached 1.62 nats per piece, so it fitted its own distribution, but that distribution came from a move network thirty iterations old (about 10,300 environment steps per second for 21 minutes) and setups drawn from a setup network that had not yet learned where the Flag belongs. What it learned does not transfer to games between random, greedy, expectimax and rollout, where the uniform posterior is exactly right whenever nothing has moved and everything hidden is exchangeable: the counting prior is hard to beat in the opening, and it stays ahead through the middlegame and the endgame here.

Negative log-likelihood of the true hidden types by game phase

Negative log-likelihood of the true hidden types by game phase
Row	Value	Low	High	n
counting prior, opening, ply under 100	2.203	2.203	2.203	155122
belief network, opening, ply under 100	2.368	2.368	2.368	155122
counting prior, middlegame, ply 100 to 300	1.98	1.98	1.98	99927
belief network, middlegame, ply 100 to 300	2.212	2.212	2.212	99927
counting prior, late, ply over 300	1.765	1.765	1.765	15174
belief network, late, ply over 300	2.054	2.054	2.054	15174

Mean negative log-likelihood of every hidden opponent piece's true type on the 64 replay-library games, counting prior against belief network, 64 sampled assignments per position and probabilities floored at 1/128; built by scripts/build_belief_assets.py.

Cost matters as much as the score. The belief side spent 52.9 ms per move against the control's 4.3 ms, because every candidate move sampled 16 assignments from the network on the GPU before searching them, both measured while other agents shared the machine. Ataraxos trains its belief on the final policy's trajectories for four days on four H100s before search uses it (57.1 million parameters); this study spent 102,400 positions from the self-play of a 2.8-million-parameter move network thirty iterations old, which is the honest scale of one GPU-hour. The negative result bounds that scale: a belief trained on a barely-super-random policy carries no information the exchangeable prior does not already have.

Why both outcomes are filed

The study's design separated two failures: if only the match had failed, the belief would fit the searcher's own style; if only the likelihood had failed, the search would ignore what the belief knows. Both failed, which points at the training scale, not the bridge: the sampler that hands the search its worlds is verified against the prior-sampling path bit for bit (a test in the server's search crate asserts the two entry points agree on the same worlds), and the match measured the belief, not a plumbing difference. The study closes the direction at this rung and hands the retest to the scaled run: when the T5 checkpoint has a policy that beats greedy, its self-play trajectories are the data on which a belief has something to learn that the prior does not know.

Run record

Reproduction

Registration experiments/d72d5305-0de2-4f4c-b8fd-f1f24c3ab4dd.toml. Commands, from the research root with PYTHONPATH=training:.armies-module (the shared venv module was repeatedly overwritten by other agents' builds, so this study ran against a private install of the same crate):

.venv/bin/python -m armies_train train --preset small --iterations 30 --seed 0 --out runs/belief-policy --registration d72d5305-0de2-4f4c-b8fd-f1f24c3ab4dd
.venv/bin/python -m armies_train belief --run runs/belief-policy --steps 400 --envs 256 --epochs 2
.venv/bin/python scripts/belief_eval.py --run runs/belief-policy --out runs/belief-policy/belief-eval.json
.venv/bin/python scripts/belief_match.py --run runs/belief-policy --games 200 --seed 0 --out runs/belief-policy/belief-match.json
python3 scripts/build_belief_assets.py --run runs/belief-policy

The training crashed at iteration 21 on a setup-network setup that immobilises the first mover; the pool refresh now filters such setups and the run resumed from the iteration-20 checkpoint. The belief-eval pass took 24 minutes of GPU, the match 12; the raw JSON files sit beside the record with their SHA-256 digests.