Log · 2026-09-19
Opponent-limited subgame solving
Hypothesis, stated before any code or game (study
subgame-solving/opponent-portfolio, theory T2 of the
program, brief agents/briefs/opponent-portfolio.md):
limiting the opponent in the kluss subgame to a choice among N whole-game
strategies at the root (OLSS-II, Liu et al. 2023) turns the subgame nearly
single-agent, so the same wall-clock budget buys a much deeper tree. At a
50 ms budget OLSS-II with N of two beats full KLUSS at the same budget on
200 paired games; at 200 ms and above the order reverses, because the
portfolio caps what the opponent can do while full KLUSS keeps its safety.
The crossing budget is the measurement; if no measured budget shows OLSS-II
ahead, or it loses to plain expectimax, the theory is disproved here. The
T1 result (kluss lost to expectimax at both budgets) means the crossing may
sit below 200 ms. The result of this study, one way or the other, is filed
below this hypothesis after the runs.
Result: the portfolio wins at every measured budget, the predicted reversal never comes, and expectimax stays ahead
(report, five records from
2436cdbe to
4ffa3c2f;
the registration ids are in the report's reproduction note.) The
searcher gained mode: "olss2" and the portfolio list: the opponent
chooses one of N whole-game strategies at a CFR root node and plays it
below the root as chance, the portfolio members (greedy, a cheap
expectimax, self, the leaf policy at a temperature) act on the
opponent's observation, and tests pin determinism, the information
boundary and the reduction of N of one to single-agent expectimax on a
small position. Smokes of four games per cell on throwaway seed 900
verified the plumbing before the runs; smokes are not evidence.
Five cells under the training rules, --threads 8, other agents sharing
the machine. Against full kluss at equal bench cost: N of one at 50 ms
scored 61.0% effective (54.1% to 67.5%) on 200 paired games, contrast
0.220 (0.066 to 0.374); N of two at 50 ms scored 60.5% (53.6% to 67.0%),
contrast 0.210 (0.064 to 0.356); N of two at 200 ms scored 62.0% (55.1%
to 68.4%), contrast 0.240 (0.085 to 0.395). The registered reversal at
200 ms did not happen. The optional 1 s cell (registered at 100 paired
games, seeds 0 to 49) scored 55.0% (45.2% to 64.4%), contrast 0.100
(minus 0.111 to 0.311) covering zero: the margin narrows as the budget
grows and no crossing sits inside the measured range. Against
expectimax:{"samples":16,"depth":2} the 200 ms portfolio player scored
38.0% (31.6% to 44.9%), contrast minus 0.240 (minus 0.390 to minus
0.090), so the log's disproof condition (a loss to plain expectimax) is
met even though the mechanism works at every measured budget. The
diagnostic trace puts 92% of the solved opponent mixture on greedy
against 8% on self 0.3, and the purified root keeps one action: the
portfolio pays by making the subgame single-agent, not by covering
styles. Costs on the idle bench, one thread: kluss 53, 139 and 467 ms at
2,500, 5,000 and 10,000 expansions; olss2 N of two 51, 160 and 770 ms at
5,000, 10,000 and 20,000; olss2 cost reaches 8.1 s at 40,000, so the
crossing hunt above 1 s is expensive and left to a future registration.
OLSS-I was not implemented; the wider-portfolio cell against expectimax
is the named next step.
Failures and deviations. The session that ran cells (a) to (c) was ended from outside (exit -15) with cell (d) in flight; the matchup process ran to completion uninterrupted and wrote all 200 records, and the record for cell (d) says so. The 1 s cell was calibrated under load (818 against 848 ms on the bench at the time); on the idle machine its sides read 467 and 770 ms, so that cell favours the olss2 side more than the smaller cells do theirs. One clippy fix to the uncommitted trace tool was needed before the checks passed. No games were censored: every planned game completed, 900 of 900 across the five cells.
Attribution: baseten/moonshotai/Kimi-K3 through opencode.
A structured prior over initial placements
Hypothesis, stated before any code or game (study setups/structured-prior,
theory T9 of the program, brief
agents/briefs/setup-prior.md): with the same move policy in both seats, a
structured setup source, Flag on the back two rows with Bombs likely beside
it, Scouts forward, Miners back, Marshal and General apart and the Spy near
the General, beats uniform setups on a per-seed paired contrast with the 95%
interval above zero for greedy, rollout and
expectimax:{"samples":16,"depth":2} under the training rules, and the gain
is larger for the players that attack early. The structured source keeps more
than 3 bits of entropy on the Flag's square, and a null cell (uniform
against uniform on the same seeds) has an interval covering zero. Exploratory
motivation, not evidence: on 1,200 unregistered games a front-row Flag lost
61.5% of sides against 32.7% on the back row.
Result: supported on all three policies
The sampler is Setup::structured in the engine (constructive placement,
sixteen bounded retries, uniform fallback; 10,000-sample property tests
cover every rule, determinism and stream separation), the setup
parameter is one WithSetup wrapper applied in armies_search::policy
and listed in every catalog schema, and the rules table lives in the
server's docs/setups.md. Smoke runs of four games per cell (throwaway
seed 900) verified the plumbing before registration: the structured
contestant's Flag sat in the back two rows and paired seeds shared its
setup across seats. Smokes are not evidence.
Four registrations ran under the training rules, 400 paired games each on
seeds 0 to 199, --threads 8, each cell alone on the machine
(report):
- greedy: structured 71.5% effective (0.669 to 0.757), contrast 0.430 (0.339 to 0.521). Record
- rollout: structured 81.5% (0.774 to 0.850), contrast 0.630 (0.551 to 0.709). Record
- expectimax sixteen worlds depth 2: structured 83.5% (0.795 to 0.868), contrast 0.670 (0.589 to 0.751). Record
- null, uniform against uniform: 48.4% (0.435 to 0.533), contrast minus 0.033 (minus 0.143 to 0.078), covering zero as registered. Record
The decision rule is met for all three policies. Diversity over 10,000
samples per source (setupstats binary, scripts/setup_stats.py):
Flag-square entropy 4.14 bits structured against 5.32 uniform (the rule
asks for 3); a Flag with no adjacent Bomb in 0.3% of structured setups
against 57.1% of uniform ones. Early endings: the null cell ended within
20 plies in 7.2% of games, split evenly; the treatment cells in 4.0%, 7.2%
and 6.2%, and all seventy of those games were lost by the uniform side.
The brief expected the largest gain for the early attackers; the measured
gradient runs the other way (greedy 0.43, rollout 0.63, expectimax 0.67),
which the report reads as search converting a setup edge more reliably.
Deviations and failures: none against the brief. The diversity stats use
the small arena binary the brief allows, because the Python bindings do
not expose setup sources. The cells were re-run twice for provenance:
once after the server commit, and again after rebasing over the merged
kluss branch (the registry conflict resolved by wrapping its arm with the
same setup parameter). Every re-run reproduced the game outcomes exactly;
only wall-clock timings moved, and the records say which runs shared the
machine. The pre-existing
tests/training/test_runlog.py::test_resume_equality_cpu flake (residual
CPU nondeterminism documented yesterday by the run-layout study) failed
once on this worktree before any change of mine; unrelated to this study.
Whether structured becomes the registry default is left to the user, as
the brief instructs.
Curated games: seed 0 from both seats of each treatment cell under
records/games/structured-v-uniform/ (6 records, indexed and replayed).
The compiled armies Python module was not rebuilt; no
server/crates/python change.
Attribution: baseten/moonshotai/Kimi-K3 through opencode.
The scaled run is registered
Hypothesis, stated before any measurement: of the three candidates (the
small preset, a medium one at embedding 256 and 8 layers over 1,536
environments, and the paper's per-GPU numbers), the largest whose iteration
is under three minutes is the one the multi-day run takes, and after 48
hours its EMA move network beats greedy on 200 paired games, after 96
hours expectimax at sixteen worlds and depth two, with the
Elo-against-time curve the result either way. This is study
ataraxos/scaled-run, theory T5 of the program;
the training run itself is launched by the user, never by this agent.
Sizing, measured under the GPU lock on 2026-09-19 with
scripts/size_presets.py, three iterations per candidate from scratch,
under the day's load (load average 2.4 to 9.9 of 32 threads): small 31.5 s
per iteration (25.5, 32.6, 36.4) at 5.96 GiB and 10,350 collection steps
per second; medium 76.4 s (63.7, 72.1, 93.4) at 9.51 GiB and 5,970 steps
per second; large, the paper's numbers, 135.4 s (95.2, 168.0, 143.1) at
12.14 GiB and 3,900 steps per second. All three sit under three minutes,
so the largest qualifies and PRESETS["scaled"] takes the paper's
per-GPU numbers with the measurement in its docstring. At that pace the
run reaches about 1,280 iterations by hour 48.
The draw rule is confirmed as the paper's, not changed: the training rules
preset carries exactly Appendix D Table 17 (a draw after 100 consecutive
battleless moves and after 4,000 moves, Rules::TRAINING in
crates/engine/src/state.rs), and the advantage filter is on (top
quantile 0.75 with a 0.01 magnitude floor). The smoke run's battleless
stalling reads as an early-training artefact, so no change to the
objective is proposed; the draw share is logged per iteration and is a
first-class diagnostic beside entropy and clip rate (a share above half
sustained is the registered disproof's second clause).
Both milestones are registered and nothing runs yet: the
48-hour match against greedy
and the 96-hour match against expectimax,
200 paired games each (both seats per seed, matched setups, seeds 2000
to 2099, training rules), decided by the per-seed contrast with its 95%
interval above zero. scripts/milestone.sh RUN HOUR OPPONENT finds the
checkpoint whose run wall clock is nearest the hour, plays the registered
games, writes runs/<uuid>/ output and files the record; it reads the
registration, so the match cannot drift from it, and its --smoke mode
plays throwaway games and files nothing.
The dry run (runs/scaled-dry, kept) proved the schedule end to end:
launched through just launch as the systemd unit armies-scaled-dry,
four iterations of the scaled preset with --evaluate-every 2 --eval-games 4 wrote checkpoints at every even iteration and evaluations
against greedy and expectimax to evaluations.jsonl; stopped by signal
mid-run it marked itself stopped; resumed, it finished, and a second
resume with the schedule flags repeated evaluated again at iteration 6.
Six iterations took 868 s of run wall clock at 133 to 180 s per
iteration, draw share 0.61 to 0.67 early in training, and the
expectimax evaluations ran through the new Python search bridge. The
milestone smokes against both registrations played four throwaway paired
games each and filed nothing.
Failures and deviations, all recorded: the run-layout study's CPU
resume-equality test flaked before any of my changes (a policy_loss
delta of 1.1e-5 against its 1e-5 tolerance, while training/README.md
documents up to 5e-4 of per-process CPU noise); its float tolerance is
widened to the documented bound in a separate commit. The Python
bindings could not play expectimax, which the brief's launch and
milestone commands need, so armies.RustPolicy now builds from the full
policy registry (armies_search::policy, a two-line server change) and
the shared armies module was rebuilt; the learned-belief agent works
in the same crate and rebuilt the shared module twice during the
session, so my GPU jobs re-verified the bridge before each launch and
the merge will carry both changes. A resumed run keeps its
hyperparameters but not its schedule flags, so the resume command must
repeat --save-every and --evaluate-every; the first dry-run resume
omitted them and silently evaluated nothing, which is how it was found.
just watch -- -c watch.toml never worked (the recipe is positional, so
-c became a recipe name); the recipe now passes any .toml argument
as a config, and docs/monitoring.md says so. The dry-run's second
resume had to be re-polled after a stale status.json made the first
wait exit early; no training time was lost.
The commands the user runs are in the report: launch, watch, both milestones, and the resume.
Attribution: baseten/zai-org/GLM-5.3 through opencode.
The learned belief against the counting prior
Hypothesis, stated before any measurement (study
subgame-solving/learned-belief, theory T3 of the
program): a belief network trained on the small
preset's self-play assigns a higher log-likelihood to the true types of
hidden opponent pieces on held-out games between other players (the
replay library's greedy, expectimax, rollout and random games) than the
counting prior the observations already carry, and expectimax at 16
worlds and depth 2 that samples its worlds from the network beats the
same expectimax sampling from the prior on 200 paired games under the
training rules. If only the match holds, the belief fits the searcher's
own style; if only the likelihood holds, the search does not use what
the belief knows. Both outcomes are filed. The disproof is no gain in
either measure.
Method, fixed in advance: the move network comes from thirty small preset iterations from seed 0; the belief network trains on its self-play (400 collection steps over 256 environments, 2 epochs). The likelihood measurement replays every game in the replay library through the engine and scores, at every ply of the side to move, the log probability of every hidden opponent piece's true type under the counting prior's belief planes and under the network's marginal estimated from 64 sampled assignments, floored at 1/128 so a zero count is not minus infinity. The match plays 200 paired games, both seats, the same two setups drawn per seed, the candidate sampling 16 assignments from the network per move and the control sampling from the prior, both searching depth 2. The decision rule is the per-seed contrast with a 95% interval above zero.
A sampling bridge is needed first: the Python bindings gain a Searcher
wrapping the expectimax search, seeded for determinism, that either
samples worlds from the counting prior or searches worlds handed to it
as hidden-type assignments. Positions from games the belief policy itself
played are not held out anywhere in this measurement; the replay library
holds none of its games.
Result
Both measurements are negative
(report,
record,
registration d72d5305).
The held-out likelihood, on the library's 64 games, 10,588 positions and
270,223 hidden pieces: the counting prior 2.096 nats per piece and 24.2%
top-1, the network 2.293 and 21.4%, worse in every phase and for every
acting player. The match on seeds 0 to 99 from both seats: expectimax
with belief-sampled worlds 90 wins, 1 draw, 109 losses (45.25%, interval
38.5% to 52.2%) against prior-sampled expectimax, per-seed contrast
minus 0.095 (minus 0.217 to 0.027), 196 of 200 games ending by Flag
capture. The decision rule is not met; the interval overlaps zero with a
negative point estimate, which is inconclusive, not equivalence. The
belief side cost 52.9 ms per move against the control's 4.3, both
measured while other agents used the machine. On its own self-play the
network trained to 1.62 nats per piece, so it fitted a policy thirty
iterations old rather than a transferable belief. The study closes at
this rung; the retest belongs to the scaled run's checkpoint, and the
bridge (the Searcher class, a search-crate test that the sampled and
the given-worlds entry points agree bit for bit) is on main for it.
Failures and deviations. The small-preset training crashed at iteration
21 with an illegal action: the setup network sampled a legal setup whose
frontier row is Bombs and the Flag with the movables facing lakes, the
side to move had no legal move at ply 0, and the engine ends a no-move
game only after a move is applied, so the collector stepped an empty
legal mask and went uniform over all 10,000 actions. The crash was
reproducible from the iteration-20 checkpoint; the fix filters such
setups out of the self-play pool (first_mover_can_move in
armies_train.selfplay, with tests pinning the engine behaviour), the
run resumed from the checkpoint, and the trajectory from iteration 21
differs from an unguarded uninterrupted run. BeliefNet.sample had a
second latent bug this study tripped on: decoding in row-major order
could spend its movable types before reaching squares that have moved,
emptying the whole feasibility mask and degenerating the softmax to
uniform, which the engine's assignment validation rejected; the sampler
now keeps a lookahead so a completion always exists
(armies_train.networks, test tests/training/test_belief_sample.py).
The shared venv armies module was overwritten twice by other agents'
just build-env runs while this study used it, so every Python here ran
against a private install of the module built from this worktree
(.armies-module, on PYTHONPATH); the module was also rebuilt once
with just build-env as the brief asks. The optional per-game
GameRecordDto records were not written. The first launch of the
training died to a tool timeout after two minutes; it and all later GPU
jobs ran detached with setsid under the lock.
The training log: 30 small-preset iterations at about 10,300 steps per second and 42 seconds each (the throughput agent's attention fix), the belief network 6,394,124 parameters on 102,400 positions in 2 epochs.
Attribution: baseten/zai-org/GLM-5.3 through opencode.
Three briefs for the next wave, and two flaws found while writing them
Planning, no new evidence. Three briefs were added under agents/briefs/
and one was amended, for the work that follows the scaled run's launch:
ntuple-leaf (rung 3, CPU only, so it can run beside the GPU run),
rnad-equal-compute (T8, prepared by an agent, then twelve GPU hours per
arm launched by the user) and belief-retest (T3 again, on the scaled
run's hour-48 checkpoint). docs/program.md has the order of work.
The n-tuple prototype leaks hidden information. training/armies_train/ntuple.py
builds each afterstate by playing the move on a copy of the true game, so
an attack on a hidden piece is resolved with the defender's real type
before the move is chosen, and a capture of the Flag shows as a win. No
registration ever used it (runs/ntuple-60 was a smoke run and is not
evidence), so nothing is withdrawn, but none of its numbers may be cited.
The brief replaces it with a Rust player that works from the observation
alone and scores an attack on a hidden piece as an expectation over the
defender's type, and with a second table set over fully typed worlds for
the search leaf, which is what a determinized leaf actually sees.
The first belief study's test set favoured the prior by construction.
Its library games are engine players with uniform random setups, and
under a uniform setup the counting prior is the exact distribution of
hidden types before the first move. A network trained on a policy's own
setups is off its distribution there. The record stands as filed; the
retest and the infoset-belief study now decide on held-out self-play of
the policy the model was trained on, add structured-setup games, keep the
library beside them for continuity, and score the model against training
steps so that too little training can be told from a limit. T7's
prediction in docs/program.md was amended before any T7 evidence: its
bar was the belief decoder, which the first study put behind the counting
prior (2.293 against 2.096 nats per hidden piece), and is now the prior.
Two rules joined the agent contract. A run started with just launch
does not take the GPU lock, so an agent checks for an active training
unit before GPU work and does not share the card with it; run wall clock
counts only running time, so the user can pause the scaled run for a GPU
study without moving its milestones. And agents/delegate clean now
moves a worktree's run output into the canonical runs/ before removing
the worktree, because git worktree remove --force deletes ignored
files and the first belief study's trained checkpoint lived only there.
Attribution: claude-fable-5-1 (Claude Code).
Selecting a pool of setups by self-play
Hypothesis, stated before any code or game (study
setups/selection-by-self-play, theory T9 of the
program, brief agents/briefs/setup-selection.md): a
pool of 32 setups selected by successive halving from 1,024 draws of the
structured prior, under expectimax:{"samples":16,"depth":2} in both seats,
beats the prior by a per-seed contrast above zero at 95% on 400 paired games
when both sides face held-out opposing setups, and its exploitability proxy,
the score of the best single opposing setup the same procedure can find
against it, is within the prior's interval. If the gain disappears against
held-out opponents, selection was fitting the opposing pool and the report
says so. The mechanism, fixed before measuring: every surviving setup plays
eight paired games a round (four fresh prior opponents, each from both
seats), the top half survives, and from the second round a quarter of the
pool is replaced by mutants of survivors (two pieces swapped, rejected when
the mutant breaks a rule of the prior). A 200-paired-game repeat with
greedy in both seats asks whether a pool selected under one policy
transfers to another.
Result: supported on both clauses
The pool of 32 setups (five halving rounds, 15,872 selection games,
procedure seed 20260919, 3,344.6 s on 8 threads; run
0b727d71)
beat the prior on held-out opposing setups, per-seed contrast 0.315
(0.223 to 0.407) on 400 paired games, effective 65.8%
(c75d346c),
and beat uniform by 0.792 (0.728 to 0.857) against the prior's 0.670 on
the same seeds
(6fb87c90).
The exploit procedures (16,368 games each, seeds 20260920 and 20260921;
9fbb92bf)
found two mutant exploiters, and their 400-game evaluation cells priced
the pool the harder source: the pool's proxy 0.150 (0.043 to 0.257)
(ca2fd11a)
against the prior's 0.458 (0.364 to 0.551)
(1cea5672),
difference minus 0.308 (minus 0.450 to minus 0.166). The greedy transfer
held at 0.230 (0.111 to 0.349) over 100 seeds with a 34% draw share
(be95e294).
Diversity is the trade: all 32 Flags on the back row, 3.2 bits of
Flag-square entropy against the prior's 4.14. The study's rule is met;
the pool is the strongest native setup source, the browser build cannot
read pool files, and the default stays uniform for the user to change.
Report. Six curated games sit under
records/games/setup-pool/.
Failures and smokes worth keeping: the previous session
(setup-selection-20260919T044651Z) was ended by a signal from outside
while the selection run was detached; the job survived it and finished all
rounds, and this session filed its output. A four-game matchup smoke of
the pool and exploiter sources and a 48-game exploit smoke (seed 901)
checked the plumbing before the registered runs; smokes are never
evidence. cargo test --workspace, the wasm32 check of the server and
scripts/check_content.py all pass. pytest tests/training failed twice
on the two resume-equality tests in tests/training/test_runlog.py
while another agent's pytest ran beside mine (their docstring warns of
per-process noise; two of my own leftover pytest processes were part of
the contention and I killed them); the same tests and then the full
suite passed when the machine quieted, 32 passed, 1 skipped.
Attribution: baseten/zai-org/GLM-5.3 through opencode.
Update-equivalence search on the small checkpoint
Hypothesis, stated before any code or game of this study (study
ataraxos/update-equivalence, theory T4 of the
program, brief agents/briefs/update-equivalence.md):
on the small preset's checkpoint, a player that runs the paper's
update-equivalence search with 200 rollouts at depth 10 (alpha 0.002,
beta 0.02) beats the same checkpoint sampling its move policy directly on
200 paired games with a per-seed contrast whose 95% interval is above
zero; 1,000 rollouts at depth 40 beats the 200-by-10 cell under the same
rule; and setting the network KL coefficient beta to zero loses to the
network alone, reproducing the sign of Table 17 of the Ataraxos paper,
where the beta-zero ablation scored Elo 1733 against the raw network's
2095. A fourth cell, the network alone against itself, is the null pair
and its interval must cover zero. If no search setting beats the network
alone, the value head is the first suspect and its calibration
(Pearson correlation and a five-bin reliability table against realised
self-play outcomes on 2,000 positions) is reported beside the matches.
Method, fixed in advance: the checkpoint is the small preset trained 30
iterations from seed 0 with its belief network (400 steps), which the
learned-belief study left at runs/belief-policy in the canonical
checkout; this study reuses it unchanged rather than retraining. The
search player samples its opponent-configuration worlds from the belief
network and plays the magnetic-mirror-descent move as
armies_train.search.search_move computes it; the network-alone player
samples its move policy directly. Every cell is 200 paired games (100
seeds, both seats, the same two setups per seed drawn from the seed
through each contestant's own setup stream) under the training rules,
played by a new match subcommand of the training package that writes
the arena's matchup JSON shape with per-game search diagnostics. The
1,000-by-40 cell is priced from a four-game smoke before its full count
is committed; the brief allows it on 100 games if the budget demands,
and the smaller count is registered before it starts. Smokes are
plumbing, never evidence.
Continuation, 2026-09-19 (second session, same study): the measured
costs changed the counts, and the registrations follow the machine
rather than the brief's first choice. A search move at 200 rollouts and
depth 10 costs 1.1 s under the shared machine's load (profiled move and
the four-game smokes), and the checkpoint's games run 350 to 1,000 plies,
so a pair costs 320 to 650 s: the brief's 100 pairs per cell are 18 to
36 hours of machine time, not one afternoon. The 1,000-by-40 cell
measures 8.4 s per move (one probe, 988 rollouts), about 1.8 hours per
pair and about 90 hours for the brief's count, which is a user-scale
run; it is registered and priced, not played. Registered instead, before
any evidence game: the 200-by-10 cell and the beta-zero cell at 12
paired seeds each (24 games, seeds 0-11), the null pair at the full 100
paired seeds (200 games, seeds 0-99), and the 1,000-by-40 cell at its
brief count, status registered. Deviations found by the second session
while testing the match module: a seat-attribution bug in the first
session's code (the red seat always used contestant 0's player; fixed,
both seats now play their own contestant, and the tests assert the
search diagnostics follow the contestant), a missing set_setups
binding on TorchEnv that the calibrate smoke caught, and a
reproducibility limit: the CPU net forward jitters by about 1e-5 between
identical calls (allocation-dependent kernels), which can flip a
near-uniform setup choice, so games pair exactly within a run (same
seed, same two setups, same seats, streams seeded per game) but are not
bit-reproducible across runs on CPU; the GPU path reproduced setups
exactly across processes in the smokes. The null pair and the
calibration measure the value head beside the matches. Smokes with
throwaway seeds 9002-9003 were plumbing; the smoke preset's value head
predicts a near-constant value, which also showed the calibrate module
needs a defined answer for a constant predictor (Pearson absent, not
nan). Attribution: baseten/zai-org/GLM-5.3 through opencode.
Results, update-equivalence, 2026-09-19. The three cells that ran are
inconclusive with positive point estimates, the null pair passes, the
paper's beta-zero sign did not appear, and the calibration names the
mechanism. Cell (a), search at 200 rollouts and depth 10 against the
network alone, registration
experiments/77aebde0-74c7-4a7a-9835-9c1f853b159e, 24 games on seeds 0
to 11: 5 wins, 19 draws, no losses, 60.42% effective, per-seed contrast
0.208 with interval -0.016 to 0.433, which overlaps zero, so
inconclusive; 339.7 ms per search move against 2.99 ms for the plain
side, both measured while other agents shared the machine. Cell (c),
beta zero, registration experiments/8c2d3e34-5fb9-4892-81a2-00689163a3f9,
24 games: 2 wins, 22 draws, no losses, 54.17% effective, contrast 0.083
with interval -0.027 to 0.194, inconclusive; the paper's Table 17 sign
(beta zero losing catastrophically) is not reproduced, and with the q
values spreading 0.023 across legal moves there is nothing to overfit.
Cell (d), the null pair, registration
experiments/85f9ec2b-1530-462d-9a8b-84b465a7917a, 200 games on seeds 0
to 99: contrast 0.015 with interval -0.054 to 0.084, covers zero, rule
met; 159 of 200 games drew to the quiet limit, which is the checkpoint's
own game shape and the power limit of every cell. Cell (b), 1,000
rollouts at depth 40, registration
experiments/40af5091-8456-47e8-88ba-a74a6f3002ba, was priced at 8.4 s
per search move measured on one middlegame probe, about 1.8 hours per
paired seed and about 90 hours for its 100 pairs, so no game was played
under it and it stays registered for a user-scale run. The calibration,
2,000 positions from the checkpoint's own self-play (16 games, 12 draws
and 4 blue wins, mean 747.8 plies): the value head's predictions never
leave the middle fifth of the outcome scale (all 2,000 in the -0.2 to
0.2 bin, mean -0.074) and their Pearson correlation with the realised
outcomes is -0.19, so the head carries no ranking for the search's
update to use; this is filed as the first suspect the brief named.
Failures and deviations: the first session's seat-attribution bug (its
smoke is superseded), a missing TorchEnv.set_setups the calibrate
smoke caught, one flaky pre-existing training test
(test_resume_equality_cpu, passes in isolation, its CPU noise is
documented in training/README.md), and the cell counts above, all
registered before the first game. Records: cells (a), (c) and (d) in
records/runs/ with their raw JSON beside them; the report is
reports/update-equivalence.mdx with the chart asset
content/assets/update-equivalence.json. Attribution:
baseten/zai-org/GLM-5.3 through opencode.