Approaches

Regularised Nash dynamics under equal compute

Which learning rule reaches further on one machine: the paper's damped clipped policy loss, or DeepNash's reward transformation with replicator dynamics?

The mechanism

The training package's R-NaD variant keeps the move network and the environment and swaps the update: rewards are transformed by the log-ratio of the current policy to a regularisation policy with η = 0.2, values are estimated with v-trace on the transformed rewards, the policy takes NeuRD steps on its logits, and the regularisation policy is replaced by the current policy on a fixed schedule. The DeepNash page sets it against dynamic damping.

The plan

Two runs of equal wall-clock time on the same hardware, same network size and same environment count, one with each update, with setups from the same fixed pool so that setup learning is not a confound. Evaluate both against greedy and expectimax at matched checkpoints, then against each other on 200 paired games. A second pair adds the paper's advantage filtering to both, to see whether the filter is rule-specific.

Decision rule

Neither rule is promoted over the other unless the head-to-head paired contrast has a 95% interval excluding zero. A tie with a difference in time per iteration is recorded as such; the cheaper rule becomes the default for later scaled runs.

What could disprove it

R-NaD's regularisation-policy schedule (Δm) was tuned for millions of learner steps; a short run may never complete a regularisation cycle. The number of completed cycles is recorded beside the result, and a run with fewer than three is labelled inconclusive regardless of its win rate.