Training runs
Did the rollouts that earned reward actually do the task?
No training replays yet
Runs appear here once watch-training has replayed sampled rollouts into this suite's evidence folder.
touchstone watch-training SUITE --inbox INBOX --policy POLICY.json
TRAINING_WATCH.md covers the emission hook and the replay policy.
Reward compared with verified success
Share of replayed rollouts, by training step
When the rewarded line climbs above the verified line, rollouts are earning reward without doing the task. That gap is the reward hack.
Replay verdicts by step
Every sampled rollout is replayed twice beside known-good and known-bad controls
Why replays could not verify
Failed controls usually mean the grader or the replay setup is broken, not the policy
| Reason | Rollouts | Share |
|---|
Sampled rollouts
| Step | Verdict | Reward | Task done | Regression test | Replayed |
|---|
Rollout input
Replays
| Replay | Reward | Task done | Expected |
|---|