== select ==
model                latents   val_bps   cycle   leak  stateR2  rateR2
----------------------------------------------------------------------
kl_co_scale=0            128    0.1283   0.467   0.90    0.909   0.895 x misattribution
kl_co_scale=1.00E-02     128    0.1181   0.294   0.36    0.861   0.865
kl_co_scale=1.00E-03     128    0.1208   0.412   0.21    0.897   0.881
kl_co_scale=1.00E-04     128    0.1250   0.390   0.27    0.893   0.892 <- recommended
kl_co_scale=1.00E-05     128    0.1287   0.465   0.75    0.900   0.896 x misattribution
kl_co_scale=1.00E-06     128    0.1287   0.463   0.89    0.908   0.896 x misattribution

WARNING MISATTRIBUTION_EXCLUDED
  3 model(s) were not considered for recommendation because their inferred inputs are
  predictable from the preceding latent state (lag-1 leakage > 0.5):
  kl_co_scale=0_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0,
  kl_co_scale=1.00E-05_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0,
  kl_co_scale=1.00E-06_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0. Offloading
  dynamics to inputs improves reconstruction, so these models would otherwise win on it.
Recommended: kl_co_scale=1.00E-04_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0.

== validation against the task-trained ground truth ==
6 bundles with ground truth

detector                    tier   TP   FP   FN   TN  precision  recall
DYNAMICAL_MISATTRIBUTION       0    3    0    3    0       1.00    0.50
DYNAMICAL_MISATTRIBUTION       1    3    0    3    0       1.00    0.50
UNDERFITTING                   0    0    0    6    0        n/a    0.00

rank agreement with ground truth (Spearman)
metric                  truth          n     rho  expected
recoverable_fraction    rate_r2        6   -1.00  negative
cycle_consistency       state_r2       6    0.94  positive
input_leakage_r2        input_r2       6   -0.60  negative
rollout_gap             input_r2       6   -0.94  negative

disagreements
  kl_co_scale=0_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0 UNDERFITTING tier 0: missed (truth: present)
  kl_co_scale=1.00E-02_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0 UNDERFITTING tier 0: missed (truth: present); DYNAMICAL_MISATTRIBUTION tier 0: missed (truth: present)
  kl_co_scale=1.00E-03_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0 UNDERFITTING tier 0: missed (truth: present); DYNAMICAL_MISATTRIBUTION tier 0: missed (truth: present); DYNAMICAL_MISATTRIBUTION tier 1: missed (truth: present)
  kl_co_scale=1.00E-04_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0 UNDERFITTING tier 0: missed (truth: present); DYNAMICAL_MISATTRIBUTION tier 0: missed (truth: present); DYNAMICAL_MISATTRIBUTION tier 1: missed (truth: present)
  kl_co_scale=1.00E-05_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0 UNDERFITTING tier 0: missed (truth: present); DYNAMICAL_MISATTRIBUTION tier 1: missed (truth: present)
  kl_co_scale=1.00E-06_prefix=20241004_NBFF_NoisyGRU_Final_df=5_seed=0 UNDERFITTING tier 0: missed (truth: present)
