### probe_N60_b20.8_log10_eps1e-8 (beta2=0.8, last step 3000, log_every 10)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 930 | 250 | 1.81e-03 | 6.14e-03 | 4.2e-07 | 3.48e-07 | 9.78 | -0.04265 | 0.181 |
| 1180 | 210 | 8.44e-04 | 5.63e-03 | 3.6e-07 | 3.76e-07 | 9.61 | -0.05039 | 0.182 |
| 1390 | 280 | 1.78e-02 | 1.72e-02 | 4.2e-07 | 9.75e-07 | 9.78 | -0.03530 | 0.189 |
| 1670 | 300 | 3.14e-02 | 1.40e-02 | 1.8e-06 | 2.28e-06 | 8.72 | -0.02643 | 0.182 |
| 1970 | - | 2.01e-04 | 5.93e-04 | 8.7e-07 | - | - | - | - |

### probe_N60_b20.9_log10_eps1e-8 (beta2=0.9, last step 4000, log_every 10)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 970 | 240 | 6.34e-03 | 1.56e-02 | 1.3e-07 | 1.91e-07 | 11.31 | -0.05235 | 0.158 |
| 1210 | 220 | 1.28e-02 | 8.55e-03 | 2.9e-07 | 2.43e-07 | 10.47 | -0.05266 | 0.136 |
| 1430 | 240 | 2.58e-02 | 1.66e-02 | 6.3e-08 | 3.14e-07 | 10.88 | -0.05186 | 0.175 |
| 1670 | 240 | 1.62e-02 | 5.23e-02 | 7.1e-07 | 4.13e-07 | 11.75 | -0.05267 | 0.127 |
| 1910 | 230 | 3.82e-03 | 1.41e-02 | 1.1e-08 | 4.17e-07 | 10.43 | -0.05212 | 0.176 |
| 2140 | 230 | 2.45e-02 | 2.14e-02 | 8.2e-07 | 2.72e-07 | 11.27 | -0.05242 | 0.169 |
| 2370 | 250 | 2.36e-02 | 9.14e-02 | 4.1e-07 | 3.62e-07 | 12.44 | -0.05267 | 0.144 |
| 2620 | 240 | 2.23e-02 | 3.13e-02 | 6.6e-08 | 3.64e-07 | 11.36 | -0.05266 | 0.143 |
| 2860 | 230 | 1.04e-02 | 1.10e-02 | 1.2e-07 | 2.69e-07 | 10.62 | -0.05219 | 0.173 |
| 3090 | 220 | 2.86e-03 | 1.61e-02 | 4.7e-07 | 3.61e-07 | 10.70 | -0.05239 | 0.167 |
| 3310 | 260 | 4.23e-02 | 1.28e-01 | 5.4e-07 | 6.02e-07 | 12.26 | -0.05186 | 0.170 |
| 3570 | 210 | 1.80e-02 | 1.06e-02 | 1.1e-06 | 3.58e-07 | 10.29 | -0.05264 | 0.148 |
| 3780 | - | 2.32e-02 | 2.10e-02 | 1.5e-07 | - | - | - | - |

### probe_N60_b20.9_eps1e-8 (beta2=0.9, last step 5000, log_every 50)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 900 | 350 | 1.92e-04 | 5.59e-03 | 5.6e-09 | 1.91e-07 | 10.28 | -0.05016 | 0.158 |
| 1250 | 200 | 2.91e-05 | 1.87e-03 | 2.9e-07 | 6.91e-07 | 7.90 | - | 0.073 |
| 1450 | 250 | 2.17e-03 | 1.06e-02 | 8.8e-09 | 2.84e-07 | 10.53 | -0.05266 | 0.123 |
| 1700 | 250 | 6.14e-04 | 1.56e-02 | 2.6e-08 | 4.13e-07 | 10.54 | -0.05268 | 0.127 |
| 1950 | 200 | 5.56e-05 | 3.71e-03 | 1.1e-08 | 1.37e-06 | 7.90 | - | 0.059 |
| 2150 | 250 | 2.67e-03 | 1.41e-02 | 9.5e-09 | 3.78e-07 | 10.53 | -0.05267 | 0.119 |
| 2400 | 250 | 2.81e-03 | 2.30e-02 | 4.0e-08 | 6.11e-07 | 10.53 | -0.05268 | 0.108 |
| 2650 | 250 | 1.29e-03 | 1.35e-02 | 9.5e-09 | 3.64e-07 | 10.52 | -0.05257 | 0.143 |
| 2900 | 200 | 6.11e-05 | 2.52e-03 | 1.2e-07 | 9.32e-07 | 7.90 | - | 0.063 |
| 3100 | 250 | 2.86e-03 | 1.10e-02 | 8.4e-09 | 3.61e-07 | 10.33 | -0.05071 | 0.167 |
| 3350 | 250 | 5.79e-04 | 1.81e-02 | 5.4e-07 | 4.81e-07 | 10.54 | -0.05268 | 0.135 |
| 3600 | 200 | 8.18e-05 | 2.72e-03 | 2.3e-08 | 1.01e-06 | 7.90 | - | 0.080 |
| 3800 | 250 | 1.45e-03 | 1.32e-02 | 2.0e-08 | 3.53e-07 | 10.53 | -0.05264 | 0.136 |
| 4050 | 200 | 2.51e-04 | 5.61e-03 | 4.4e-08 | 2.08e-06 | 7.90 | - | 0.046 |
| 4250 | 250 | 2.07e-03 | 2.25e-02 | 9.0e-09 | 9.22e-07 | 10.10 | -0.05268 | 0.092 |
| 4500 | 200 | 4.11e-04 | 4.83e-03 | 2.0e-08 | 1.79e-06 | 7.90 | - | 0.054 |
| 4700 | 250 | 2.34e-02 | 2.74e-02 | 1.7e-08 | 1.08e-06 | 10.15 | -0.05268 | 0.092 |
| 4950 | - | 2.04e-02 | 5.05e-02 | 1.7e-08 | - | - | - | - |

### probe_N60_b20.95_eps1e-8 (beta2=0.95, last step 6000, log_every 25)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 1075 | 500 | 1.00e-02 | 2.89e-02 | 1.0e-07 | 1.56e-07 | 12.13 | -0.02562 | 0.148 |
| 1575 | 475 | 3.09e-03 | 2.03e-02 | 1.0e-07 | 1.99e-07 | 11.53 | -0.02564 | 0.128 |
| 2050 | 400 | 1.45e-03 | 5.44e-03 | 4.4e-08 | 4.92e-07 | 9.31 | -0.02530 | 0.188 |
| 2450 | 425 | 7.42e-04 | 1.75e-02 | 1.3e-06 | 6.12e-07 | 10.26 | -0.02565 | 0.084 |
| 2875 | 475 | 3.21e-02 | 5.06e-02 | 1.4e-08 | 5.22e-07 | 11.48 | -0.02565 | 0.103 |
| 3350 | 450 | 1.28e-02 | 3.09e-02 | 1.3e-08 | 5.78e-07 | 10.89 | -0.02565 | 0.086 |
| 3800 | 400 | 2.21e-02 | 1.45e-02 | 2.4e-08 | 1.11e-06 | 9.48 | -0.02565 | 0.066 |
| 4200 | 475 | 5.16e-02 | 5.93e-02 | 9.9e-09 | 6.58e-07 | 11.41 | -0.02565 | 0.096 |
| 4675 | 450 | 2.77e-02 | 3.56e-02 | 4.2e-08 | 7.10e-07 | 10.82 | -0.02565 | 0.090 |
| 5125 | 425 | 8.70e-03 | 2.74e-02 | 2.3e-08 | 9.64e-07 | 10.25 | -0.02565 | 0.072 |
| 5550 | - | 1.23e-02 | 3.52e-02 | 1.0e-08 | - | - | - | - |

### probe_N60_b20.95_seed1_eps1e-8 (beta2=0.95, last step 6000, log_every 25)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 1025 | 550 | 1.07e-02 | 3.01e-02 | 7.8e-09 | 1.61e-07 | 12.14 | -0.02563 | 0.144 |
| 1575 | 400 | 1.30e-03 | 1.94e-03 | 7.5e-08 | 2.50e-07 | 8.96 | -0.02562 | 0.116 |
| 1975 | 425 | 2.54e-03 | 1.71e-02 | 8.7e-08 | 6.01e-07 | 10.26 | -0.02565 | 0.083 |
| 2400 | 450 | 7.25e-02 | 3.10e-02 | 2.9e-08 | 7.22e-07 | 10.67 | -0.02565 | 0.080 |
| 2850 | 475 | 3.72e-03 | 2.41e-02 | 1.2e-08 | 4.27e-07 | 10.94 | -0.02558 | 0.145 |
| 3325 | 450 | 7.65e-03 | 4.97e-02 | 4.4e-07 | 9.19e-07 | 10.90 | -0.02565 | 0.077 |
| 3775 | 450 | 2.61e-02 | 3.40e-02 | 1.0e-08 | 9.33e-07 | 10.51 | -0.02565 | 0.087 |
| 4225 | 500 | 3.88e-02 | 1.06e-01 | 2.2e-08 | 5.76e-07 | 12.12 | -0.02564 | 0.128 |
| 4725 | 475 | 1.90e-02 | 8.18e-02 | 1.2e-07 | 8.16e-07 | 11.52 | -0.02565 | 0.090 |
| 5200 | 425 | 2.35e-02 | 1.40e-02 | 9.3e-09 | 7.53e-07 | 9.83 | -0.02565 | 0.095 |
| 5625 | - | 3.17e-02 | 2.15e-01 | 1.1e-07 | - | - | - | - |

### probe_N60_eps1e-8 (beta2=0.98, last step 23650, log_every 50)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 1650 | 1300 | 8.28e-03 | 6.76e-02 | 8.9e-09 | 3.68e-07 | 12.12 | -0.01010 | 0.092 |
| 2950 | 1000 | 7.56e-04 | 1.13e-02 | 6.0e-09 | 7.71e-07 | 9.60 | -0.01010 | 0.055 |
| 3950 | 1100 | 4.86e-02 | 2.39e-02 | 6.8e-09 | 6.73e-07 | 10.48 | -0.01010 | 0.066 |
| 5050 | 1050 | 7.28e-04 | 2.49e-02 | 2.3e-08 | 1.02e-06 | 10.10 | -0.01010 | 0.050 |
| 6100 | 1100 | 9.41e-02 | 2.87e-02 | 7.5e-09 | 9.73e-07 | 10.29 | -0.01010 | 0.056 |
| 7200 | 1200 | 1.28e-01 | 4.96e-02 | 7.8e-09 | 5.63e-07 | 11.38 | -0.01010 | 0.088 |
| 8400 | 1050 | 9.23e-05 | 2.29e-02 | 8.9e-09 | 9.39e-07 | 10.10 | -0.01010 | 0.060 |
| 9450 | 1250 | 2.25e-02 | 1.47e-01 | 7.8e-09 | 8.02e-07 | 12.12 | -0.01010 | 0.089 |
| 10700 | 1100 | 9.94e-05 | 1.80e-02 | 2.0e-08 | 4.45e-07 | 10.61 | -0.01010 | 0.078 |
| 11800 | 1100 | 4.53e-04 | 2.05e-02 | 8.7e-09 | 5.08e-07 | 10.61 | -0.01010 | 0.074 |
| 12900 | 1250 | 2.17e-02 | 3.90e-02 | 6.9e-09 | 2.15e-07 | 12.11 | -0.01010 | 0.134 |
| 14150 | 1150 | 1.40e-04 | 1.18e-02 | 3.6e-08 | 1.81e-07 | 11.09 | -0.01009 | 0.158 |
| 15300 | 1250 | 9.56e-05 | 1.35e-02 | 1.1e-07 | 1.80e-07 | 11.23 | -0.00990 | 0.315 |
| 16550 | 1150 | 2.42e-05 | 7.03e-03 | 1.1e-06 | 1.09e-07 | 11.07 | -0.01009 | 0.136 |
| 17700 | 1150 | 4.41e-05 | 1.10e-02 | 9.0e-08 | 1.65e-07 | 11.11 | -0.01010 | 0.134 |
| 18850 | 1150 | 1.18e-04 | 7.96e-03 | 3.3e-08 | 1.21e-07 | 11.09 | -0.01010 | 0.126 |
| 20000 | 1200 | 3.64e-04 | 9.69e-03 | 2.4e-08 | 9.14e-08 | 11.57 | -0.01009 | 0.125 |
| 21200 | 1150 | 1.63e-04 | 1.41e-02 | 2.8e-08 | 2.12e-07 | 11.11 | -0.01010 | 0.114 |
| 22350 | 1200 | 1.33e-02 | 9.54e-03 | 3.5e-08 | 1.43e-07 | 11.11 | -0.01010 | 0.122 |
| 23550 | - | 2.71e-04 | 1.77e-02 | 1.9e-08 | - | - | - | - |

### probe_N60_eps1e-16 (beta2=0.98, last step 6300, log_every 50)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 1500 | 1000 | 3.39e-06 | 1.02e-03 | 1.2e-08 | 8.09e-08 | 9.44 | -0.01005 | 0.354 |
| 2500 | 1100 | 6.64e-04 | 4.08e-03 | 6.4e-08 | 1.10e-07 | 10.52 | -0.01008 | 0.392 |
| 3600 | 900 | 1.84e-04 | 3.68e-03 | 1.3e-07 | 7.11e-07 | 8.55 | -0.01009 | 0.543 |
| 4500 | 1300 | 1.71e-03 | 1.70e-03 | 8.0e-07 | 9.47e-09 | 12.10 | -0.00988 | 0.431 |
| 5800 | - | 2.84e-04 | 1.87e-03 | 7.1e-09 | - | - | - | - |

### probe_N60_b20.99_eps1e-8 (beta2=0.99, last step 14000, log_every 50)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 2700 | 2100 | 7.51e-04 | 4.94e-03 | 3.7e-09 | 1.66e-07 | 10.30 | -0.00503 | 0.090 |
| 4800 | 2050 | 2.55e-05 | 9.28e-03 | 5.6e-08 | 4.01e-07 | 10.05 | -0.00503 | 0.065 |
| 6850 | 2300 | 2.40e-03 | 3.30e-02 | 5.5e-09 | 4.05e-07 | 11.31 | -0.00503 | 0.066 |
| 9150 | 2500 | 1.51e-02 | 1.61e-01 | 5.5e-09 | 7.24e-07 | 12.31 | -0.00503 | 0.084 |
| 11650 | 2200 | 3.43e-03 | 2.55e-02 | 7.3e-09 | 5.18e-07 | 10.80 | -0.00503 | 0.080 |
| 13850 | - | 7.63e-04 | 2.75e-02 | 1.5e-08 | - | - | - | - |

### probe_N60_b20.995_eps1e-8 (beta2=0.995, last step 22000, log_every 50)
| spike step | period to next | peak rms_grad | peak rms_sqrt_vhat | rms_grad just before | floor rms_sqrt_vhat before next | ln(peak/floor) | slope ln sqrt(vhat)/step | step/lr before next |
|---|---|---|---|---|---|---|---|---|
| 4700 | 4200 | 8.06e-05 | 8.64e-03 | 1.1e-08 | 2.63e-07 | 10.40 | -0.00251 | 0.060 |
| 8900 | 4650 | 1.85e-02 | 6.62e-02 | 4.7e-09 | 6.53e-07 | 11.53 | -0.00251 | 0.071 |
| 13550 | 4550 | 9.91e-03 | 5.20e-02 | 6.2e-09 | 6.59e-07 | 11.28 | -0.00251 | 0.085 |
| 18100 | - | 6.80e-02 | 3.48e-02 | 7.2e-09 | - | - | - | - |

| beta2 | run | steps | spikes | mean period (sd) [median] | predicted 2ln(peak/floor)/(1-b2) | ln(peak/floor) (sd) | median peak / floor sqrt(vhat) | median rms_grad spike / floor | decay slope meas. (sd) vs ln(b2)/2 | mean (median) step/lr | step/lr before spike | test acc end (max) | first step test acc 0.5/0.9/0.99 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.8 | probe_N60_b20.8_log10_eps1e-8 | 3000 | 5 | 260 (34) [265] | 95 | 9.47 (0.44) | 6.1e-03 / 6.8e-07 | 1.8e-03 / 4.2e-07 | -0.0387 (0.0089) vs -0.1116 | 0.218 (0.2010) | 0.182 | 0.160 (0.160) | None/None/None |
| 0.9 | probe_N60_b20.9_log10_eps1e-8 | 4000 | 13 | 234 (13) [235] | 223 | 11.15 (0.68) | 1.7e-02 / 3.6e-07 | 1.8e-02 / 2.9e-07 | -0.0524 (0.0003) vs -0.0527 | 0.073 (0.0451) | 0.163 | 0.120 (0.130) | None/None/None |
| 0.9 | probe_N60_b20.9_eps1e-8 | 5000 | 18 | 238 (36) [250] | 191 | 9.53 (1.21) | 1.2e-02 / 6.1e-07 | 9.5e-04 / 1.8e-08 | -0.0523 (0.0009) vs -0.0527 | 0.072 (0.0457) | 0.108 | 0.129 (0.134) | None/None/None |
| 0.95 | probe_N60_b20.95_eps1e-8 | 6000 | 11 | 448 (32) [450] | 430 | 10.76 (0.88) | 2.9e-02 / 5.9e-07 | 1.2e-02 / 2.4e-08 | -0.0256 (0.0001) vs -0.0256 | 0.045 (0.0114) | 0.093 | 0.115 (0.126) | None/None/None |
| 0.95 | probe_N60_b20.95_seed1_eps1e-8 | 6000 | 11 | 460 (41) [450] | 431 | 10.78 (0.94) | 3.1e-02 / 6.6e-07 | 1.9e-02 / 2.9e-08 | -0.0256 (0.0000) vs -0.0256 | 0.051 (0.0147) | 0.092 | 0.171 (0.171) | None/None/None |
| 0.98 | probe_N60_eps1e-8 | 23650 | 20 | 1153 (77) [1150] | 1099 | 10.99 (0.69) | 1.8e-02 / 3.7e-07 | 4.1e-04 / 1.9e-08 | -0.0101 (0.0000) vs -0.0101 | 0.034 (0.0081) | 0.092 | 0.211 (0.211) | None/None/None |
| 0.98 | probe_N60_eps1e-16 | 6300 | 5 | 1075 (148) [1050] | 1015 | 10.15 (1.32) | 1.9e-03 / 9.5e-08 | 2.8e-04 / 6.4e-08 | -0.0100 (0.0001) vs -0.0101 | 0.121 (0.0335) | 0.411 | 0.223 (0.223) | None/None/None |
| 0.99 | probe_N60_b20.99_eps1e-8 | 14000 | 6 | 2230 (160) [2200] | 2191 | 10.95 (0.80) | 2.6e-02 / 4.1e-07 | 1.6e-03 / 6.4e-09 | -0.0050 (0.0000) vs -0.0050 | 0.027 (0.0062) | 0.080 | 0.177 (0.177) | None/None/None |
| 0.995 | probe_N60_b20.995_eps1e-8 | 22000 | 4 | 4467 (193) [4550] | 4427 | 11.07 (0.48) | 4.3e-02 / 6.5e-07 | 1.4e-02 / 6.7e-09 | -0.0025 (0.0000) vs -0.0025 | 0.026 (0.0072) | 0.071 | 0.202 (0.202) | None/None/None |