Follow-up to #209 (work item #4). #209's #1 shipped in #287.
Problem
BatchVerify captures per-token GDN recurrent state into the snapshot ring with gdnSnapRing=true, which forces the per-position recurrence loop (k×8 launches/layer) plus ~96×(k−1) CopyDeviceRegion launches per step (~3 ms at k=4) — instead of the #114-B fused GdnRecurrenceScan path the prefill trunk uses.
Goal
Extend GdnRecurrenceScan to optionally dump each token's post-update (scan, conv) state into the ring slots as it scans, so the verify trunk keeps the fused path and drops the per-position relaunch + the bulk CopyDeviceRegion fan-out.
Notes
- Ring slot layout is per-layer
(scan, conv) after batch token j; the fused scan already produces these states transiently.
- Must preserve rollback correctness (
RestoreBatchSnapshot reads slot lengthAfter − startPos − 1).
Follow-up to #209 (work item #4). #209's #1 shipped in #287.
Problem
BatchVerifycaptures per-token GDN recurrent state into the snapshot ring withgdnSnapRing=true, which forces the per-position recurrence loop (k×8 launches/layer) plus ~96×(k−1)CopyDeviceRegionlaunches per step (~3 ms at k=4) — instead of the #114-B fusedGdnRecurrenceScanpath the prefill trunk uses.Goal
Extend
GdnRecurrenceScanto optionally dump each token's post-update (scan, conv) state into the ring slots as it scans, so the verify trunk keeps the fused path and drops the per-position relaunch + the bulkCopyDeviceRegionfan-out.Notes
(scan, conv)after batch token j; the fused scan already produces these states transiently.RestoreBatchSnapshotreads slotlengthAfter − startPos − 1).