S1-30: SSM_CONV_BACK CPU kernel#24
Open
dillon-blake wants to merge 1 commit into
Open
Conversation
d_sx[i2 + i0, i1, i3] += dy[i1, i2, i3] * c[i0, i1]
The forward is a depthwise causal convolution over a sliding window, so its gradient
scatters each output's gradient back across the d_conv inputs that produced it. Written as
a SCATTER rather than the equivalent gather (d_sx[j] = sum over t of dy[t]*c[j-t], with its
two-sided bounds on t) because the scatter needs no boundary arithmetic at all: every
(i2, i0) pair lands in range by construction.
The leading d_conv - 1 columns of sx are the carried convolution state, and they receive a
gradient like any other input. Dropping them would silently truncate the gradient at every
sequence boundary.
Threaded by (row, sequence): each (i1, i3) owns its own d_sx column, so no atomics, no
barrier, and the accumulation order within a column is fixed by the loops rather than by
thread arrival.
UNDER sum(out) THIS TEST WAS VACUOUS, AND I ONLY FOUND OUT BY MUTATING THE KERNEL.
sum(out) makes the incoming gradient all-ones, and the scatter is dy * c -- so with dy == 1
a kernel that IGNORES dy entirely and scatters c alone produces exactly the same answer.
Measured:
ignore the incoming gradient + sum(out) NOT CAUGHT AT ALL
ignore the incoming gradient + grad_loss MAA 0.84
Third instance of the same structural trap (SOFT_MAX in S1-34, MUL_MAT_ID in S1-27). The
weighted grad_loss hook is now the default assumption for any new VJP, not an afterthought.
max_maa_err 1e-2, measured: noise floor 2.2e-3 on the widest shapes, three injected
mutations at 0.47-0.84, all three caught.
Also checked exactly -- against a naive DOUBLE reference and with a TRANSPOSED grad
(nb[0] != 4), the stride trap that bit both MoE kernels and GLU_BACK. Exact to ~1e-7 in
both layouts; this kernel reads dy through src0->nb[0] from the start.
54 grad cases now EXECUTE. They previously reported OK while requesting no gradients at all.
This was referenced Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #23 (S1-29b).
The forward is a depthwise causal convolution over a sliding window, so its gradient scatters each output's gradient back across the
d_convinputs that produced it. Written as a scatter rather than the equivalent gather (d_sx[j] = sum_t dy[t]*c[j-t], with its two-sided bounds ont) because the scatter needs no boundary arithmetic at all — every(i2, i0)pair lands in range by construction.The leading
d_conv - 1columns ofsxare the carried convolution state and receive a gradient like any other input. Dropping them would silently truncate the gradient at every sequence boundary.Threaded by
(row, sequence): each(i1, i3)owns its ownd_sxcolumn — no atomics, no barrier, fixed accumulation order.Under
sum(out)this test was vacuous, and I only found out by mutating the kernelsum(out)makes the incoming gradient all-ones, and the scatter isdy * c— so withdy == 1, a kernel that ignoresdyentirely and scatterscalone produces exactly the same answer.sum(out)grad_lossThird instance of the same structural trap (SOFT_MAX in S1-34, MUL_MAT_ID in S1-27). The weighted
grad_losshook is now the default assumption for any new VJP, not an afterthought.Verification
max_maa_err = 1e-2, measured with numbers on both sides:Three mutations injected, three caught.
Also checked exactly — against a naive double-precision reference and with a transposed grad (
nb[0] != 4), the stride trap that bit both MoE kernels and GLU_BACK. Exact to ~1e-7 in both layouts; this kernel readsdythroughsrc0->nb[0]from the start.54 grad cases now execute. They previously reported
OKwhile requesting no gradients at all.