[Math] New QIPC ops for single-threaded linalg#683
Merged
Conversation
Adds a free function quadrants.lang.matrix_ops.frobenius_inner(A, B) and a matching Matrix.frobenius_inner(other) method, computing ⟨A, B⟩ = Σ_ij A_ij B_ij. Mirrors the existing norm_sqr function — they are the same operation when A == B. Tests parametrise over arch=qd.gpu (CUDA / Metal / Vulkan / AMDGPU) for both f32 and f64, on square sizes 2, 3, 6, 9, 12 (matching qipc's IPC needs) plus rectangular shapes 9×12, 12×3, 2×4 to cover the non-square use cases (Hessian blocks in qipc). Closes the "Frobenius inner product" gap row in perso_hugh/doc/qipc/qipc_gaps_linalg.md.
…12 · 12×9)
Adds test_matmul_chain_qipc_sizes_{f32,f64} verifying that the largest matmul
chain qipc's IPC pipeline needs (9×12 · 12×12 · 12×9 → 9×9) compiles cleanly
and matches numpy. Both the chained form (A @ B @ C) and the staged form
(AB = A @ B; AB @ C) are checked, since the chained form may stress the
backend codegen differently (intermediate has 1296 FMAs unrolled).
Parametrised over arch=qd.gpu so CUDA / Metal / Vulkan / AMDGPU all run.
Quadrants imposes no enforced size cap on matmul; the "n·m ≤ 32" warning
in qipc's design doc is qipc-side only.
Closes the "Matrix.__matmul__ correctness at large sizes" gap row in
perso_hugh/doc/qipc/qipc_gaps_linalg.md (gap (verify) → ✅).
qipc's ARAP rotation R = U @ V.T must be a proper rotation (det R = +1) for
any input deformation gradient F. The libuipc convention enforced by qipc is
det(U) = det(V) = +1 always, with the sign of det(F) absorbed into σ
(σ may have a negative entry when det(F) < 0).
This test verifies that quadrants' qd.svd at 3×3 follows the same convention,
across the cases qipc actually exercises:
- identity (det = +1), reflection (det = -1), generic positive- and
negative-det matrices, SPD, near rank-deficient, near-degenerate
singular values.
Parametrised over arch=qd.gpu × {f32, f64}.
…pivoting
Adds _inverse_lu in matrix_ops.py: in-place Gauss elimination with partial
pivoting, fully unrolled (static range for all loop bounds, runtime int for
pivot-row index). The inverse function dispatches to it for N >= 5; sizes
1–4 keep the existing closed-form cofactor-expansion paths. Precondition
relaxed from dim_lt(0, 5) to dim_lt(0, 13).
The implementation maintains a working copy `a` for in-place LU and a
parallel matrix `b` initialised to identity that receives the same row
swaps + row eliminations; at the end `b = L⁻¹ P` and the inverse is read
column-by-column by back-solving `U x = b[:, c]`.
Tests at tests/python/test_linalg.py::test_inverse_large_{f32,f64} cover
N ∈ {5..12} × {diagonally-dominant, SPD, permuted-upper-triangular}. The
permuted-upper-triangular factory has a zero in [0, 0] so it specifically
exercises the pivoting path. Tolerance scales with condition number ×
machine epsilon (50 × cond × eps + dtype floor).
Parametrised over arch=qd.gpu so CUDA / Metal / Vulkan / AMDGPU all run.
Adds python/quadrants/_funcs_sym_eig_general.py with sym_eig_general(A, dt)
and make_spd(A, dt). Implements Eigen 3.4's SelfAdjointEigenSolver compute()
path: Householder tridiagonalisation + implicit QR with Wilkinson shift +
ascending sort. Direct port of qipc/_src/core/linalg/evd.py — qipc can drop
its private copy once this lands.
qd.sym_eig now dispatches: N=2/3 keep the existing closed-form
_sym_eig{2,3}x3 paths; 4 ≤ N ≤ 12 → sym_eig_general. Also exposes
qd.make_spd(A) which projects a symmetric matrix to the nearest PSD matrix
in Frobenius norm by clamping eigenvalues to ≥ 0 — qipc's per-element
Hessian projection.
Tests at tests/python/test_eig.py:
- test_sym_eig_general_{f32,f64}: N ∈ {4, 5, 6, 9, 12} × {random symmetric,
SPD, indefinite, diagonal, repeated-eigenvalues}. Verifies eigenvalues
match numpy, eigenvectors are orthogonal, and Q diag(λ) Qᵀ ≈ A.
- test_make_spd_{f32,f64}: N ∈ {4, 6, 9, 12} × {indefinite, random, SPD}.
Verifies symmetry, PSD-ness (min eig ≥ -tol), and that the result
matches numpy-reference clamping.
Parametrised over arch=qd.gpu so CUDA / Metal / Vulkan / AMDGPU all run.
The previous Householder + implicit-QR port produced wrong eigenvalues for N>3 (off-diagonal residue ~50% of the input scale), and the algorithm's many static branches did not lend themselves to debugging via printf. Switch sym_eig_general (and the make_spd that builds on it) to cyclic Jacobi. The Jacobi loop is fully unrolled with static(range): runtime range loops in @func that return values were observed to iterate only once on this branch, so static unrolling of MAX_SWEEPS=6 sweeps is what actually reduces the off-diagonals across passes. The 6-sweep budget gives ~6 digits in f32 and ~12 digits in f64 for N≤9 on the test factories. N=12 (used by qipc's 12×12 contact Hessians) is dropped from this path: the fully static-unrolled Jacobi at N=12 with 6 sweeps does not finish compiling within 15 minutes on CUDA. Either a blocked/partially-runtime implementation, or porting qipc's exact `sym_evd` template-mutation pattern (ndarray of compound types passed via template()), is needed to recover N=12 — tracked as a follow-up. - _MAX_SWEEPS = 6 (sweep / (p,q) / per-row updates all static). - 2-pass right/left rotation (no `r != p, q` static guards) keeps the unrolled body lean enough to compile for N≤9. - sym_eig() now raises for N≥10 with a follow-up note. - test_eig.py: parametrize sym_eig_general / make_spd at N ∈ {4, 5, 6, 9} (drop 12).
Hugh's catch: the previous sweep loop wasn't broken — it was being
parallelized. quadrants @qd.kernel reaches into a callee @qd.func and
parallelizes the func's outermost runtime range loop when the kernel
itself doesn't have one. The sweep loop saw _MAX_SWEEPS threads each
running one sweep on a fresh copy of the locals, with last-write-wins.
That's why static(range) was the only thing that "worked", and that's
what blew up the compile time at N=12.
Fix is a one-liner in the test kernel (for _tid in range(1):) plus
flipping the sweep loop back to runtime range:
for _sweep in range(_MAX_SWEEPS): # was static(range(_MAX_SWEEPS))
With that, N=12 compiles in ~2 minutes (vs not finishing in 15 min
before) and the cap in qd.sym_eig() goes back up to 12, restoring qipc's
target sizes.
- _MAX_SWEEPS = 12 (was 6) — runtime, so cost is per-call, not compile.
- qd.sym_eig() supports N ∈ {2..12}; N≥13 raises.
- test_eig.py: re-add N=12 to the parametrize lists; wrap kernel calls
in `for _tid in range(1):`.
- Docstring on _funcs_sym_eig_general points at the gotcha note in
perso_hugh that explains the parallel-for behavior.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 40af9db7ab
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| eigvecs[i, j] = zero | ||
| eigvecs[i, i] = one | ||
|
|
||
| for _sweep in range(_MAX_SWEEPS): |
There was a problem hiding this comment.
For callers that use qd.sym_eig in the normal scalar-kernel style, e.g. eigvals[None], eigvecs[None] = qd.sym_eig(A[None]) without a dummy outer for, this runtime range is parallelized by the kernel machinery instead of executing the 12 sweeps sequentially; the new implementation's own docstring calls out that this produces incorrect iteration semantics. That means the newly supported n >= 4 path can return a non-converged eigensystem unless every caller restructures its kernel, unlike the existing 2x2/3x3 API. Since _MAX_SWEEPS is a fixed constant, this should be hidden inside the implementation rather than imposed as a caller requirement.
Useful? React with 👍 / 👎.
|
|
||
|
|
||
| __all__ = ["randn", "polar_decompose", "eig", "sym_eig", "svd", "solve"] | ||
| __all__ = ["randn", "polar_decompose", "eig", "sym_eig", "make_spd", "svd", "solve"] |
There was a problem hiding this comment.
Adding make_spd to _funcs.__all__ exposes qd.make_spd through the top-level wildcard import, and this same commit also exposes Matrix.frobenius_inner; however tests/python/test_api.py::test_api compares dir(qd) and dir(qd.Matrix) against hard-coded lists that contain neither name. The existing CPU API snapshot test will therefore fail even though the new APIs are intentional, so please update the expected API lists with these additions.
Useful? React with 👍 / 👎.
Comment on lines
+622
to
+623
| if A.n <= 12: | ||
| return sym_eig_general(A, dt) |
There was a problem hiding this comment.
This expands the public qd.sym_eig contract from 2x2/3x3 to sizes up to 12, but no docs/ files were changed; AGENTS.md specifically requires docs updates for public API or usage changes. The current user guide still advertises qd.sym_eig as 2x2/3x3-only, and related changes in this commit also leave the documented inverse() size cap and new public helpers stale, so users will get contradictory guidance.
Useful? React with 👍 / 👎.
Tests (test_eig.py): four new contract / edge-case tests for the
N≥4 cyclic-Jacobi path, all parametrized over qd.gpu × N ∈ {4,6,9,12}
and wrapped in the required `for _tid in range(1):` outer loop:
- test_sym_eig_alpha_identity_f64 — α·I (incl. α=0) at every N to
cover the fully-degenerate / repeated-eigenvalue case.
- test_make_spd_idempotent_f64 — make_spd(make_spd(A)) ≈ make_spd(A)
over indefinite / negative-definite / SPD inputs.
- test_make_spd_negative_definite_zero_f64 — all-negative-eig
inputs project to the zero matrix.
- test_sym_eig_above_cap_raises — N=13 raises with a clear
"up to 12" message instead of silently miscompiling.
30/30 pass on cluster (CUDA + Vulkan + CPU, ~32 min) and amddesktop
(AMDGPU + Vulkan + CPU, ~10 min).
Docs (user_guide/decompositions.md): updated the table to include
N up to 12 for qd.sym_eig and add qd.make_spd; documented the
cyclic-Jacobi path, the for-_tid kernel pattern requirement, and
compile/runtime cost characteristics. Replaced the inlined make_spd
3×3 snippet with a real qd.make_spd example for N≥4 (kept the 3×3
recipe for users below the cap). Added a Frobenius inner-product
section to user_guide/matrix_vector.md and updated the Matrix.inverse
size cap from 4×4 to 12×12.
- decompositions.md → linalg_per_thread.md (rename + retitle "Per-thread linear algebra"; updated opening paragraph to make the per-thread, non-cooperative semantics explicit; updated "Related" cross-links). - matrix_vector.md → split: type / storage / declaration content stays here; element-wise + closed-form ops (arithmetic, dot/cross, norm, transpose/det/trace/inverse, frobenius_inner, mat-vec/mat-mat multiply) move to matrix_vector_per_thread.md. - index.md toctree: replaces `decompositions` with `matrix_vector_per_thread` + `linalg_per_thread` under Core concepts. The "_per_thread" suffix marks pages whose ops run per thread in registers with no cross-thread cooperation (no shared memory, no syncs, no warp/subgroup primitives). Cross-thread / sparse linalg under qd.linalg.* is a separate axis and is not covered yet. No source / test changes — pure docs reorg.
The dispatch shape-cap exceptions in qd.polar_decompose / qd.svd / qd.eig / qd.solve and the dim assertion in qd.solve all said "2D matrix" / "3D matrix" when they meant "2×2 / 3×3 matrix". "2D matrix" conventionally means "rank-2 tensor", which is true of every matrix — so the message was actively misleading. Updated to: - "Polar decomposition only supports 2×2 and 3×3 matrices." - "SVD only supports 2×2 and 3×3 matrices." - "Eigen solver only supports 2×2 matrices." - "Solver only supports 2×2 and 3×3 matrices." - assert "Only 2×2 and 3×3 matrices are supported" Also updated the one test that pinned on the old wording (test_ast_refactor.py::test_raise) and dropped the FIXME from the linalg_per_thread.md doc. Verified locally on amddesktop: test_raise passes 3/3 (cpu + amdgpu + vulkan).
hughperkins
commented
May 10, 2026
hughperkins
commented
May 10, 2026
Drops the caller-pattern requirement that qd.sym_eig / qd.make_spd be
called from inside a top-level `for _tid in range(N):` in the
@qd.kernel. The sweep loop in _funcs_sym_eig_general.sym_eig_general
now opens with `qd.loop_config(serialize=True)`, which pins its
parallelism to 1 — so even when the kernel parallelizer reaches into
the callee @qd.func (the underlying gotcha) it serializes the sweep
loop and iterates the requested MAX_SWEEPS times on a single thread.
Tests (test_eig.py): removed all `for _tid in range(1):` wrappers from
the kernels in test_sym_eig_general / test_make_spd /
test_sym_eig_alpha_identity / test_make_spd_idempotent /
test_make_spd_negative_definite_zero / test_sym_eig_above_cap_raises.
Each test now calls `qd.sym_eig(...)` or `qd.make_spd(...)` as a plain
single-element kernel body.
Verified:
- amd (CPU + AMDGPU + Vulkan): full test_eig.py 211/211 pass in 18:22.
- MRE without any wrapper at N ∈ {4,6,9,12} on amdgpu / vulkan / cpu:
all pass with eig err ≤ 1.7e-14, orth err ≤ 5e-15 (f64).
- cluster (CPU + CUDA + Vulkan): 126/126 of the new sym_eig_general
+ make_spd test parametrizations pass before Slurm step time-out
(no failures or crashes from this code path).
Docs (linalg_per_thread.md): removed the "Caller pattern" subsection
and the top-level "needs a top-level `for` in the calling kernel"
bullet — the constraint no longer exists.
CI test_api[arch=arm64-quadrants] and test_api[arch=arm64-Matrix] were
failing because the new public symbols added on hp/new-qipc-ops-linalg
(`qd.make_spd` module-level entry point and the
`Matrix.frobenius_inner` method) weren't listed in the expected API
manifests in tests/python/test_api.py.
Added:
- "make_spd" in user_api[qd] (alphabetical, between "loop_config" and
"math").
- "frobenius_inner" in _get_expected_matrix_apis() (between "fill"
and "identity").
Verified locally: test_api[arch=x64-quadrants] and
test_api[arch=x64-Matrix] both pass on amd. The other test_api
parametrizations failing locally are pre-existing API drift between
quadrants==0.7.6 wheel and main; unrelated to this PR.
Reflowed AI-default ~75-80c wraps to the project's 120c budget on the
new docstrings touched by this PR:
- sym_eig (`_funcs.py`): closed-form/cyclic-Jacobi dispatch note,
plus dropped a stale `.. note::` block describing the old
"needs `for _tid in range(...)` wrap" caller-pattern requirement
(no longer true after loop_config(serialize=True)).
- make_spd (`_funcs.py` and `_funcs_sym_eig_general.py`): both
docstrings.
- sym_eig_general (`_funcs_sym_eig_general.py`): module docstring
+ Returns block.
- Matrix.frobenius_inner docstring (`lang/matrix.py`).
- _inverse_lu docstring (`lang/matrix_ops.py`).
After: all touched runs sit in the 104-115c range, well within 120c
and well clear of the ~80c AI-default that the CI line-wrapping
checker flags.
Black reformat from `pre-commit run -a`:
- test_linalg.py: split a multi-arg `np.testing.assert_allclose(...)`
onto separate lines.
- test_svd.py: collapsed a few oversplit f-strings / `print(...)`
calls back onto single lines (now under 120c).
No behavioural change.
CI was timing out on test_make_spd_idempotent_f64[arch=cuda-*-12]
(~10 min/test budget). The test was defining two @qd.kernel closures
each with its own qd.make_spd(...) call — each closure JIT-compiles
the cyclic-Jacobi + reconstruct path independently, so N=12 on CUDA
hit 2× single-kernel compile time and exceeded the timeout.
Refactor to a single parametric kernel taking ndarray args:
@qd.kernel
def project(src: NDArray[mat_t, 1], dst: NDArray[mat_t, 1]):
dst[0] = qd.make_spd(src[0], dt)
project(A, A_spd_1)
project(A_spd_1, A_spd_2)
Now qd.make_spd is JIT-compiled exactly once per test and called
twice with different ndarray bindings; the second call is a pure
launch with no recompile.
Verified locally: 12/12 (4 sizes × 3 factories) pass on amd
(amdgpu+vulkan+cpu) in 2:18.
…-step numerical algorithm' (clearer scope)
…nt already covered in perf section
Reflows comment blocks and docstrings that were wrapped at the AI default ~75-80c instead of the project's 120c target, flagged by the line-wrapping CI check. Touches comments only — no behavior change.
Tightens 2-line docstrings whose first line was wrapped at ~76-83c instead of using the 120c budget more evenly. Found by the line-wrapping CI on the previous push.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
MuGdxy
reviewed
May 12, 2026
MuGdxy
left a comment
Contributor
There was a problem hiding this comment.
n * m > 32 warning contradicts the new 12×12 support
python/quadrants/lang/matrix.py L305–316 emits a UserWarning when constructing any matrix with more than 32 entries:
if self.n * self.m > 32:
warning(
f"Quadrants matrices/vectors with {self.n}x{self.m} > 32 entries are not suggested."
" Matrices/vectors will be automatically unrolled at compile-time for performance."
" So the compilation time could be extremely long if the matrix size is too big."
...
)The new APIs in this PR (qd.sym_eig N≤12, qd.make_spd, Matrix.inverse() N≤12) internally construct matrices via Matrix.zero(dt, N, N) → _filled_matrix → Matrix(...), which hits this check. For N≥6 (6×6 = 36 > 32), users will see a "not suggested" warning when calling officially supported APIs — contradicting the PR's intent.
Suggestions:
- Suppress the warning on internal call paths (e.g. via
warnings.filterwarningsor an internal_no_size_warningflag); or - Raise the threshold from 32 to match the new caps (e.g. 144 = 12×12); or
- At minimum, document that users can safely ignore this warning for the new APIs.
Not a blocker, but it hurts UX — especially qd.make_spd which constructs multiple large matrices internally and may emit the warning several times per call.
Addresses MuGdxy's review on PR #683: the n*m > 32 UserWarning fires during JIT trace of officially-supported APIs that internally construct ≥6×6 register-resident matrices (`qd.sym_eig` N≤12, `qd.make_spd`, `Matrix.inverse` N≤12), contradicting the PR's intent to support those sizes. 144 = 12×12 is the new natural cap: it matches the largest size every officially-supported per-thread linalg op accepts, so internal constructions stay below the warning threshold while user code that builds a register-resident matrix beyond that still gets the "consider qd.field instead" advice. Doc updated in `matrix_vector.md` (intro line + "Size limit" section).
Collaborator
Author
Addressed by Opus: Threshold raised from 32 → 144 in lang/matrix.py; warning text updated to match; |
Collaborator
Author
|
😅 |
Collaborator
I'm thinking about the register spill problem and the compile time problem, if everything is inline, like mat12 @ mat12, what will happen? Will it ruin the compile speed? Will the runtime performance get worse? |
Collaborator
Author
Very possible. But that's true in your original version too I assume? Or, how does your UIPC version work? Generally, up to about ~200 registers ish I would expect to be ~ok-ish. 1,200 seems likely eithre to spill, or have terrible occupancy, or both? |
Collaborator
|
I guess, the huge amount of elements in the mat12x12 are finally placed in the thread local memory(performance equal to global memory), and I think it still works well, at least not the bottleneck. I'm not sure how qd will process the mat12x12, will it agressively place them to register? |
Collaborator
Author
Quadrants doesnt decide what goes in registers or not. The GPU driver decides (or maybe the GPU itself; I'm not sure. But either way it's not under our control, or quadrants control, AFAIK.) |
hughperkins
added a commit
that referenced
this pull request
May 15, 2026
…de files PR #683 (just merged into main) deleted ``decompositions.md`` and split it into ``linalg_per_thread.md`` + ``matrix_vector_per_thread.md``. The spiritual successor (``linalg_per_thread.md``) reintroduced the "raises an exception at trace time" wording that I'd previously converted to "compile time" in ``decompositions.md``. ``atomics.md`` similarly carried three "trace time" occurrences left over from earlier PRs. Convert all four to "compile time" for consistency with the rest of the user guide (everything is resolved during AST -> IR compilation, not at trace time).
npoulad1
added a commit
to ROCm/quadrants
that referenced
this pull request
Jun 8, 2026
* [Misc] Warn user to disable caching when print_ir/QD_DUMP_IR enabled (Genesis-Embodied-AI#425) Co-authored-by: v01dxyz <v01dxyz@v01d.xyz> * [Build] Pin torch version to CUDA 12.8 for CUDA tests (Genesis-Embodied-AI#428) * [Misc] Fixing up taichi-dev urls (Genesis-Embodied-AI#429) * [Perf] Rename cuda_graph to gpu_graph across the codebase (Genesis-Embodied-AI#430) * Misc: fix typo integeral -> integral (Genesis-Embodied-AI#434) Co-authored-by: v01dxyz <v01dxyz@v01d.xyz> * [Perf] CUDA graph 4: call from multiple locations (Genesis-Embodied-AI#420) * [Bug] Fix fastcache not restoring graph_do_while_arg (Genesis-Embodied-AI#435) * [Perf] Cache last-call result in perf_dispatch for single-compatible case (Genesis-Embodied-AI#438) * Fix gpu_graph fallback on old Nvidia GPU. (Genesis-Embodied-AI#443) * Fix shared memory offset not reset between CUDA kernels. (Genesis-Embodied-AI#442) * [Misc] Allow disabling GPU graph via QD_GPU_GRAPH=0 env var (Genesis-Embodied-AI#439) * [Misc] Add named top-level loops (Genesis-Embodied-AI#440) * [Misc] Rename gpu_graph to graph (Genesis-Embodied-AI#446) * [Misc] Add cross-platform shuffle (Genesis-Embodied-AI#447) * [Bug] Fix graph_do_while on Windows: search for cudadevrt.lib (Genesis-Embodied-AI#456) * [Bug] Also search default CUDA toolkit install location on Windows (Genesis-Embodied-AI#461) * [SPIRV] Feature Parity Atomics & Shared Array (Genesis-Embodied-AI#432) * [Misc] Change clang format to 120 characters (Genesis-Embodied-AI#463) * [Misc] CUDA graph 5 Add fatbin (Genesis-Embodied-AI#464) * [Bug] Reuse VkInstance across init/reset cycles (Genesis-Embodied-AI#465) * [Perf] Tiles 1: _load, _store, _eye_ (Genesis-Embodied-AI#466) * [Misc] Remove dead InternalFuncStmt type_check override (Genesis-Embodied-AI#471) * [Perf] Tiles 2: add cholesky and ger (Genesis-Embodied-AI#472) * [Perf] Tiles 2b: add triangular solve (Genesis-Embodied-AI#474) * [Misc] Refactor: use _get_col/_set_col in tiles load/store/init (Genesis-Embodied-AI#475) * [Build] Fix flaky test_clock_accuracy (Genesis-Embodied-AI#436) * Fix AARCH64 emitting invalid asm in CUDA kernels. (Genesis-Embodied-AI#473) Co-authored-by: Hugh Perkins <hughperkins@gmail.com> * [AMDGPU] Enable HIP memory pool and surface pool-exhaustion errors. (Genesis-Embodied-AI#485) * [AMDGPU] Scope hsaco tmp dir per-user to avoid collisions. (Genesis-Embodied-AI#484) * [Perf] Tiles 3: Add slice syntax, qd.outer() and initial doc (Genesis-Embodied-AI#477) * [AMDGPU] Fix gradient computation. (Genesis-Embodied-AI#486) * Enable all backends that are supported in unit tests. (Genesis-Embodied-AI#488) * Fix SPIRV ID overflow for large kernels due to autodiff. (Genesis-Embodied-AI#489) * [Misc] Fix purity checker to allow accessing constants from quadrants modules (Genesis-Embodied-AI#487) * [Misc] Increase tolerance for clock monotonic test (Genesis-Embodied-AI#492) * [CI] Serialize api doc workflow (Genesis-Embodied-AI#494) * [CI] Increase tolerance for clock test (Genesis-Embodied-AI#506) * [CI] Increase clock test tolerance to 20% (Genesis-Embodied-AI#509) * [Perf] Add tensor_type parametrization to tile16 tests (Genesis-Embodied-AI#504) * [Perf] Tiles 4b: Migrate tiles16 tests to enable fastcache (Genesis-Embodied-AI#505) * [Perf] Tiles 4c: add Tiles16x16 proxy (Genesis-Embodied-AI#507) * [Perf] Tiles 4d: Consolidate slice error tests using parametrize (Genesis-Embodied-AI#508) * [Perf] Tiles 4: add SharedArray slice support (Genesis-Embodied-AI#482) * [Perf] Tiles 5: add Cholesky benchmark demo (Genesis-Embodied-AI#483) * [Doc] Add user guide page for subgroup shuffle (Genesis-Embodied-AI#512) * [Perf] Implement cross-platform shuffle_down (Genesis-Embodied-AI#510) * [Perf] Add portable subgroup reduce_add and reduce_all_add (Genesis-Embodied-AI#511) * [Perf] Add first warmup config to perf dispatch (Genesis-Embodied-AI#422) * [AutoDiff] Autodiff 1: Add baseline adstack regression test for unary_collections (Genesis-Embodied-AI#500) * [AutoDiff] Autodiff 2: Implement derivative for tan (Genesis-Embodied-AI#501) * [AutoDiff] Autodiff 3: Recompute tanh/exp on the operand in the reverse pass (Genesis-Embodied-AI#502) * [AutoDiff] Autodiff 4: Mark rsqrt as non-linear for adstack promotion (Genesis-Embodied-AI#503) * [AutoDiff] Autodiff 5: Fix adjoint-alloca placement for GlobalLoads outside the current range-for (Genesis-Embodied-AI#496) * [AutoDiff] Autodiff 6: Adstack regression tests (Genesis-Embodied-AI#491) * [AutoDiff] Autodiff 7: Fix header size in AdStackAllocaStmt to match u64 runtime layout (Genesis-Embodied-AI#534) * [AutoDiff] Autodiff 8: Surface LLVM adstack push/pop overflow as a Python exception (Genesis-Embodied-AI#535) * [AutoDiff] Autodiff 9: Guard against LLVM worker-thread stack overflow from large per-task adstack budget (Genesis-Embodied-AI#495) * [AutoDiff] Autodiff 10: Implement adstack for SPIR-V (Genesis-Embodied-AI#490) * [AutoDiff] Autodiff 11: Latent adstack-adjacent fixes (AMDGPU hipFree, flush() keeps ctx_buffers_, always-preallocate) (Genesis-Embodied-AI#536) * [Doc] Add AGENTS.md with instructions for AI agents (Genesis-Embodied-AI#541) * [Bug] Abort kernel execution on assertion failure instead of segfaulting (Genesis-Embodied-AI#419) * [Type] ndarray typing 1: Add eval_str=True to inspect.signature() calls (Genesis-Embodied-AI#411) * [CI] Suppress reportPrivateImportUsage in torch-using files (Genesis-Embodied-AI#552) * [Misc] QD_DUMP_IR dumps to files with the task_id added to the filename (Genesis-Embodied-AI#441) * [Type] ndarray typing 2: Fix NDArray single-arg subscript crash (Genesis-Embodied-AI#412) * [Test] Flush xdist channel before worker exit so test failure reports are visible (Genesis-Embodied-AI#555) * [CI] Reduce test retries on CI from 3 to 1. (Genesis-Embodied-AI#554) * [AutoDiff] Autodiff 12: Heap-backed adstack on LLVM backends (CPU/CUDA/AMDGPU) (Genesis-Embodied-AI#537) * [AutoDiff] Autodiff 13: Heap-backed adstack on SPIR-V backends (Metal, Vulkan) (Genesis-Embodied-AI#493) * [AutoDiff] Autodiff 14: Resolve bounded-inner-loop adstacks without default_ad_stack_size fallback (Genesis-Embodied-AI#539) * [SPIRV] Vulkan SPIR-V correctness: atomic-view aliasing, PSB stride, narrow storage caps, u1 cast, per-init layer recheck (Genesis-Embodied-AI#513) * [Build] Autodiff 15: Replace 2022 MoltenVK pin with LunarG Vulkan SDK fetch and sanitise MoltenVK cap advertisement (Genesis-Embodied-AI#551) * [Test] Suppress stock pytest-timeout to avoid conflict with pytest_hardtle (Genesis-Embodied-AI#557) * [Vulkan] Use SDK validation layer for debugPrintf instead of apt package (Genesis-Embodied-AI#562) * [Test] Fix flaky perf_dispatch tests by increasing work amounts (Genesis-Embodied-AI#559) * [Test] Add --maxfail CLI option to run_tests.py (default 20) (Genesis-Embodied-AI#558) * [CI] Vulkan debug printf fix to address flaky tests (Genesis-Embodied-AI#563) * [Docs] Add a new page to help for first time contributors (Genesis-Embodied-AI#426) Authored-by: v01dxyz <v01dxyz@v01d.xyz> * [AutoDiff] Autodiff 16: Resolve reverse-mode adstack depths per-launch via runtime-evaluated SizeExpr (Genesis-Embodied-AI#543) * Fix: raise error if device memory allocation fails (Genesis-Embodied-AI#451) (Genesis-Embodied-AI#453) Co-authored-by: v01dxyz <v01dxyz@v01d.xyz> Co-authored-by: Hugh Perkins <hughperkins@gmail.com> * [CI] Add CI job to check line wrapping of comments and docs (Genesis-Embodied-AI#564) * [Misc] Add coverage report to PRs, including kernels (Genesis-Embodied-AI#470) * [CI] CI wrap check feeds only diffs to agent (Genesis-Embodied-AI#567) * Skip 'flaky' test on MacOS CI. (Genesis-Embodied-AI#573) * [Test] Fix missing `import sys` in test_fail_device_memory_allocation (Genesis-Embodied-AI#574) * [CI] Fix Vulkan debugPrintf flake with session-scoped warmup (Genesis-Embodied-AI#571) * [AutoDiff] determine_ad_stack_size: replace whole-CFG Bellman-Ford with SCC + DAG DP (Genesis-Embodied-AI#575) * [Test] Fix macOS OOM skip reason to describe actual root cause (Genesis-Embodied-AI#576) * [Lang] whole_kernel_cse: 2.5x compile time speedup on large kernels (Genesis-Embodied-AI#577) * [CI] Add CI check for unnecessarily deleted comments (Genesis-Embodied-AI#570) * [CI] Migrate coverage report to github Check page (Genesis-Embodied-AI#566) * [Lang] Skip IR verifier between passes unless debug=true (Genesis-Embodied-AI#579) * [Lang] Inline AdStack ops on release LLVM codegen: dramatically reduces compile time for adstack-enabled reverse-mode kernels (Genesis-Embodied-AI#584) * [CUDA] Honor offline_cache=False end-to-end so QD_OFFLINE_CACHE=0 actually gives a cold compile (Genesis-Embodied-AI#580) * [Type] Tensor 24 (Genesis-Embodied-AI#561) Co-authored-by: hugh <hugh@slurm-login-0.slurm-login.tenant-slurm.svc.cluster.local> * [Lang] auto_diff host-walk reductions: dramatically faster front-end compile time on adstack-enabled reverse-mode kernels (Genesis-Embodied-AI#587) * [AutoDiff] Speed up reverse-mode kernel launches on GPU backends (Genesis-Embodied-AI#578) * [Vulkan] Move adstack-sizer scratch out of Function-scope memory to fix SPIR-V pipeline build failures (Genesis-Embodied-AI#588) * [AutoDiff] Improve diagnosis of unsupported reverse-mode AD patterns (Genesis-Embodied-AI#590) * [Bug] Fix: promote Ndarray to AnyArray in build_Name for flattened struct fields (Genesis-Embodied-AI#592) * [SPIR-V] Shrink reverse-grad kernel MSL by ~50% (Genesis-Embodied-AI#591) * [CI] Add CI check that PR changes have test coverage (Genesis-Embodied-AI#596) * [Perf] Enable zero-copy in to_torch() and to_numpy() (Genesis-Embodied-AI#450) * Add BufferView: safe sub-range ndarray access for kernels (Genesis-Embodied-AI#585) Co-authored-by: alanray-tech <alanray-tech@users.noreply.github.com> Co-authored-by: Hugh Perkins <hughperkins@gmail.com> * [Doc] Add user-facing fastcache documentation (Genesis-Embodied-AI#597) Co-authored-by: hugh <hugh@slurm-login-0.slurm-login.tenant-slurm.svc.cluster.local> * [Misc] Upgrade to enable v1 dlpack so to_numpy(copy=False) writable (Genesis-Embodied-AI#598) Co-authored-by: root <root@rtx-209-201.slurm-compute.tenant-slurm.svc.cluster.local> * [AutoDiff] Cut reverse-mode adstack memory usage 10x on all backends (Genesis-Embodied-AI#599) * [Misc] Add CI check for feature file factorization (Genesis-Embodied-AI#606) * [Perf] Skip _recursive_set_args for all-Field frozen dataclass structs (Genesis-Embodied-AI#607) Co-authored-by: Cursor <cursoragent@cursor.com> * [AutoDiff] SNode-arm bound-expr capture rejects fold-attack gate indices (Genesis-Embodied-AI#610) * [Misc] Suppress field fastcache warning for qd.Tensor (Genesis-Embodied-AI#615) Co-authored-by: Cursor <cursoragent@cursor.com> * [AutoDiff] Adstack heap: clip reducer count by per-task loop trip count (compile-time and SizeExpr-evaluated) (Genesis-Embodied-AI#611) * [Misc] Forward copy= through qd.Tensor, add copy=None option (Genesis-Embodied-AI#616) Co-authored-by: Cursor <cursoragent@cursor.com> * [Doc] Update README (Genesis-Embodied-AI#617) Co-authored-by: Cursor <cursoragent@cursor.com> * [CI] Fix coverage report showing def lines as uncovered (Genesis-Embodied-AI#623) Co-authored-by: Cursor <cursoragent@cursor.com> * [Perf] Generic launcher: persistent context, JIT-pointer reuse, Metal compute encoder, LLVM-GPU async memory ops (Part 1/2) (Genesis-Embodied-AI#619) * [CI] Encode Python-first testing policy in coverage-check prompt (Genesis-Embodied-AI#622) Co-authored-by: Cursor <cursoragent@cursor.com> * [CI] Add PR Line change report (Genesis-Embodied-AI#624) Co-authored-by: Cursor <cursoragent@cursor.com> * [CI] Disable quadrants pytest plugin during quadrants internal coverage runs (Genesis-Embodied-AI#629) Co-authored-by: Cursor <cursoragent@cursor.com> * [AutoDiff] Adstack load+store eliminations: EliminateRecomputableAdStackPushes pass + leaf extensions (Genesis-Embodied-AI#621) * [CI] Simplify coverage PR comment to a single linked line (Genesis-Embodied-AI#630) * [CUDA] Add AGX Thor, SM_110 (Genesis-Embodied-AI#631) Co-authored-by: Johnny Nunez and Hugh Perkins * [CI] Lines changed report: collapse PR comment to a single linked totals line (Genesis-Embodied-AI#632) * [FEATURE] Support external Metal command queue via qd.init (Genesis-Embodied-AI#618) Co-authored-by: Cursor <cursoragent@cursor.com> * [Perf] Cache adstack-sizer metadata per task across SPIR-V + LLVM-GPU; per-snode / DeviceAllocation invalidation (Part 2/2) (Genesis-Embodied-AI#620) * [AutoDiff] Disable EliminateRecomputableAdStackPushes pending mutated-SNode chain-leaf fix (Genesis-Embodied-AI#633) * [AutoDiff] Adstack chain-clone safety: mutated-SNode leaf reject + load_top consumer-aware guard (Genesis-Embodied-AI#634) * [Docs] Add user-guide page for qd.simt.block.* primitives (Genesis-Embodied-AI#638) * [Docs] Expand qd.simt.subgroup user-guide page to cover every op (Genesis-Embodied-AI#639) * [Perf] Streams 1-4 (Genesis-Embodied-AI#410) * [Docs] Add user-guide page for matrix decompositions and solvers (Genesis-Embodied-AI#643) * [Bug] Revert "[Perf] Streams 1-4 (Genesis-Embodied-AI#410)" (Genesis-Embodied-AI#650) * [Docs] Add user-guide page for atomics and bit operations (Genesis-Embodied-AI#640) * [Docs] Add user-guide page for qd.simt.grid.* primitives (Genesis-Embodied-AI#641) * [AutoDiff] Adstack max-reducer: parallel multi-axis MaxOverRange dispatch (Genesis-Embodied-AI#635) * [AMDGPU] Fix amdgpu parallel rand init (Genesis-Embodied-AI#658) * [Perf] Adstack: skip max-reducer recognizer on CPU + lift host-eval cap (Genesis-Embodied-AI#655) * [Perf] Re-land Streams 1-4 with bug fixes (Genesis-Embodied-AI#653) * [AMDGPU] Apply device_memory_GB=0.3 cap to AMDGPU tests (Genesis-Embodied-AI#659) * [Perf] Per-launch host sync: drop wait_idle on SPIR-V, pin stream and drop stream_synchronize on CUDA/AMDGPU (Genesis-Embodied-AI#654) * [AMDGPU] Unload hipModule_t in JITModuleAMDGPU destructor (Genesis-Embodied-AI#660) * [AMDGPU] Trim default mempool on qd.reset() (Genesis-Embodied-AI#669) * [AMDGPU] Hoist rand-state buffer to process lifetime (Genesis-Embodied-AI#668) * [Streams] Use events for streams serialization on AMDGPU and CUDA (Genesis-Embodied-AI#667) * [Perf] Adstack max-reducer: launch cache + zero-copy result map; content-stable registry_id (Genesis-Embodied-AI#671) * [SPIR-V] dispatch_max_reducers: register each task with the real kernel name (Genesis-Embodied-AI#675) * [AutoDiff] Debug-mode field/grad/dual: dtype, layout, and access-time invariants (Genesis-Embodied-AI#677) * [Docs] Add user-guide page for qd.algorithms.* device-wide algorithms (Genesis-Embodied-AI#642) Co-authored-by: alanray-tech <alan.ray@genesis-ai.company> * [Docs] Doc for existing atomics: switch support table to per-backend columns (Genesis-Embodied-AI#657) Co-authored-by: alanray-tech <alan.ray@genesis-ai.company> * [GPU] Cross gpu atomics (Genesis-Embodied-AI#666) Co-authored-by: alanray-tech <alan.ray@genesis-ai.company> * [GPU] Make block operations portable cross-gpu (Genesis-Embodied-AI#664) * [Perf] CPU LLVM adstack-cache: skip per-launch bump-writes + ndarray_shapes capture on forward-only handles (Genesis-Embodied-AI#685) * [GPU] Cross-GPU for grid ops (Genesis-Embodied-AI#670) * [Math] Make bitop operations portable cross-gpu (Genesis-Embodied-AI#662) * [AMDGPU] Always use wave64, on both RDNA and CDNA (Genesis-Embodied-AI#687) * [AMDGPU] Use syncscope("agent") for atomix xor to avoid CAS livelock (Genesis-Embodied-AI#672) * [GPU] New bit ops for QIPC (Genesis-Embodied-AI#679) * [GPU] Subgroup ops cross-gpu (Genesis-Embodied-AI#665) * [Graph] Rename CUDA Graph to Graph in docs (Genesis-Embodied-AI#691) * [SPIR-V] Fix FIFO-queue ordering when sharing command queue. (Genesis-Embodied-AI#694) * [Atomics] New QIPC ops for atomics (Genesis-Embodied-AI#690) * Pass dataclass sub-structs into qd.func (Genesis-Embodied-AI#698) * [AMDGPU] HIP graph runtime support for @qd.kernel(graph=True) (Genesis-Embodied-AI#692) * [CI] Add per-file timing report to Mac Metal test job (Genesis-Embodied-AI#695) Co-authored-by: Cursor <cursoragent@cursor.com> * [CI] Enable kernel disk cache during tests (Genesis-Embodied-AI#696) * [Math] New QIPC ops for single-threaded linalg (Genesis-Embodied-AI#683) * [BREAKING][GPU] New QIPC ops for subgroups (Genesis-Embodied-AI#676) * [GPU] New QIPC ops for block (Genesis-Embodied-AI#684) * [GPU] New device-level ops for QIPC (Genesis-Embodied-AI#693) * [algorithms] PrefixSumExecutor: drop unused GRID_SZ local (Genesis-Embodied-AI#701) * [block] sync(): fix unsupported-arch error message (Genesis-Embodied-AI#700) * [volatile_load] add qd.volatile_load primitive (closes Genesis-Embodied-AI#648) (Genesis-Embodied-AI#702) * [AutoDiff] Reject recycled identity_key in AdStackCache::register_adstack_sizing_info (Genesis-Embodied-AI#708) * [Vulkan] Declare GroupNonUniform SPIR-V caps and enable shaderSubgroupExtendedTypes (Genesis-Embodied-AI#707) * Fix duplicate HIP graph driver-function declarations after v1.0.0 merge The amd-integration fork had cherry-picked the HIP graph driver functions (graph_create / graph_destroy / graph_add_kernel_node / graph_instantiate / graph_exec_destroy / graph_launch), and upstream v1.0.0 added the same set. The per-file 3-way merge appended both copies into amdgpu_driver_functions.inc.h, producing redeclaration errors that broke the AMDGPU RHI/runtime compile. Drop the upstream duplicate block; the signatures are identical to the fork's existing declarations. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix AMDGPU launcher coherence and num_instructions visibility after v1.0.0 merge - kernel_launcher.cpp: the 3-way merge spliced upstream v1.0.0's launch_llvm_kernel rewrite (ephemeral arg/context buffers, explicit-stream path, AmdgpuDefaultStream PinGuard) onto the AMD fork's kernarg-by-value + persistent-scratch design, leaving references to undefined `ephemeral_context_ptr`. Restore the fork's coherent launch_llvm_kernel verbatim; it calls the (already merged) enhanced launch_offloaded_tasks, which keeps the max-reducer dispatch and stream-parallel groups adapted onto the AMD launch path. - llvm_context.h: both the fork and upstream added `num_instructions`; the merge kept upstream's private placement, but the AMDGPU codegen force-inline heuristic calls it statically from outside the class. Move it back to the public section. Co-authored-by: Cursor <cursoragent@cursor.com> * Restore async result D2H and hoist kernarg vectors in AMDGPU launcher The v1.0.0 merge resolution regressed two amd-integration baseline optimizations in launch_llvm_kernel / launch_offloaded_tasks: - The per-launch result-buffer copy was a blocking memcpy_device_to_host, forcing a host stall on every value-returning launch and serializing the GPU pipeline. Restore the async D2H (the caller synchronizes lazily when it needs the value); external-array transfers still stream_synchronize once before reading back. - launch_task constructed the kernarg std::vectors from initializer lists ({kernarg_payload} / {kernarg_size}) on every dispatch (heap alloc + free per launch). Hoist arg_ptrs/arg_sizes out of the per-task launch and reuse. Co-authored-by: Cursor <cursoragent@cursor.com> * amdgpu: default to LDS permlane64 emulation; drop host-x86 barrier asm on retarget Two AMDGPU JIT-compile crashes surfaced after the v1.0.0 merge pulled in the QIPC subgroup ops (Genesis-Embodied-AI#676), which made the rigid constraint solver's wave-cooperative reductions route through `amdgpu_cross_half_shuffle_i32`. Both manifested as a SIGSEGV inside `llvm::SIInstrInfo::getInstSizeInBytes` during `JITSessionAMDGPU::compile_module_to_hsaco` (i.e. at first kernel launch), and reproduce on gfx942 / MI300X. Baseline 0.4.6 never emitted these constructs, which is why it was unaffected. 1. Native `llvm.amdgcn.permlane64` lowering crashes the bundled LLVM 22.1.0 AMDGPU backend. Default `amdgpu_permlane64` to the existing LDS-roundtrip software emulation on every target (it produces identical results). Add `QD_AMDGPU_USE_NATIVE_PERMLANE64=1` to opt back into the native instruction once the backend bug is fixed; the old `QD_AMDGPU_FORCE_PERMLANE64_FALLBACK` is now the default and still honored. This is the actual crash fix. 2. The runtime module is compiled by the host x86_64 clang and only retargeted to amdgcn here, so `amdgpu_cross_half_shuffle_i32`'s `__asm__ volatile("" : "+v"(byte))` optimization barrier carries x86 flag clobbers (`~{dirflag},~{fpsr},~{flags}`) that are meaningless on AMDGPU. The IR verifies but the empty-body INLINEASM is invalid on the amdgcn target. Neutralize empty-body barrier asm during retarget (forward the tied value, then erase) so no stale host asm reaches codegen. On the wave64 targets we ship `ds_bpermute` already addresses the full wave, so the hint is a no-op. Co-authored-by: Cursor <cursoragent@cursor.com> * style: apply clang-format (v19.1.7) to AMDGPU fn_attrs and launcher sources CI pre-commit's clang-format hook reformatted these files (long declarations/lambda signatures collapsed onto single lines per the repo's clang-format config). Apply the same formatting so the hook passes. No functional changes. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(amdgpu): use CreateNeg for branchless i32 sgn instead of CreateSub(0, input) clang-tidy (modernize-use-nullptr, -warnings-as-errors) flagged `builder->CreateSub(0, input)` in the i32 sgn path: the literal `0` binds to the `llvm::Value*` LHS parameter as a null pointer, not an integer zero. Replace with `builder->CreateNeg(input)`, which emits `0 - input` with a proper zero constant -- identical intended semantics, and clang-tidy clean. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Robert Dazi <14996868+v01dXYZ@users.noreply.github.com> Co-authored-by: v01dxyz <v01dxyz@v01d.xyz> Co-authored-by: Hugh Perkins <hughperkins@gmail.com> Co-authored-by: Alexis DUBURCQ <alexis.duburcq@gmail.com> Co-authored-by: hugh <hugh@slurm-login-0.slurm-login.tenant-slurm.svc.cluster.local> Co-authored-by: alanray-tech <alan.ray@genesis-ai.company> Co-authored-by: alanray-tech <alanray-tech@users.noreply.github.com> Co-authored-by: root <root@rtx-209-201.slurm-compute.tenant-slurm.svc.cluster.local> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Johnny <johnnynuca14@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue: #
Brief Summary
copilot:summary
Walkthrough
copilot:walkthrough