[Perf] Tiles 5: add Cholesky benchmark demo

[hughperkins]

Issue: #

Brief Summary

Summary

Adds a complete blocked Cholesky factorization demo (misc/demos/cholesky_blocked.py) that benchmarks register-resident Tile16x16 against two scalar-Crout baselines, and
links it from the Tile16x16 user guide.

What's in this PR

misc/demos/cholesky_blocked.py (323 lines) — a self-contained benchmark script that implements and compares three batched 92×92 Cholesky kernels across 4096
environments:

1. Baseline: scalar Cholesky-Crout in shared memory, 64 threads, 2N+1 sequential block.sync() calls.
2. Blocked: same 6×6 tile blocking structure as Tile16x16 but using shared memory and scalar Crout for diagonal blocks, 16 threads.
3. Tile16x16: fully register-resident via Tile16x16, no shared memory, zero syncs, 16 threads.
   The script loads SPD test matrices, runs warmup + timed iterations, verifies reconstruction error, and prints a results table.
   docs/source/user_guide/tile16.md — adds a "Example: blocked Cholesky" section at the end with a link to the demo and a results table (5.19× speedup on Blackwell).

Good points

• Pedagogically valuable: shows three progressively better implementations side-by-side, making it easy to see why register-resident tiles win (elimination of shared
  memory and sync barriers).
• Self-contained: runs with a single python misc/demos/cholesky_blocked.py — generates its own test data, verifies correctness, and benchmarks.
• Concrete performance evidence: documents real numbers on production hardware (RTX PRO 6000 Blackwell), giving users calibrated expectations.
• Small diff: only 2 files, 335 lines added, no changes to library code.

Bad points

• CUDA-only: the demo hardcodes qd.init(arch=qd.cuda) and won't run on Vulkan/Metal.
• Hardcoded problem size: N=92 and 4096 environments are baked in as module-level constants rather than being CLI-configurable.
• Not a unit test: the demo lives in misc/demos/ and isn't exercised by CI, so it could silently break if the Tile16x16 API changes.
• Benchmark numbers will drift: the results table in the docs is a static snapshot from one specific GPU; future hardware or compiler changes won't be reflected.

copilot:summary

Walkthrough

copilot:walkthrough

[hughperkins]

Opus actions to address 'bad points' in the description:

User: modify the demo to work on qd.gpu

Assistant:

Done. Changed qd.cuda to qd.gpu and pushed.

User: add a unit test to qaudrants unit tests that smoke tests that cholesksy_blocked.md runs to completion without crashing …

add a unit test to qaudrants unit tests that smoke tests that cholesksy_blocked.md runs to completion without crashing (exit code 0). This should run on qd.gpu

---

Assistant:

Let me find existing test patterns and see if there are any subprocess-based smoke tests already.

User: sorry, cholesky_blocked.py

Assistant:

Let me check how the existing test file for tile16 is structured and see test_utils.test() usage:

Let me check how test_utils.test() works with GPU requirements:

Now let me check how subprocess is used in existing tests for a pattern to follow:

The demo uses module-level qd.init() and qd.field(), so it needs to run as a subprocess. Let me add a simple smoke test:

Now add the smoke test at the end of the file:

Done. Added test_tile16_cholesky_blocked_demo at the end of test_tile16.py — it runs misc/demos/cholesky_blocked.py as a subprocess, requires qd.gpu, and fails the test if the exit code is non-zero (with stderr in the failure message). 5-minute timeout to allow for compilation.

User: run hte test, hceck it passes

Assistant:

Passes on both CUDA and Vulkan. 15 seconds total.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bfa3c81965

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Pass selected test backend into demo subprocess

This test is parametrized with @test_utils.test(arch=qd.gpu), but the subprocess command does not pass the active req_arch to the child. The demo then initializes with qd.init(arch=qd.gpu), so on multi-backend machines the vulkan/metal parametrizations can still run on a different backend (typically CUDA), which makes this test miss backend-specific regressions. Pass the current backend name to the script and initialize that exact arch in the child process.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Keep smoke test from executing full benchmark workload

The smoke test currently invokes the benchmark script with its hardcoded heavy settings (N_ENVS=4096, WARMUP=50, ITERS=200), which means one unit test run performs hundreds of large GPU launches and allocates several large fields. Because this test is not marked run_in_serial and runs in a separate subprocess, it bypasses the per-test CUDA memory throttling used in tests/python/conftest.py, increasing the chance of OOM/timeouts when xdist runs tests concurrently. Add a lightweight test mode (or CLI/env overrides) and use that here.

Useful? React with 👍 / 👎.

[claude]

🟡 The guard 'if r < TILE' in the triangular solve section of cholesky_blocked() is vacuously true and never skips any thread. Since tid = idx % TILE with TILE=16, r = tid is always in [0, 15], so 'if r < TILE' is always True. The dead conditional can be removed; it may mislead readers by analogy with the meaningful 'if row < TILE' guard in the Crout diagonal section above, where row = col + 1 + tid can genuinely exceed TILE.

Extended reasoning...

What the bug is: In the triangular solve loop inside cholesky_blocked() (lines 180-192 of misc/demos/cholesky_blocked.py), the code sets r = tid and then guards the body with 'if r < TILE'. Since tid = idx % TILE (with TILE=16), tid is always in the range [0, 15]. Setting r = tid means r is always in [0, 15], so 'if r < TILE' — i.e. 'if r < 16' — is always True and the guard never prevents execution of the body.

The specific code path: In cholesky_blocked(), the outer loop sets tid = idx % TILE (TILE=16), so tid is in {0, 1, ..., 15}. In the off-diagonal block triangular solve:

for c in range(TILE):
    r = tid
    if r < TILE:   # always True
        dot = qd.f32(0.0)
        ...
    qd.simt.block.sync()

Since r = tid and tid is always < 16 = TILE, the branch condition is never False.

Why existing code does not prevent it: The condition looks structurally similar to the meaningful guard in the Crout diagonal factorization just above:

row = col + 1 + tid   # col ranges 0..15, so row can be 16+
if row < TILE:        # meaningful guard, can be False

Here row can exceed TILE (when col + 1 + tid >= 16), so that guard does real work. The triangular solve was likely written by analogy with the Crout section, but the analogy breaks down because r = tid directly (not shifted by col + 1), so it can never reach TILE.

Step-by-step proof: Take TILE=16, tid=15 (maximum). In the Crout section, row = col + 1 + 15; when col=0, row=16 >= TILE, so 'if row < TILE' is False and that thread is skipped. In the triangular solve, r = tid = 15 < 16 = TILE, so 'if r < TILE' is True — the guard never fires for any value of tid.

Impact: The code is algorithmically correct; all 16 threads need to execute the triangular solve body for all 16 rows of the off-diagonal block. The dead guard is purely misleading: a reader may infer (by analogy with the diagonal Crout section) that some threads conditionally skip work, when in fact all threads always execute. This is a nit in a demo script.

How to fix: Remove the 'if r < TILE:' guard (and the corresponding indentation change), leaving only the unconditional body plus the sync.

[claude]

🟡 In verify(), detecting NaN/inf prints 'FAILED (nan/inf)' and returns float('inf') but does not raise or call sys.exit(); main() ignores all three return values. As a result, the smoke test added in this PR (test_tile16_cholesky_blocked_demo) can only detect crashes, not correctness failures — a kernel silently producing NaN/inf would print a FAILED message but exit 0, causing CI to pass.

Extended reasoning...

What the bug is: In misc/demos/cholesky_blocked.py, the verify() function (lines 267–280) detects NaN/inf by printing a failure message and returning float('inf'). It never raises an exception or calls sys.exit(1). In main() (lines 294, 299, 304), all three calls to verify() discard the return value entirely. Consequently, a correctness failure in any kernel exits with code 0.

Specific code path: verify() hits if np.isnan(err) or np.isinf(err) → prints → return float('inf'). main() calls verify(...) with no assignment or check. The script reaches its normal end and exits 0.

Why existing code doesn't prevent it: The smoke test in test_tile16.py checks only result.returncode \!= 0. Since the script exits 0 in all correctness-failure scenarios, the test always passes regardless of whether the kernels produced correct output.

Impact: The smoke test was explicitly added to address the PR's stated "bad point" — that the demo "could silently break if the Tile16x16 API changes." A Tile16x16 API regression that causes NaN/inf output would: print "FAILED (nan/inf)" to stdout, exit 0, and pass CI. The test catches outright crashes (import errors, kernel compile failures, exceptions) but not the more subtle case of a kernel producing garbage output.

Addressing the refutation: The refutation argues that the smoke test is intentionally checking only API stability (exit code 0), not correctness, and that individual operations are unit-tested in isolation. This is a reasonable position — the test docstring says "runs to completion" and the unit tests in test_tile16.py do cover individual tile primitives. However, the multi-tile blocked composition itself is not unit-tested (as noted in Q11 of the review), so the demo serves as the only exercise of the end-to-end blocked algorithm. A regression that passes all unit tests but corrupts the composition (e.g. a change to solve_triangular_()'s interaction with outer() in the Schur complement loop) would go undetected by CI.

Step-by-step proof: (1) Suppose a future change causes cholesky_tile16 to write NaN into L_tile16_field. (2) verify("tile16", L_tile16_field, A_np) computes err = np.max(np.abs(...)) → np.nan. (3) np.isnan(err) is True → prints "tile16 env 0: FAILED (nan/inf)", returns float('inf'). (4) Back in main(), no assignment: the return value is dropped. (5) Script prints benchmark numbers (which may still run since the kernel doesn't crash) and exits normally with code 0. (6) subprocess.run returns returncode=0. (7) pytest.fail is never reached. CI passes.

Fix: Check verify's return value in main() and call sys.exit(1) if it is infinite, or raise an exception inside verify() on failure.