wipe out existing CI#2
Closed
hughperkins wants to merge 2 commits into
Closed
Conversation
Contributor
|
I'm not sure to understand why you need to start again from scratch. That would kind of obfuscate your changes compared to the original repo and make it harder to track changes. If you did something completely different without any intersection then go ahead, but I don't like much the idea to have no workflow anymore while they are seemingly doing something better than nothing? |
Collaborator
Author
|
Yeah, that's fair. Alright, so, I have the following things that I could bring in:
The linux and windows runners are fairly straight forward, as is the linters. The mac os x runner is a lootttttt of changes. Thoughts on what you would like me to bring into this? Maybe linux runner + linters? linux runner + windows runner + linters? |
Collaborator
Author
|
Well... why dont we go the other way around? I'll add in the linters, and each runner (separate prs), and then once I've added what we want, we can remove what we dont want? |
Contributor
|
I like this idea :) |
This was referenced Mar 12, 2026
This was referenced Apr 15, 2026
hughperkins
added a commit
that referenced
this pull request
Apr 26, 2026
Introduce the helper machinery that the per-class to_torch / to_numpy methods will migrate to in subsequent commits. Existing public symbols (can_zerocopy, dlpack_to_torch, invalidate_zerocopy_cache, current_arch_is_cpu) are preserved as deprecated shims so the in-tree pre-rework callers continue to work; they will be removed once every call site is migrated. New surface: - _ZerocopyCache: per-instance container with two independent slots (torch tensor + numpy ndarray), each filled lazily on first access via torch.utils.dlpack.from_dlpack and numpy.from_dlpack respectively. Numpy zero-copy now bypasses torch entirely (closes review #6). - make_zerocopy_cache_if_supported(owner, ...): constructs a cache when zero-copy is supported and registers `owner` with `pyquadrants.cache_holders` so invalidation is wired automatically (closes review #18). - get_zerocopy_torch / get_zerocopy_numpy: thin entry points that implement the always-zerocopy-then-clone semantic (closes review #15, #16, #21) and the Apple Metal double-sync (qd.sync() on read AND torch.mps.synchronize() after .clone()/.to(); closes review #1, #22, #23). Also applies the small lints from the review: - Module-level constant for the torch>2.9.1 MPS bytes_offset probe; drops the pointless lru_cache wrapper around a zero-arg helper (closes review #2). - ASCII '...' instead of Unicode horizontal ellipsis '\u2026' in the docstring (closes review #3). - Top-level imports for numpy and torch (try/except for the no-torch CI case); no per-call lazy imports in the new code path (closes review #7, #9). The deprecated shim still does what the existing per-class methods expect; the new helpers are torch-clean. cache_holders is still empty until the next commits register Ndarray / ScalarField / MatrixField; this commit alone is no-op behaviourally.
Merged
hughperkins
added a commit
that referenced
this pull request
May 18, 2026
…llback
Unrecognised types in fastcache argument hashing previously had two failure modes, both bad:
- Top-level: ``[FASTCACHE][PARAM_INVALID]`` warn + return None, disabling fastcache for the whole call.
Any solver-like object carrying a single opaque metadata field (Genesis ``UID``, Pydantic config,
back-pointer) silently killed the cache.
- Nested under ``@qd.data_oriented(stable_members=True)``: silent skip. Worked for the Genesis case but is
dangerous: if someone later adds a new tensor-like type (e.g. ``BFloat16Tensor``) whose value affects
kernel codegen but forgets to register it in args_hasher's recognised set, the silent skip serves stale
cache results without any indication.
Both paths are replaced with a single ``type(v).__qualname__``-based fallback (``opaque-<module>.<qualname>``)
that emits a one-shot ``[FASTCACHE][UNKNOWN_TYPE]`` warning per type. Properties:
- Cache key stable across instances of the same opaque class (Genesis UID #1 and UID #2 produce the same
key). Kernels cannot read non-recognised Python types so opaque metadata cannot affect codegen, making
type-identity-only hashing correct for genuinely opaque members.
- Loud diagnostic for the dangerous case: any unrecognised type that ever gets hashed prints a warning
pointing at args_hasher.stringify_obj_type so a missed tensor-like registration is impossible to miss.
- ``ScalarField`` / ``MatrixField`` (recognised-but-unsupported tensor-like) still disable fastcache via
a new ``_FAIL_FASTCACHE`` sentinel — their shape/dtype affect codegen but fastcache doesn't yet handle
them. Distinct from the qualname fallback so the field path remains correct.
Also adds ``pruning_paths`` and ``parent_flat`` plumbing through ``stringify_obj_type`` / ``dataclass_to_repr`` /
``hash_args`` for the upcoming pruning-driven narrow walk (L1 cache lookup of kernel-accessed flat names);
the new parameters default to None so this commit alone is the qualname-fallback baseline.
``test_src_ll_cache_arg_warnings`` updated to assert the new ``[UNKNOWN_TYPE]`` warning (instead of the old
``[PARAM_INVALID]`` + ``[INVALID_FUNC]`` dead-end).
The ``_qd_stable_members`` flag is no longer read by args_hasher; its launch-context role
(``_mutable_nd_cached_val`` short-circuit) is unchanged in this commit and will be addressed separately.
hughperkins
added a commit
that referenced
this pull request
May 19, 2026
…llback
Unrecognised types in fastcache argument hashing previously had two failure modes, both bad:
- Top-level: ``[FASTCACHE][PARAM_INVALID]`` warn + return None, disabling fastcache for the whole call.
Any solver-like object carrying a single opaque metadata field (Genesis ``UID``, Pydantic config,
back-pointer) silently killed the cache.
- Nested under ``@qd.data_oriented(stable_members=True)``: silent skip. Worked for the Genesis case but is
dangerous: if someone later adds a new tensor-like type (e.g. ``BFloat16Tensor``) whose value affects
kernel codegen but forgets to register it in args_hasher's recognised set, the silent skip serves stale
cache results without any indication.
Both paths are replaced with a single ``type(v).__qualname__``-based fallback (``opaque-<module>.<qualname>``)
that emits a one-shot ``[FASTCACHE][UNKNOWN_TYPE]`` warning per type. Properties:
- Cache key stable across instances of the same opaque class (Genesis UID #1 and UID #2 produce the same
key). Kernels cannot read non-recognised Python types so opaque metadata cannot affect codegen, making
type-identity-only hashing correct for genuinely opaque members.
- Loud diagnostic for the dangerous case: any unrecognised type that ever gets hashed prints a warning
pointing at args_hasher.stringify_obj_type so a missed tensor-like registration is impossible to miss.
- ``ScalarField`` / ``MatrixField`` (recognised-but-unsupported tensor-like) still disable fastcache via
a new ``_FAIL_FASTCACHE`` sentinel — their shape/dtype affect codegen but fastcache doesn't yet handle
them. Distinct from the qualname fallback so the field path remains correct.
Also adds ``pruning_paths`` and ``parent_flat`` plumbing through ``stringify_obj_type`` / ``dataclass_to_repr`` /
``hash_args`` for the upcoming pruning-driven narrow walk (L1 cache lookup of kernel-accessed flat names);
the new parameters default to None so this commit alone is the qualname-fallback baseline.
``test_src_ll_cache_arg_warnings`` updated to assert the new ``[UNKNOWN_TYPE]`` warning (instead of the old
``[PARAM_INVALID]`` + ``[INVALID_FUNC]`` dead-end).
The ``_qd_stable_members`` flag is no longer read by args_hasher; its launch-context role
(``_mutable_nd_cached_val`` short-circuit) is unchanged in this commit and will be addressed separately.
hughperkins
added a commit
that referenced
this pull request
May 19, 2026
…llback
Unrecognised types in fastcache argument hashing previously had two failure modes, both bad:
- Top-level: ``[FASTCACHE][PARAM_INVALID]`` warn + return None, disabling fastcache for the whole call.
Any solver-like object carrying a single opaque metadata field (Genesis ``UID``, Pydantic config,
back-pointer) silently killed the cache.
- Nested under ``@qd.data_oriented(stable_members=True)``: silent skip. Worked for the Genesis case but is
dangerous: if someone later adds a new tensor-like type (e.g. ``BFloat16Tensor``) whose value affects
kernel codegen but forgets to register it in args_hasher's recognised set, the silent skip serves stale
cache results without any indication.
Both paths are replaced with a single ``type(v).__qualname__``-based fallback (``opaque-<module>.<qualname>``)
that emits a one-shot ``[FASTCACHE][UNKNOWN_TYPE]`` warning per type. Properties:
- Cache key stable across instances of the same opaque class (Genesis UID #1 and UID #2 produce the same
key). Kernels cannot read non-recognised Python types so opaque metadata cannot affect codegen, making
type-identity-only hashing correct for genuinely opaque members.
- Loud diagnostic for the dangerous case: any unrecognised type that ever gets hashed prints a warning
pointing at args_hasher.stringify_obj_type so a missed tensor-like registration is impossible to miss.
- ``ScalarField`` / ``MatrixField`` (recognised-but-unsupported tensor-like) still disable fastcache via
a new ``_FAIL_FASTCACHE`` sentinel — their shape/dtype affect codegen but fastcache doesn't yet handle
them. Distinct from the qualname fallback so the field path remains correct.
Also adds ``pruning_paths`` and ``parent_flat`` plumbing through ``stringify_obj_type`` / ``dataclass_to_repr`` /
``hash_args`` for the upcoming pruning-driven narrow walk (L1 cache lookup of kernel-accessed flat names);
the new parameters default to None so this commit alone is the qualname-fallback baseline.
``test_src_ll_cache_arg_warnings`` updated to assert the new ``[UNKNOWN_TYPE]`` warning (instead of the old
``[PARAM_INVALID]`` + ``[INVALID_FUNC]`` dead-end).
The ``_qd_stable_members`` flag is no longer read by args_hasher; its launch-context role
(``_mutable_nd_cached_val`` short-circuit) is unchanged in this commit and will be addressed separately.
hughperkins
added a commit
that referenced
this pull request
Jun 1, 2026
…ning & ndarray-wrapper tests Three changes addressing the PR #704 CI feedback agents: 1. **Feature-factorization fix** — moved ``Kernel._chain_has_mutable_container`` (an ``@staticmethod`` with zero coupling to ``Kernel`` internals) out of the 812-line ``kernel.py`` and into ``_template_mapper_hotpath.py`` as the free function ``chain_has_mutable_container``. This sits alongside the sibling struct-walking helpers ``_struct_nd_paths_for`` / ``_build_struct_nd_paths`` / ``_collect_struct_nd_descriptors`` that this same PR introduced — the "Check feature factorization" agent (run 26662318558) correctly noted those were placed there but the chain-walker was left behind on ``Kernel``. ``kernel.py`` now imports ``chain_has_mutable_container`` and the only call-site in ``_get_launch_ctx`` calls the free function directly. 2. **Pytest coverage for the DeprecationWarning** — added three tests to ``test_py_dataclass.py`` mirroring the probe in ``tmp/probe_template_dataclass_warning.py``: * ``test_dataclass_with_template_emits_deprecation_warning`` — asserts the warning fires (``pytest.warns`` with the message-regex anchor). * ``test_data_oriented_with_template_does_not_emit_deprecation_warning`` — asserts the warning is suppressed for ``@qd.data_oriented`` instances (the ``is_data_oriented(val)`` exclusion in ``Kernel.materialize``). * ``test_typed_dataclass_does_not_emit_deprecation_warning`` — asserts the supported flat-by-fields path emits no warning. Addresses the "Check test coverage" gap #1 (run 26662318494). 3. **Pytest coverage for the ndarray-wrapper unwrap branch** — added ``test_data_oriented_ndarray_wrapper`` to ``test_data_oriented_ndarray.py`` that constructs a ``@qd.data_oriented`` whose member is a ``qd.Tensor`` wrapper around a ``qd.ndarray`` (``qd.tensor(..., backend=qd.Backend.NDARRAY)``). Confirmed via the throwaway script in ``tmp/verify_unwrap_hit.py`` that both ``_build_struct_nd_paths`` and ``_collect_struct_nd_descriptors`` hit their ``if type(v) in _TENSOR_WRAPPER_TYPES: v = v._unwrap()`` branches (1 hit each) during this test. Addresses the "Check test coverage" gap #2. Regression: 143 tests pass (was 123 + new) across ``test_data_oriented_ndarray.py`` + ``test_data_oriented_mixed_combos.py`` + ``test_py_dataclass.py`` + ``test_tensor_wrapper_in_struct.py``, no failures.
hughperkins
added a commit
that referenced
this pull request
Jul 16, 2026
Pre-offload the pass found nothing (PTRMERGE modified=0): offload is what clones each global's address computation into per-task read/write GlobalPtrStmts. Run the merge on the offloaded IR right before flag_access #2 so the still-activate=true duplicates unify into one shared pointer before flag_access can stamp the read-only copy activate=false. This is the split that blocks cache_loop_invariant_global_vars.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue: #
Brief Summary
Make a clean sheet, to start from fresh.
You can see where I'm going here:
(a bnch of tests fail, but the build and install works)
copilot:summary
Walkthrough
copilot:walkthrough