Arm64: Improve support for HW_Flag_ReturnsPerElementMask

[snickolls-arm]

When wrapping an intrinsic node that has an embedded mask with a ConditionalSelect, ensure that the constant node in op3 has a mask type when the intrinsic has the HW_Flag_ReturnsPerElementMask flag.

Build out further support for ConditionalSelect_Predicates, and use this to wrap nodes with HW_Flag_ReturnsPerElementMask. Add GenTree::IsSelectZero and update various areas in HW intrinsic codegen to ensure this intrinsic assembles correctly.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[dhartglassMSFT]

FYI if it's not known, assert in PR testing

11:15:45.200 Running test: JIT/opt/SVE/ConditionalSelectConstants/ConditionalSelectConstants.cmd

Assert failure(PID 5320 [0x000014c8], Thread: 6404 [0x1904]): Assertion failed 'maskBit == 0' in 'ConditionalSelectConstants:ConditionalSelect1CC():System.Numerics.Vector`1[int]' during 'Morph - Global' (IL size 24; hash 0x36e039d8; FullOpts)

    File: D:\a\_work\1\s\src\coreclr\jit\simd.h:2145
    Image: C:\h\w\AEC00944\p\corerun.exe


Return code:      1

[snickolls-arm]

FYI if it's not known, assert in PR testing

11:15:45.200 Running test: JIT/opt/SVE/ConditionalSelectConstants/ConditionalSelectConstants.cmd

Assert failure(PID 5320 [0x000014c8], Thread: 6404 [0x1904]): Assertion failed 'maskBit == 0' in 'ConditionalSelectConstants:ConditionalSelect1CC():System.Numerics.Vector`1[int]' during 'Morph - Global' (IL size 24; hash 0x36e039d8; FullOpts)

    File: D:\a\_work\1\s\src\coreclr\jit\simd.h:2145
    Image: C:\h\w\AEC00944\p\corerun.exe


Return code:      1

I didn't see thanks, will take a look.

[Copilot]

Pull request overview

This PR extends Arm64 SVE support for intrinsics that return per-element masks by introducing shared recognition of SVE conditional-select variants, adding a “select-zero” helper, updating containment/codegen to treat mask-zero similarly to vector-zero, and updating constant-folding / emitter behavior to match SVE predicate element semantics.

Changes:

• Adds HWIntrinsicInfo::IsSveConditionalSelect and GenTree::IsSelectZero, and updates lowering/LSRA/codegen paths to use them.
• Updates SVE mask creation intrinsics to be treated as returning per-element masks and adjusts import/constant construction accordingly.
• Adjusts SVE emitter options handling and updates/extends a JIT test to validate the new mask element representation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/tests/JIT/opt/SVE/ConditionalSelectConstants.cs	Updates expectations and adds a smoke check validating mask element values.
src/coreclr/jit/simd.h	Changes mask-to-vector constant evaluation semantics on Arm64 and adds helper to convert elementwise mask values to bitwise masks for folding.
src/coreclr/jit/lsraarm64.cpp	Treats both vector and predicate conditional-select forms uniformly when identifying embedded/contained selects.
src/coreclr/jit/lowerarmarch.cpp	Expands lowering/containment to include predicate conditional-select and uses IsSelectZero for zero-detection.
src/coreclr/jit/hwintrinsiclistarm64sve.h	Marks SVE CreateTrue/FalseMask intrinsics as returning per-element masks; updates containment flag for predicate select.
src/coreclr/jit/hwintrinsiccodegenarm64.cpp	Switches embedded-mask setup and containment checks from vector-zero to select-zero, enabling mask-zero handling.
src/coreclr/jit/hwintrinsicarm64.cpp	Imports SVE false masks via gtNewZeroConNode and updates true-mask constant folding to use mask constants.
src/coreclr/jit/hwintrinsic.h	Introduces IsSveConditionalSelect helper for consistent conditional-select recognition.
src/coreclr/jit/gentree.h	Adds GenTree::IsSelectZero helper for conditional-select zero detection across vectors/masks.
src/coreclr/jit/gentree.cpp	Extends gtNewZeroConNode to create mask-zero constants and updates folding for SVE mask representation.
src/coreclr/jit/emitarm64sve.cpp	Forces byte arrangement for predicate encodings where required and relaxes/adjusts scalable option assertions.

Comments suppressed due to low confidence (1)

src/coreclr/jit/lowerarmarch.cpp:3636

• TryContainingCselOp now accepts both NI_Sve_ConditionalSelect and NI_Sve_ConditionalSelect_Predicates, but the containment fast-path is still gated on childNode->Op(3)->IsVectorZero(). For the predicate variant, op3 is expected to be a mask-zero (pfalse) rather than a vector-zero, so this will never trigger and makes the new broader check ineffective for mask selects.

    if (!HWIntrinsicInfo::IsSveConditionalSelect(childNode->GetHWIntrinsicId()))
    {
        return false;
    }

    if (childNode->Op(2)->IsEmbMaskOp())
    {
        // Do not optimise if the conditional select node is added to embed the mask for its op2. Such conditional
        // select nodes are optimised out while emitting the embedded variant of an instruction associated with op2.
        assert(childNode->Op(2)->isContained());
        return false;
    }

    bool canContain = false;

    var_types simdBaseType = parentNode->GetSimdBaseType();
    if (childNode->Op(3)->IsVectorZero())
    {

[tannergooding]

For now I'd rather just get this PR completed so I can start contributing the Vector intrinsic implementations I've done using SVE.

@snickolls-arm, as an FYI this is going to change a bit with #130049.

It is effectively merging most of the tables for the xplat intrinsics into a new combined table and the import logic itself into a combined handler as well. The handling between Arm64 AdvSimd and xarch was already nearly identical and the same will again be true for Arm64 SVE and the WASM support we're onboarding, with most of the differences (if any) being in the gtNewSimd* helper APIs or in the actual codegen handling, which was often special.

That should make it a bit simpler to lightup for SVE, as you'll not need to extend or duplicate the tables again or add a bunch of NI_VectorT_* entries alongside the existing NI_Vector64_* and NI_Vector128_* handlers. Instead, it's purely NI_Vector_* for all cases and differentiated based on simdSize (i.e. SIZE_UNKNOWN for SVE).

[tannergooding]

Not quite sure I understand this assert?

Sve.CreateTrueMaskByte is a managed API, and so I'd expect it to be producing a TYP_SIMD, same as for NI_Sve_CreateFalseMask.

So we should either produce a CNS_VEC or a CvtMaskToVector(truemaskall(op1)). Downstream consumers that expect a mask will then have CvtVectorToMask(CNS_VEC) which folds to CNS_MSK or CvtVectorToMask(CvtMaskToVector(truemaskall(op1)) which folds to just truemaskall(op1)

[snickolls-arm]

It's due to the way it's wired up with the main import function:

runtime/src/coreclr/jit/hwintrinsic.cpp

Lines 2417 to 2422 in 8fe2929

	#if defined(FEATURE_MASKED_HW_INTRINSICS) && defined(TARGET_ARM64)
	if (HWIntrinsicInfo::ReturnsPerElementMask(intrinsic))
	{
	// Ensure the result is generated to a mask.
	nodeRetType = TYP_MASK;
	}

runtime/src/coreclr/jit/hwintrinsic.cpp

Lines 2872 to 2876 in 8fe2929

	if (nodeRetType == TYP_MASK)
	{
	// HWInstrinsic returns a mask, but all returns must be vectors, so convert mask to vector.
	retNode = gtNewSimdCvtMaskToVectorNode(retType, retNode, simdBaseType, simdSize);
	}

It will pass nodeRetType == TYP_MASK to impSpecialIntrinsic for any intrinsics that have HW_Flag_ReturnsPerElementMask. Then we insert the CvtMaskToVector conversion on anything that reaches the end that has TYP_MASK.

[snickolls-arm]

For now I'd rather just get this PR completed so I can start contributing the Vector intrinsic implementations I've done using SVE.

@snickolls-arm, as an FYI this is going to change a bit with #130049.

It is effectively merging most of the tables for the xplat intrinsics into a new combined table and the import logic itself into a combined handler as well. The handling between Arm64 AdvSimd and xarch was already nearly identical and the same will again be true for Arm64 SVE and the WASM support we're onboarding, with most of the differences (if any) being in the gtNewSimd* helper APIs or in the actual codegen handling, which was often special.

That should make it a bit simpler to lightup for SVE, as you'll not need to extend or duplicate the tables again or add a bunch of NI_VectorT_* entries alongside the existing NI_Vector64_* and NI_Vector128_* handlers. Instead, it's purely NI_Vector_* for all cases and differentiated based on simdSize (i.e. SIZE_UNKNOWN for SVE).

Thanks for the heads up. This should reduce the diffs I have for implementing the Vector<T> API quite a bit, I'll hold off from sending too much of that then until #130049 is merged and I've rebased on it.