Add visit budget checks to IsMonotonicallyIncreasing and ComputeDoesOverflow

[Copilot]

IsMonotonicallyIncreasing and ComputeDoesOverflow lacked the m_nVisitBudget guard that ComputeRange already uses, allowing unbounded recursion cost in pathological cases.

Description

Added early-exit budget checks to both functions, consistent with the existing pattern in ComputeRange:

if (IsOverBudget())
{
    return false;
}
m_nVisitBudget--;

• IsMonotonicallyIncreasing: false is the conservative result ("not proven monotonic"), so this is safe to return when over budget.
• ComputeDoesOverflow: false ("doesn't overflow") is slightly optimistic in isolation, but ComputeRange independently returns Unknown when over budget, which already blocks bounds-check elimination — so no unsound optimization can occur.

Both checks are placed before GetSearchPath()->Set(...) to avoid leaving stale entries in the search path on early exit.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

Adds visit-budget guards to additional RangeCheck helpers to prevent pathological recursion cost during JIT range analysis, aligning them with the existing budget mechanism used by ComputeRange.

Changes:

• Add IsOverBudget() early-exit + m_nVisitBudget-- to RangeCheck::IsMonotonicallyIncreasing.
• Add IsOverBudget() early-exit + m_nVisitBudget-- to RangeCheck::ComputeDoesOverflow.

[EgorBo]

PTAL @AndyAyersMS @dotnet/jit-contrib I've checked it on your MC file - it did fix the issue, although, the initial issue didn't reproduce as a hang in VMR for me.

Diffs

[EgorBo]

Actually, no, I was able to repro locally, here is the minimal repro: https://gist.github.com/EgorBo/2962e5e5a94961f6c461df0e077361e1
(Reproduces on main + Linux-x64 + setting max assertion count to 256)

It's difficult to test on CI as it doesn't fail, just never reaches Tier1 (since promotion is asynchronous) and consumes memory. Doesn't repro with TC=0, probably can be added as a benchmark.