Skip to content

[Perf] Linux/arm64: 24 Regressions on 5/26/2023 11:50:09 PM #18434

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 549.62 ns 618.84 ns 1.13 0.02 False
IndexOf_Word_NotFound - Duration of single invocation 549.22 ns 618.09 ns 1.13 0.01 False
IndexOf_Word_NotFound - Duration of single invocation 550.05 ns 618.06 ns 1.12 0.01 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
Details

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 618.8398270754278 > 576.9870239637595.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -6.409886065609311 (T) = (0 -608.9679727554028) / Math.Sqrt((0.13662878007732648 / (14)) + (599.5335616023624 / (7))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (7) - 2, .025) and -0.10793233329838221 = (549.6436510183505 - 608.9679727554028) / 549.6436510183505 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 618.0930262877821 > 576.7585966523885.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -6.332547979543937 (T) = (0 -608.9057207129692) / Math.Sqrt((0.14424305811183716 / (15)) + (614.0909586379906 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.10792716750537114 = (549.5900259256142 - 608.9057207129692) / 549.5900259256142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 618.0569305998685 > 576.7140310541726.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -485.5772504677802 (T) = (0 -618.0269268746015) / Math.Sqrt((0.19934395764946247 / (15)) + (0.04646808123516565 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.12474905966981621 = (549.47983424501 - 618.0269268746015) / 549.47983424501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSingle - Duration of single invocation 68.82 ns 74.05 ns 1.08 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.05134980623163 > 72.44162946001455.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -19.90995688313724 (T) = (0 -73.25416052556558) / Math.Sqrt((0.2151562089186892 / (15)) + (0.22685198382860605 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.06243630368382694 = (68.94922572917413 - 73.25416052556558) / 68.94922572917413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseExactD - Duration of single invocation 32.98 ns 39.42 ns 1.20 0.01 False
ctor_str - Duration of single invocation 30.19 ns 39.37 ns 1.30 0.02 False
Parse - Duration of single invocation 30.03 ns 38.79 ns 1.29 0.02 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.ParseExactD


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.41822678843227 > 35.39273124073686.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -33.21698319512984 (T) = (0 -39.30841978303204) / Math.Sqrt((0.17784349255680962 / (14)) + (0.12691007120859674 / (7))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (7) - 2, .025) and -0.1742363835002088 = (33.47572970432069 - 39.30841978303204) / 33.47572970432069 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.ctor_str


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.371466805745435 > 31.649773542637373.
IsChangePoint: Marked as a change because one of 4/1/2023 5:53:03 AM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -61.89821868942194 (T) = (0 -40.07958785268969) / Math.Sqrt((0.1445991140994894 / (15)) + (0.11385900816731427 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.33079827513379323 = (30.11695205921442 - 40.07958785268969) / 30.11695205921442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.Parse


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.790000150963735 > 31.692458029349176.
IsChangePoint: Marked as a change because one of 4/1/2023 5:53:03 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -29.456166842611516 (T) = (0 -39.78789837419) / Math.Sqrt((0.13814406073325358 / (15)) + (0.6903756359478791 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.3211975731848152 = (30.115025323789546 - 39.78789837419) / 30.115025323789546 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same_Upper - Duration of single invocation 1.80 μs 2.88 μs 1.60 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'
Details

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8760772323065638 > 1.887547782669673.
IsChangePoint: Marked as a change because one of 3/28/2023 2:42:21 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -4.456653977032153 (T) = (0 -2449.084541832258) / Math.Sqrt((5.025172177723559 / (15)) + (149282.29850869643 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.3619230655224062 = (1798.2546913491317 - 2449.084541832258) / 1798.2546913491317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqOrderByExtension - Duration of single invocation 23.75 μs 32.17 μs 1.35 0.40 False
LinqQuery - Duration of single invocation 23.36 μs 32.41 μs 1.39 0.39 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.17458918571428 > 24.633268243846153.
IsChangePoint: Marked as a change because one of 4/21/2023 8:40:59 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -22.822822738435555 (T) = (0 -32489.47148744113) / Math.Sqrt((61655.530210649864 / (15)) + (1054458.587327602 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.3818577422504241 = (23511.44440854701 - 32489.47148744113) / 23511.44440854701 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.LinqQuery(Size: 512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.410423785714286 > 24.5975162925.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -23.228861976844694 (T) = (0 -32677.703257687073) / Math.Sqrt((66081.29533785854 / (15)) + (1059426.3249538257 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.3899294461598066 = (23510.332375482292 - 32677.703257687073) / 23510.332375482292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EscapeDataString - Duration of single invocation 8.16 μs 12.48 μs 1.53 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{")


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.482333918026804 > 9.841629778743025.
IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -21.71495721220194 (T) = (0 -12479.794777774305) / Math.Sqrt((410128.7679794463 / (14)) + (453.4340234956242 / (7))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (7) - 2, .025) and -0.4247953352052378 = (8759.008728770596 - 12479.794777774305) / 8759.008728770596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiCharArray - Duration of single invocation 162.57 μs 171.42 μs 1.05 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 171.42251906702896 > 170.75294991261015.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -53.39570030158307 (T) = (0 -171394.24647986912) / Math.Sqrt((385250.8506773665 / (15)) + (3755.649125757051 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.053125854974705095 = (162748.11378929237 - 171394.24647986912) / 162748.11378929237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InterpolateIntoSpan_Flags - Duration of single invocation 113.48 ns 124.45 ns 1.10 0.01 False
InterpolateIntoSpan_NonFlags - Duration of single invocation 122.64 ns 132.90 ns 1.08 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 124.45267264988873 > 118.92928015818414.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 4/6/2023 1:55:21 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -17.457147283339218 (T) = (0 -125.09055564807245) / Math.Sqrt((0.34145063503974216 / (15)) + (3.10371928669035 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.1053172220059725 = (113.17163358863917 - 125.09055564807245) / 113.17163358863917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.90211723097497 > 127.9759915516894.
IsChangePoint: Marked as a change because one of 3/22/2023 10:32:21 PM, 3/25/2023 3:46:49 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -40.18579079772538 (T) = (0 -133.5654363192309) / Math.Sqrt((0.5945277862708089 / (14)) + (0.2806559947879864 / (7))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (7) - 2, .025) and -0.09463041459406142 = (122.01875129585453 - 133.5654363192309) / 122.01875129585453 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqQuery - Duration of single invocation 44.55 μs 52.28 μs 1.17 0.04 False
LinqOrderByExtension - Duration of single invocation 47.44 μs 51.73 μs 1.09 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.275895799999994 > 48.387110705000005.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -12.645791030538879 (T) = (0 -51833.130848163266) / Math.Sqrt((2865549.5496481233 / (15)) + (96521.39516154786 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.12412044344101852 = (46109.94413507693 - 51833.130848163266) / 46109.94413507693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.73032109333333 > 49.766545841500005.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsRegressionStdDev: Marked as regression because -12.468465196384184 (T) = (0 -52025.528478503395) / Math.Sqrt((2568306.3555916385 / (15)) + (135745.81802449908 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.11686157766408307 = (46581.894765611716 - 52025.528478503395) / 46581.894765611716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions