Skip to content

[Perf] Changes at 5/31/2022 9:21:33 PM #5804

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 17.68 ns 21.22 ns 1.20 0.38 False
Fill - Duration of single invocation 17.43 ns 28.55 ns 1.64 0.36 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfValue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.22258433170919 > 18.580044945644673.
IsChangePoint: Marked as a change because one of 4/16/2022 3:23:01 AM, 5/11/2022 3:53:15 AM, 5/20/2022 10:18:01 PM, 5/25/2022 3:58:08 AM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -8.823372771280228 (T) = (0 -19.438626196362527) / Math.Sqrt((0.5439745738679618 / (29)) + (1.1324357204972006 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.10748898351398163 = (17.551981541780385 - 19.438626196362527) / 17.551981541780385 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.Fill(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.551839295351158 > 18.45516988974589.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/22/2022 6:17:37 AM, 5/8/2022 7:59:02 PM, 5/15/2022 3:27:46 PM, 5/20/2022 9:09:19 PM, 5/23/2022 3:43:39 PM, 6/2/2022 10:27:31 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -20.912874960852633 (T) = (0 -29.36014100439192) / Math.Sqrt((3.0925572347161188 / (44)) + (6.040408149467225 / (27))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (27) - 2, .025) and -0.6292426477719101 = (18.020729474853685 - 29.36014100439192) / 18.020729474853685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Numerics.Tests.Perf_Plane

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromVerticesBenchmark - Duration of single invocation 6.21 ns 7.52 ns 1.21 0.02 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Plane*'
Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Plane.CreateFromVerticesBenchmark


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.52437750457585 > 6.518412376672993.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -1142.7629602526292 (T) = (0 -7.518611178525513) / Math.Sqrt((7.075477802821817E-06 / (29)) + (4.491651811665559E-05 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.21093088662386994 = (6.208951527768642 - 7.518611178525513) / 6.208951527768642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiCharArray - Duration of single invocation 325.93 ns 365.61 ns 1.12 0.01 True
WriteAsciiString - Duration of single invocation 324.91 ns 368.73 ns 1.13 0.01 True

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 365.6135640902402 > 343.41249329321784.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -44.68452617763572 (T) = (0 -366.05725715094513) / Math.Sqrt((11.898172816882738 / (29)) + (12.123107329054168 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.1136509905313598 = (328.70015854454283 - 366.05725715094513) / 328.70015854454283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 368.73120208456834 > 346.33279915096875.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -42.2112753225681 (T) = (0 -371.7923479896483) / Math.Sqrt((20.838040153726418 / (29)) + (8.698431148963131 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.12262789676607054 = (331.1803929518074 - 371.7923479896483) / 331.1803929518074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 13.90 ns 15.06 ns 1.08 0.09 False
ToString - Duration of single invocation 9.56 ns 10.73 ns 1.12 0.09 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 4294967295)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.061278222648081 > 14.541279726858264.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -18.82856590564754 (T) = (0 -15.888250188710039) / Math.Sqrt((0.16509236592627552 / (29)) + (0.07862334835584078 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.1149166650242242 = (14.250616828267455 - 15.888250188710039) / 14.250616828267455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.72594488594447 > 10.00061858553078.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -11.766212956587175 (T) = (0 -10.46563457278499) / Math.Sqrt((0.07266284979706175 / (21)) + (0.013910696763992994 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.07381425881322372 = (9.746224253300174 - 10.46563457278499) / 9.746224253300174 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateBillboardBenchmark - Duration of single invocation 17.42 ns 18.52 ns 1.06 0.04 False
CreateWorldBenchmark - Duration of single invocation 16.92 ns 18.43 ns 1.09 0.11 False
CreateConstrainedBillboardBenchmark - Duration of single invocation 23.41 ns 26.40 ns 1.13 0.02 True
CreateShadowBenchmark - Duration of single invocation 15.90 ns 24.96 ns 1.57 0.04 True
CreateLookAtBenchmark - Duration of single invocation 24.77 ns 26.17 ns 1.06 0.03 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.523112767821978 > 18.017291260327045.
IsChangePoint: Marked as a change because one of 4/16/2022 2:55:52 PM, 4/17/2022 3:50:42 AM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -41.57995134983327 (T) = (0 -18.924540468016183) / Math.Sqrt((0.022933190647709444 / (29)) + (0.04429984432748734 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.10422715723219121 = (17.13826756031941 - 18.924540468016183) / 17.13826756031941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.433946240957816 > 17.919202376114114.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.55094461206433 (T) = (0 -18.454598217930222) / Math.Sqrt((0.024715214376997533 / (29)) + (0.20380934912059345 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.07739118996805477 = (17.12896707321081 - 18.454598217930222) / 17.12896707321081 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.401939150068277 > 24.66576704874909.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -43.675949613329514 (T) = (0 -26.67248640298882) / Math.Sqrt((0.10670627273905242 / (29)) + (0.04526585006906499 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.12732298107780216 = (23.660021884312158 - 26.67248640298882) / 23.660021884312158 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.956635417025655 > 17.03520427338729.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -170.493799608565 (T) = (0 -25.195902997239223) / Math.Sqrt((0.05782127743149311 / (29)) + (0.03112417669100979 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.5476854669403829 = (16.279730950145165 - 25.195902997239223) / 16.279730950145165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.16678058022764 > 25.62595354286628.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -40.6252232765232 (T) = (0 -26.318011385644635) / Math.Sqrt((0.018604493852935677 / (29)) + (0.0605901659154336 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.07581319523067512 = (24.463365482333153 - 26.318011385644635) / 24.463365482333153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 9.00 μs 10.28 μs 1.14 0.02 False
Utf8JsonReaderCommentParsing - Duration of single invocation 8.99 μs 10.29 μs 1.14 0.02 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.276710422333739 > 9.443435014112689.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -28.170432022829196 (T) = (0 -10369.511285482598) / Math.Sqrt((58450.13319641083 / (29)) + (8455.352436008941 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.14666981689662517 = (9043.15360244407 - 10369.511285482598) / 9043.15360244407 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.285677068374673 > 9.43898962667122.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -108.2947909826349 (T) = (0 -10227.37433125835) / Math.Sqrt((80.37916959117628 / (29)) + (5339.284329116493 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.13724548087308094 = (8993.110549365856 - 10227.37433125835) / 8993.110549365856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Base64Encode - Duration of single invocation 56.52 ns 61.59 ns 1.09 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64Tests.Base64Encode(NumberOfBytes: 1000)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.58729461317633 > 60.49645001107261.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -5.722998727056304 (T) = (0 -62.51937092214449) / Math.Sqrt((1.3878531980284703 / (29)) + (33.5424477306782 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.0919662177519276 = (57.253942389221066 - 62.51937092214449) / 57.253942389221066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Max - Duration of single invocation 323.48 ns 345.53 ns 1.07 0.03 False
WhereAny_LastElementMatches - Duration of single invocation 284.19 ns 302.13 ns 1.06 0.06 False
Min - Duration of single invocation 344.25 ns 366.84 ns 1.07 0.03 False
Sum - Duration of single invocation 344.51 ns 367.14 ns 1.07 0.02 False
WhereSingle_LastElementMatches - Duration of single invocation 283.94 ns 302.01 ns 1.06 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 345.5321424072675 > 339.4937772950543.
IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -109.29258419908157 (T) = (0 -344.4852175864091) / Math.Sqrt((0.9354257424443341 / (29)) + (0.18860399588045115 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.06475583373982487 = (323.5344730410603 - 344.4852175864091) / 323.5344730410603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 302.131612318434 > 298.93366687418165.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 5/9/2022 11:56:12 AM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -16.48453870049516 (T) = (0 -323.86694113337893) / Math.Sqrt((90.7700353201703 / (29)) + (36.63251967539187 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.11337033766964205 = (290.88878172491457 - 323.86694113337893) / 290.88878172491457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 366.84358094693306 > 361.4744511391038.
IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/13/2022 5:30:39 PM, 5/9/2022 11:56:12 AM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -91.96529738556421 (T) = (0 -369.16547384827294) / Math.Sqrt((0.11046724237629651 / (42)) + (2.02585826344153 / (29))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (29) - 2, .025) and -0.07189279668776864 = (344.4052194296134 - 369.16547384827294) / 344.4052194296134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 367.1397064031732 > 361.8260573767792.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -13.884795703321224 (T) = (0 -367.5233250185926) / Math.Sqrt((120.06522173910017 / (21)) + (0.4077478568439224 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.09938299338889603 = (334.2996273625135 - 367.5233250185926) / 334.2996273625135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 302.0104438674098 > 297.32895887848696.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -18.307878582568872 (T) = (0 -326.68657721622475) / Math.Sqrt((85.30322528409945 / (29)) + (50.28257879583233 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.1286794013661375 = (289.44142758413767 - 326.68657721622475) / 289.44142758413767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6010dc0604a9f2708dba8d21f57d97b613dfcabe
Compare 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 44.71 ns 48.35 ns 1.08 0.30 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.3530238890695 > 47.0540391985978.
IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 4/21/2022 5:54:30 PM, 5/23/2022 7:50:08 PM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -7.0936861988147335 (T) = (0 -49.251821969768855) / Math.Sqrt((4.047350330405335 / (29)) + (3.293741472024483 / (42))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (42) - 2, .025) and -0.07209363265105986 = (45.93985121241655 - 49.251821969768855) / 45.93985121241655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions