Add MTP artifact post-processing to dotnet test - #55419
Merged
Merged
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a56f3957-e35c-47df-bad9-c5f342ee302a
|
Azure Pipelines: Successfully started running 1 pipeline(s). 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds SDK-side orchestration for Microsoft.Testing.Platform (MTP) artifact post-processing during dotnet test, enabling consolidation of multi-module artifacts (e.g., TRX) before the final summary is printed.
Changes:
- Introduces artifact post-processing planning/execution (capability + artifact metadata capture, deterministic planning, dispatcher relaunch, and summary artifact replacement).
- Updates MTP execution pipeline (new
IsArtifactPostProcessingmode, handshake/host-type validation, artifact collection hooks, and reporter artifact removal support). - Adds unit and end-to-end tests for the planner/manager/handler and TRX merge behavior; bumps TestFX/TestPlatform/MSTest versions and adds new localized strings.
Show a summary per file
| File | Description |
|---|---|
| test/dotnet.Tests/CommandTests/Test/TestApplicationHandlerTests.cs | Adds handler tests for post-processing capability/artifact capture and failure-path behavior. |
| test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsArtifactPostProcessingMTP.cs | End-to-end test validating TRX consolidation behavior and output reporting. |
| test/dotnet.Tests/CommandTests/Test/ArtifactPostProcessingPlannerTests.cs | Unit tests for the planner’s grouping, coverage, and set-cover selection logic. |
| test/dotnet.Tests/CommandTests/Test/ArtifactPostProcessingManagerTests.cs | Unit tests for applying merged outputs and consuming originals in the reporter. |
| src/Cli/dotnet/Commands/Test/MTP/TestApplicationHandler.cs | Adds post-processing handshake host-type validation and routes artifact messages into manager/invocation. |
| src/Cli/dotnet/Commands/Test/MTP/TestApplicationActionQueue.cs | Threads the post-processing manager through test application execution. |
| src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs | Adds post-processing launch mode/args, timeout behavior, and launch-settings suppression for tool runs. |
| src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs | Makes artifact tracking thread-safe and adds RemoveArtifacts for post-processed consolidation. |
| src/Cli/dotnet/Commands/Test/MTP/Options.cs | Extends TestOptions with IsArtifactPostProcessing. |
| src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs | Executes post-processing after test runs complete (when applicable). |
| src/Cli/dotnet/Commands/Test/MTP/ArtifactPostProcessingPlanner.cs | New planner: groups artifacts and computes a deterministic minimal job set. |
| src/Cli/dotnet/Commands/Test/MTP/ArtifactPostProcessingManager.cs | New manager: records capabilities/artifacts, runs dispatcher tool jobs, and applies merged outputs. |
| src/Cli/dotnet/Commands/Test/CliConstants.cs | Adds protocol constants for post-processing tool name/options, handshake host type, and capability properties. |
| src/Cli/dotnet/Commands/CliCommandStrings.resx | Adds new user-facing strings for post-processing failures and host-type mismatch. |
| src/Cli/dotnet/Commands/xlf/CliCommandStrings.*.xlf | Updates localized string entries corresponding to the new resx strings. |
| global.json | Bumps MSTest.Sdk version to match the updated TestFX build. |
| eng/Version.Details.xml | Bumps Microsoft.Testing.Platform/MSTest/MSTest.Sdk dependency versions and SHA. |
| eng/Version.Details.props | Updates the corresponding version properties for the TestFX packages. |
Copilot's findings
- Files reviewed: 30/30 changed files
- Comments generated: 0
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a56f3957-e35c-47df-bad9-c5f342ee302a
Evangelink
enabled auto-merge
July 23, 2026 07:37
JanKrivanek
approved these changes
Jul 23, 2026
Member
Author
|
/backport to release/11.0.1xx-preview7 |
Contributor
|
Started backporting to |
This was referenced Jul 27, 2026
Evangelink
added a commit
to microsoft/testfx
that referenced
this pull request
Jul 28, 2026
…orts Bumps global.json (tools.dotnet + sdk.version) to 11.0.100-preview.7.26376.106. That build carries the SDK-side artifact post-processing orchestration (dotnet/sdk#55419, backported to release/11.0.1xx-preview7 as dotnet/sdk#55453), which flowed into the VMR via '[release/11.0.1xx-preview7] Source code updates from dotnet/sdk' (dotnet/dotnet#7944), so RFC 018 phase 2 now actually runs for this repo's multi-module 'dotnet test' invocations. Enabling it surfaced a problem that is not specific to this repo. The SDK points the processor's output directory at the run's --results-directory, which already holds the per-module reports the merge consumed, so TrxArtifactPostProcessor was writing the merged report as a sibling of its own inputs. Collecting a results directory with a non-recursive '*.trx' glob is near-universal (Arcade's 'Publish TRX Test Results' step, the PublishTestResults@2 defaults, typical ReportGenerator setups), and every such consumer would ingest both the merged report and the reports it summarizes, double-counting every test. TrxArtifactPostProcessor now writes to '<outputDirectory>/merged/', which keeps the merged report out of those globs by construction instead of asking each consumer to special-case the merged file name. Nesting also keeps the report co-located with its attachments, which a CI-side move could not. MergeToFileAsync derives the attachment deployment root from the output path and records it in the TRX as a relative runDeploymentRoot, so the report and its attachment tree have to stay in the same directory for those references to resolve; writing the report into the subdirectory moves both together. Adds a regression test covering that, and asserting nothing of the merged report is left in the results-directory root. Verified end to end on this SDK: two MTP modules (23 + 2 tests) run through 'dotnet test' with a shared --results-directory produce 'merged/merged-<runId>.trx' with total=25, reported back over the existing pipe as an out-of-process file artifact, with the per-module reports left intact at the root and nothing else matching a non-recursive '*.trx' glob there. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4a6c06b2-7dcb-407e-bb9e-ca0c8302fa93
Evangelink
added a commit
to microsoft/testfx
that referenced
this pull request
Jul 28, 2026
…orts Bumps global.json (tools.dotnet + sdk.version) to 11.0.100-preview.7.26376.106. That build carries the SDK-side artifact post-processing orchestration (dotnet/sdk#55419, backported to release/11.0.1xx-preview7 as dotnet/sdk#55453), which flowed into the VMR via '[release/11.0.1xx-preview7] Source code updates from dotnet/sdk' (dotnet/dotnet#7944), so RFC 018 phase 2 now actually runs for this repo's multi-module 'dotnet test' invocations. Enabling it surfaced a problem that is not specific to this repo. The SDK points the processor's output directory at the run's --results-directory, which already holds the per-module reports the merge consumed, so TrxArtifactPostProcessor was writing the merged report as a sibling of its own inputs. Collecting a results directory with a non-recursive '*.trx' glob is near-universal (Arcade's 'Publish TRX Test Results' step, the PublishTestResults@2 defaults, typical ReportGenerator setups), and every such consumer would ingest both the merged report and the reports it summarizes, double-counting every test. TrxArtifactPostProcessor now writes to '<outputDirectory>/merged/', which keeps the merged report out of those globs by construction instead of asking each consumer to special-case the merged file name. Nesting also keeps the report co-located with its attachments, which a CI-side move could not. MergeToFileAsync derives the attachment deployment root from the output path and records it in the TRX as a relative runDeploymentRoot, so the report and its attachment tree have to stay in the same directory for those references to resolve; writing the report into the subdirectory moves both together. Adds a regression test covering that, and asserting nothing of the merged report is left in the results-directory root. Verified end to end on this SDK: two MTP modules (23 + 2 tests) run through 'dotnet test' with a shared --results-directory produce 'merged/merged-<runId>.trx' with total=25, reported back over the existing pipe as an out-of-process file artifact, with the per-module reports left intact at the root and nothing else matching a non-recursive '*.trx' glob there. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4a6c06b2-7dcb-407e-bb9e-ca0c8302fa93
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multi-module Microsoft.Testing.Platform runs currently report one artifact per test application, leaving users with many TRX or coverage files. This adds SDK-side orchestration for the TestFX artifact post-processing protocol so compatible artifacts are consolidated before the final
dotnet testsummary.The implementation:
Merged report naming
TRX post-processing writes
merged-{runId:N}.trx. TestFX derivesrunIdby hashing the ordered normalized input paths together with their execution IDs. This keeps the name deterministic within an invocation while giving separatedotnet testinvocations distinct names, even when they share a results directory. The end-to-end test runs the same solution twice into one directory and verifies that both merged reports are preserved without collision.Adds planner, protocol, reporter, failure-path, mixed-version, naming-collision, and end-to-end TRX merge coverage.
Relates to #47613