Skip to content

Disambiguate TRX filenames when running multiple test modules#54555

Closed
Evangelink wants to merge 5 commits into
mainfrom
dev/amauryleve/fix-testfx-7345-trx-filename
Closed

Disambiguate TRX filenames when running multiple test modules#54555
Evangelink wants to merge 5 commits into
mainfrom
dev/amauryleve/fix-testfx-7345-trx-filename

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Fixes microsoft/testfx#7345.

When dotnet test orchestrates more than one test module (via --test-modules glob or a multi-project / multi-TFM solution), the same --report-trx-filename value was forwarded verbatim to every module, so each module's TRX overwrote the previous one.

Approach

The SDK now disambiguates the file name itself when more than one module is being run, leaving the single-module behavior untouched. The substitution is done by the SDK rather than by passing MTP's placeholder tokens, so the fix works against older versions of Microsoft.Testing.Platform that don't yet support the {asm}/{tfm} placeholders introduced in 2.3.0-preview.26274.1 (microsoft/testfx#8223).

Behavior

  • Single module: unchanged (no rewrite).
  • Multi-module + --report-trx-filename foo.trx → rewritten to foo_<asm>_<tfm>.trx per module. No timestamp is added here so a user who explicitly chose a stable file name keeps the same semantics they get today with MTP.
  • Multi-module + --report-trx only → SDK injects --report-trx-filename <asm>_<tfm>_<yyyy-MM-dd_HH_mm_ss>.trx. The UTC timestamp mirrors MTP's natural default (<user>_<machine>_<time>.trx) so re-runs don't trip the Warning: Trx file '...' already exists and will be overwritten. message.
  • User-supplied {asm} / {pname} / {pid} placeholder → left alone (already unique per process). {tfm} alone is not considered unique — two modules can share a TFM, which is exactly the original bug.
  • Preserves directory portion of paths (subdir/foo.trxsubdir/foo_<asm>_<tfm>.trx).
  • Handles both --opt value and --opt=value forms.
  • Sanitizes invalid filename chars.
  • For --test-modules (where TargetFramework is null), infers TFM by walking parent directories of TargetPath and parsing each segment as a TFM folder name (with a starts-with "net" pre-filter so RIDs like win-x64 aren't mistaken for frameworks).

Testing

  • 15 new unit tests in TrxReportArgumentsRewriterTests (incl. theory cases → 17 executed) covering: single-module pass-through, multi-module rewrite, equals-form args, {asm}/{pname}/{pid} short-circuit, {tfm} not short-circuiting, directory preservation, TFM inference from path (including RID layouts), no-extension case, unrelated arg preservation, and the timestamp-disambiguated injected default + re-run uniqueness.
  • 17 nearby existing tests (TestApplicationHandlerTests, TestModulesFilterHandlerTests, TestCommandValidationTests) still pass.
  • Built clean (0 warnings / 0 errors).

When `dotnet test` orchestrates more than one test module (via --test-modules
glob or a multi-project/multi-TFM solution), the same --report-trx-filename was
forwarded verbatim to each module, so every module's TRX overwrote the others.

The SDK now substitutes per-module assembly name and target framework into the
file name itself when more than one module is being run, leaving the
single-module behavior untouched. The substitution is done by the SDK rather
than by passing MTP's placeholder tokens, so the fix works against older
versions of Microsoft.Testing.Platform that don't yet support the
{asm}/{tfm} placeholders introduced in 2.3.0-preview.26274.1
(microsoft/testfx#8223).

When only `--report-trx` is passed (no filename), the SDK also includes a
UTC timestamp in the injected file name so re-running `dotnet test` doesn't
trip MTP's ""Trx file '...' already exists and will be overwritten."" warning
on every module. This mirrors MTP's natural default
`<user>_<machine>_<time>.trx`, which never collides across runs.

Fixes microsoft/testfx#7345

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 2, 2026 13:11
@Evangelink Evangelink requested a review from a team as a code owner June 2, 2026 13:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Disambiguates the TRX file name passed to Microsoft.Testing.Platform's TRX report extension when the SDK orchestrates multiple test modules, so each module writes a distinct file instead of overwriting siblings. Substitution is performed by the SDK so the fix works against MTP versions that predate the {asm}/{tfm} placeholders.

Changes:

  • New TrxReportArgumentsRewriter rewrites --report-trx[-filename] per module (preserving directory, handling = form, sanitizing, inferring TFM from TargetPath for --test-modules), with timestamp injection when only --report-trx was passed.
  • Plumbs an isMultiTestModule flag from MicrosoftTestingPlatformTestCommand through TestApplicationActionQueue to TestApplication, where the rewriter is invoked when assembling the child process arg list.
  • Adds TotalTestModuleCount to ITestHandler (and both MSBuildHandler / TestModulesFilterHandler) so the command knows whether it is running >1 module.
Show a summary per file
File Description
src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs New rewriter that produces a unique TRX filename per module.
src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs Stores _isMultiTestModule and routes test arguments through the rewriter.
src/Cli/dotnet/Commands/Test/MTP/TestApplicationActionQueue.cs Threads isMultiTestModule through to TestApplication construction.
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs Computes isMultiTestModule from testHandler.TotalTestModuleCount.
src/Cli/dotnet/Commands/Test/MTP/ITestHandler.cs Adds TotalTestModuleCount to the handler contract.
src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs Implements TotalTestModuleCount by summing per-group module counts.
src/Cli/dotnet/Commands/Test/MTP/TestModulesFilterHandler.cs Implements TotalTestModuleCount from the resolved test-module path list.
test/dotnet.Tests/CommandTests/Test/TrxReportArgumentsRewriterTests.cs New unit tests covering pass-through, rewrite, equals form, placeholder short-circuit, directory preservation, TFM inference, no-extension, and timestamp uniqueness.

Copilot's findings

  • Files reviewed: 8/8 changed files
  • Comments generated: 0

Adds end-to-end coverage for the fix to microsoft/testfx#7345:

* MultipleTestModulesWithExplicitTrxFilename_ProducesOneTrxPerModule -
  verifies that `dotnet test --report-trx --report-trx-filename results.trx`
  against a multi-module solution writes results_<asm>_<tfm>.trx for each
  module instead of having modules race to overwrite a shared file.

* MultipleTestModulesWithDefaultTrxFilename_ProducesUniqueFilesWithoutOverwriteWarning -
  verifies that `dotnet test --report-trx` (no explicit filename)
  produces <asm>_<tfm>_<timestamp>.trx files and that MTP's
  "will be overwritten" warning is not emitted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs Outdated
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
var testApp = new TestApplication(module, buildOptions, testOptions, output, onHelpRequested);
BuildOptions buildOptionsForModule = buildOptions with
{
TestApplicationArguments = TrxReportArgumentsRewriter.RewriteIfNeeded(buildOptions.TestApplicationArguments, module, isMultiTestModule)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "test module" naming here is confusing. Looking at the name I would expect this to apply only to --test-modules, but it's not.

Comment on lines +16 to +18
private const string ReportTrxOption = "--report-trx";
private const string ReportTrxFilenameOption = "--report-trx-filename";
private const string TrxExtension = ".trx";

@Youssef1313 Youssef1313 Jun 7, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced SDK should be concerned about any of that.

Again, I think this is purely an MTP feature, the feature was provided, and users are expected to update to latest MTP.

I wouldn't complicate the implementation in SDK and introduce additional maintenance and potential risks just because "people might be on older MTP version that doesn't support placeholders".

This even violates the initial designs around MTP that we think shipping via NuGet out of SDK release cycle helps us getting fixes to the users faster.

Move TRX argument rewriting before TestApplication creation, avoid special-casing platform placeholders, preserve malformed filename arguments, and use a higher-precision timestamp for injected TRX names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink Evangelink force-pushed the dev/amauryleve/fix-testfx-7345-trx-filename branch from e7a605c to d1e8212 Compare June 7, 2026 08:27
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
Comment thread src/Cli/dotnet/Commands/Test/MTP/TrxReportArgumentsRewriter.cs Outdated
Per review feedback on PR #54555, restrict the SDK's TRX filename

handling to the minimum: only inject --report-trx-filename when running

multiple test modules with --report-trx and no explicit filename.

If the user supplies --report-trx-filename, the SDK passes it through

untouched and leaves any overwrite warning to MTP. Removed placeholder

expansion, TFM-from-path inference, and equals-form value extraction.

Tests updated to match the simplified contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
Member Author

I'll fix by doing a change in the default name directly in MTP

@Evangelink Evangelink closed this Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Using TrxReport with --test-modules should include test assembly name and target framework in trx file name

3 participants