Allow dotnet test --list-tests json for Microsoft.Testing.Platform - #54490
Allow dotnet test --list-tests json for Microsoft.Testing.Platform#54490Evangelink wants to merge 2 commits into
dotnet test --list-tests json for Microsoft.Testing.Platform#54490Conversation
Plumbs the optional `text`/`json` value of `--list-tests` through the SDK CLI to the underlying Microsoft.Testing.Platform test host. The MTP-side support for the JSON discovery output was added in microsoft/testfx#8280; this change exposes it via `dotnet test`. Fixes #49754 - Switch the MTP `--list-tests` option arity from `Zero` to `ZeroOrOne`, restrict accepted values to `text`/`json` via `AcceptOnlyFromAmong`, and add a dedicated description resource string. - Carry the optional value through `TestOptions` and forward it to the test host in `TestApplication.GetArguments`. - Update the MTP help snapshot. - Add tests covering `--list-tests text`, `--list-tests json`, and rejection of invalid values; existing argument-escaping test still asserts the bare `--list-tests` token (no value) is forwarded as before. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds SDK CLI plumbing for Microsoft.Testing.Platform’s dotnet test --list-tests optional output format, enabling text or json to be forwarded to the test host.
Changes:
- Updates the MTP
--list-testsoption to accepttext|jsonand forwards the selected value. - Adds test coverage for forwarding valid values and rejecting invalid values.
- Updates help text, snapshots, and localization resources for the new option description.
Show a summary per file
| File | Description |
|---|---|
src/Cli/Microsoft.DotNet.Cli.Definitions/Commands/Test/TestCommandDefinition.MicrosoftTestingPlatform.cs |
Changes --list-tests option arity and accepted values. |
src/Cli/Microsoft.DotNet.Cli.Definitions/CommandDefinitionStrings.resx |
Adds MTP-specific list-tests description. |
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs |
Reads the optional list-tests value from parsing. |
src/Cli/dotnet/Commands/Test/MTP/Options.cs |
Adds storage for the list-tests argument. |
src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs |
Forwards the list-tests value to the test application. |
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndDiscoversTests.cs |
Adds valid/invalid list-tests argument tests. |
test/dotnet.Tests/CommandTests/Test/snapshots/MTPHelpSnapshotTests.VerifyMTPHelpOutput.verified.txt |
Updates expected help output. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.cs.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.de.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.es.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.fr.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.it.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ja.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ko.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pl.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pt-BR.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ru.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.tr.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hans.xlf |
Adds localized resource entry placeholder. |
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hant.xlf |
Adds localized resource entry placeholder. |
Copilot's findings
- Files reviewed: 20/20 changed files
- Comments generated: 1
|
Moving back to draft so I can double check something. |
…xt/json as format Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Closing in favor of the protocol-based approach. The MTP IPC wire contract already carries the full discovered-test payload the SDK needs (Uid, DisplayName, FilePath, LineNumber, Namespace, TypeName, MethodName, ParameterTypeFullNames, Traits) — vendored from microsoft/testfx and already on \main\ (commit cd590b6). In --list-tests\ mode the SDK already handshakes as \Discover\ and receives \DiscoveredTestMessages\ over the pipe. Rather than forwarding --list-tests json\ to the test host (which is blocked today because MTP's \TerminalOutputDevice\ early-returns under _isServerMode, always set via --server dotnettestcli), the SDK will render the JSON itself from the messages it already receives. This avoids the testfx server-mode change, avoids stdout interleaving, and matches the preference for the SDK acting as the discovery client. A follow-up PR will implement the SDK-side rendering. |
Fixes #49754
The MTP side added support for
--list-tests jsonin microsoft/testfx#8280. This PR plumbs the optionaltext/jsonvalue through the SDK CLI so users can do:Changes
TestCommandDefinition.MicrosoftTestingPlatform.ListTestsOption: arityZero→ZeroOrOne, restrict accepted values totext/jsonviaAcceptOnlyFromAmong, dedicatedHelpNameand a new resource stringCmdMTPListTestsDescription.TestOptions: newListTestsArgumentfield.MicrosoftTestingPlatformTestCommand: read the option value and pass it through.TestApplication.GetArguments: forward the value to the test host (escaped) when present; absent value still emits a bare--list-teststoken (no behavioral change for users today).DiscoverTestProjectForwardsListTestsArgument(Debug/Release × text/json) asserts the value is forwarded as a separate arg.DiscoverTestProjectRejectsInvalidListTestsArgumentassertsfoo/JSON/TEXTare rejected (validator is case-sensitive, matching MTP's lowercase keys).DiscoverTestProjectWithCustomRunArgumentsAndTestEscapingcontinues to assert the bare--list-teststoken forwarding when no value is provided.MTPHelpSnapshotTests.VerifyMTPHelpOutput.verified.txtupdated for the new help line.Follow-up
The full end-to-end JSON output also requires a fix on the MTP side:
TerminalOutputDevice.ConsumeAsyncandDisplayAfterSessionEndRunAsynccurrently early-return when_isServerModeis true (which the SDK always sets via--server dotnettestcli), so today MTP buffers/emits no JSON when invoked throughdotnet test. I'll file a follow-up issue in microsoft/testfx to address that.