Update --show-stdout / --show-stderr help text to reflect LLM-aware default - #8797
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Updates Microsoft.Testing.Platform’s CLI help text for --show-stdout / --show-stderr so it accurately describes the LLM/AI-aware default behavior introduced in #8771, and keeps the help/info acceptance tests and localized resources in sync.
Changes:
- Updated the two terminal option description resource strings to document the conditional default (
All, orFailedin detected LLM/AI agent environments). - Regenerated all 13 localized XLF files for the updated resources.
- Updated help/info acceptance test expectations (including regex escaping where needed for the
--infooutput).
Show a summary per file
| File | Description |
|---|---|
| test/IntegrationTests/MSTest.Acceptance.IntegrationTests/HelpInfoTests.cs | Updates expected --help output text for --show-stdout / --show-stderr. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.cs | Updates expected --help and --info outputs (regex-escaped) for the revised option descriptions. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs | Updates expected --help and --info outputs (with all extensions) for the revised option descriptions. |
| src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx | Updates the canonical English resource strings for the two option descriptions. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlf | Regenerated localization file reflecting updated source strings (marked for review). |
Copilot's findings
- Files reviewed: 17/17 changed files
- Comments generated: 0
Evangelink
commented
Jun 3, 2026
Evangelink
left a comment
Member
Author
There was a problem hiding this comment.
Expert Review — PR #8797: Update --show-stdout / --show-stderr help text to reflect LLM-aware default
✅ 21/21 dimensions clean — no findings.
Dimension Assessment
| # | Dimension | Verdict |
|---|---|---|
| 1 | Algorithmic Correctness | ✅ N/A — no logic changed |
| 2 | Threading & Concurrency | ✅ N/A — no shared state touched |
| 3 | Security & IPC Contract Safety | ✅ N/A |
| 4 | Public API & Binary Compatibility | ✅ N/A — no API surface changed |
| 5 | Performance & Allocations | ✅ N/A |
| 6 | Cross-TFM Compatibility | ✅ N/A |
| 7 | Resource & IDisposable Management | ✅ N/A |
| 8 | Defensive Coding at Boundaries | ✅ N/A |
| 9 | Localization & Resources | ✅ LGTM — .resx updated; all 13 XLF files regenerated via dotnet msbuild /t:UpdateXlf, not manually edited; <source> blocks updated and stale <target> state correctly demoted to needs-review-translation in each locale (cs, de, es, fr, it, ja, ko, pl, pt-BR, ru, tr, zh-Hans, zh-Hant). |
| 10 | Test Isolation | ✅ N/A |
| 11 | Assertion Quality | ✅ LGTM — test literals match the new string exactly; the one location that uses regex matching (HelpInfoTests.cs ~line 287) correctly escapes the parentheses as \( and \). |
| 12 | Flakiness Patterns | ✅ N/A |
| 13 | Test Completeness & Coverage | ✅ LGTM — all three help-info test files updated: HelpInfoTests.cs (MTP), HelpInfoAllExtensionsTests.cs (MTP), HelpInfoTests.cs (MSTest). Both the short-form help block and the verbose diagnostic block (--list-tests style) are covered. |
| 14 | Data-Driven Test Coverage | ✅ N/A |
| 15 | Code Structure & Simplification | ✅ N/A |
| 16 | Naming & Conventions | ✅ N/A |
| 17 | Documentation Accuracy | ✅ LGTM — new text accurately reflects TerminalOutputDevice.GetShowOutputMode behavior: default is All, flips to Failed when LLM/AI agent environment detected. Wording is concise and correct. |
| 18 | Analyzer & Code Fix Quality | ✅ N/A |
| 19 | IPC Wire Compatibility | ✅ N/A |
| 20 | Build Infrastructure & Dependencies | ✅ N/A |
| 21 | Scope & PR Discipline | ✅ LGTM — single concern: string update + XLF regeneration + test expectations. No mixed refactoring. References #8771 and fixes #8772. |
Notes
- The
needs-review-translationstate in the XLF files is the correct XLIFF lifecycle state for "source changed, translation pending re-review." No action needed beyond what the localization pipeline will handle automatically. - The parentheses-escaping difference between test files is intentional: files that use
Containsor verbatim string matching use literal(); files that use regex matching escape to\(\). Both are correct for their respective assertion styles.
Generated by Expert Code Review (on open) for issue #8797 · sonnet46 1.8M
Evangelink
enabled auto-merge (squash)
June 3, 2026 13:44
YuliiaKovalova
approved these changes
Jun 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up from #8771 / fixes #8772.
When an LLM/AI agent environment is detected, the defaults for
--show-stdoutand--show-stderrflip fromAlltoFailed(seeTerminalOutputDevice.GetShowOutputMode). The option descriptions inPlatformResources.resxstill hard-codedDefault is 'All'.though, so agents invoking--helpsaw misleading guidance about the very default that was changed for their benefit.Changes
Updated
TerminalShowStdoutOptionDescriptionandTerminalShowStderrOptionDescriptioninPlatformResources.resxto:Regenerated the 13 XLF localization files via
dotnet msbuild .../Microsoft.Testing.Platform.csproj /t:UpdateXlf.Updated the wildcard / regex expectations in:
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.cstest/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cstest/IntegrationTests/MSTest.Acceptance.IntegrationTests/HelpInfoTests.csValidation
Locally green on
net8.0/net10.0/net462:HelpInfoTests(21 tests) — passedHelpInfoAllExtensionsTests(9 tests) — passedHelpInfoTests(6 tests) — passed