Skip to content

Add MSTEST0067 analyzer to flag Thread.Sleep/Task.Wait/Task<T>.Result in tests#8646

Merged
Evangelink merged 5 commits into
mainfrom
dev/amauryleve/mstest0066-avoid-thread-sleep
May 28, 2026
Merged

Add MSTEST0067 analyzer to flag Thread.Sleep/Task.Wait/Task<T>.Result in tests#8646
Evangelink merged 5 commits into
mainfrom
dev/amauryleve/mstest0066-avoid-thread-sleep

Conversation

@Evangelink

@Evangelink Evangelink commented May 28, 2026

Copy link
Copy Markdown
Member

Fixes #6950.

Adds a new MSTEST0067 AvoidThreadSleepAndTaskWaitInTests analyzer that reports an Info level diagnostic when one of the following blocking APIs is invoked inside a test method or fixture method:

  • Thread.Sleep(...)
  • Task.Wait(...) (including Task<T>.Wait)
  • Task.WaitAll(...) / Task.WaitAny(...)
  • Task<T>.Result

The analyzer covers methods marked with any of: [TestMethod] (and types deriving from TestMethodAttribute such as [DataTestMethod]), [TestInitialize], [TestCleanup], [ClassInitialize], [ClassCleanup], [AssemblyInitialize], [AssemblyCleanup], [GlobalTestInitialize], [GlobalTestCleanup]. The walk also looks through local functions and lambda bodies declared inside such methods so a Thread.Sleep buried in a callback still surfaces.

Notes on scope

  • The issue text mentioned Thread.Wait which does not exist on System.Threading.Thread. I interpreted that as Task.Wait, which is the actual flakiness/deadlock source.
  • Scope was extended to Task<T>.Result for the same reason — it is a sync-over-async blocking call that is just as problematic as Task.Wait in tests.
  • SemaphoreSlim.Wait is intentionally not flagged — it is a synchronization primitive, not a sync-over-async pattern.
  • Severity is Info (the issue suggested Info or Warning). Conservative choice to avoid breaking existing test suites; users can elevate via .editorconfig.

Note on rule ID

Originally authored as MSTEST0066, this analyzer was renumbered to MSTEST0067 after rebasing on main because MSTEST0066 was concurrently assigned to IgnoreShouldHaveJustificationAnalyzer. The branch name still uses the old mstest0066 slug for historical reasons.

Files

  • src/Analyzers/MSTest.Analyzers/AvoidThreadSleepAndTaskWaitInTestsAnalyzer.cs — new analyzer (C# + VB).
  • src/Analyzers/MSTest.Analyzers/Helpers/DiagnosticIds.cs — new MSTEST0067 constant.
  • src/Analyzers/MSTest.Analyzers/Helpers/WellKnownTypeNames.cs — added System.Threading.Thread.
  • src/Analyzers/MSTest.Analyzers/Resources.resx + auto-generated xlf files.
  • src/Analyzers/MSTest.Analyzers/AnalyzerReleases.Unshipped.md — registers the new rule.
  • test/UnitTests/MSTest.Analyzers.UnitTests/AvoidThreadSleepAndTaskWaitInTestsAnalyzerTests.cs — unit tests covering positive and negative cases (Thread.Sleep variants, Task.Wait overloads, Task.WaitAll/WaitAny, Task.Result, await, helper methods, non-test classes, local functions, lambdas, SemaphoreSlim).

All analyzer unit tests pass locally.

… in tests

Fixes #6950.

Adds a new MSTEST0066 'AvoidThreadSleepAndTaskWaitInTests' analyzer that reports an Info-level diagnostic when one of the following blocking APIs is invoked inside a test method or fixture method (TestInitialize/TestCleanup/ClassInitialize/ClassCleanup/AssemblyInitialize/AssemblyCleanup/GlobalTestInitialize/GlobalTestCleanup):

- Thread.Sleep(...)
- Task.Wait(...) (including Task<T>.Wait)
- Task<T>.Result

The walk also looks through local functions and lambda bodies declared inside such methods so a Thread.Sleep buried in a callback still surfaces.

The issue text mentioned 'Thread.Wait' which does not exist on System.Threading.Thread; this is interpreted as Task.Wait, which is the actual flakiness/deadlock source. Scope was extended to Task<T>.Result for the same reason.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 28, 2026 08:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds MSTEST0066, a new MSTest analyzer that flags blocking waits in test and fixture methods to reduce test flakiness and sync-over-async deadlock risk.

Changes:

  • Adds analyzer logic for Thread.Sleep, Task.Wait, and Task<TResult>.Result in MSTest test/fixture methods, including lambdas/local functions.
  • Registers MSTEST0066 in analyzer IDs and release tracking.
  • Adds analyzer resources, localized XLF placeholders, and unit coverage for key C# scenarios.
Show a summary per file
File Description
src/Analyzers/MSTest.Analyzers/AvoidThreadSleepAndTaskWaitInTestsAnalyzer.cs Implements MSTEST0066 analyzer logic.
src/Analyzers/MSTest.Analyzers/AnalyzerReleases.Unshipped.md Registers MSTEST0066 as an unshipped rule.
src/Analyzers/MSTest.Analyzers/Helpers/DiagnosticIds.cs Adds MSTEST0066 diagnostic ID constant.
src/Analyzers/MSTest.Analyzers/Helpers/WellKnownTypeNames.cs Adds System.Threading.Thread metadata name.
src/Analyzers/MSTest.Analyzers/Resources.resx Adds title/message/description resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.cs.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.de.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.es.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.fr.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.it.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.ja.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.ko.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.pl.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.pt-BR.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.ru.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.tr.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.zh-Hans.xlf Adds localization units for new resources.
src/Analyzers/MSTest.Analyzers/xlf/Resources.zh-Hant.xlf Adds localization units for new resources.
test/UnitTests/MSTest.Analyzers.UnitTests/AvoidThreadSleepAndTaskWaitInTestsAnalyzerTests.cs Adds unit tests for diagnostics and non-diagnostic cases.

Copilot's findings

  • Files reviewed: 19/19 changed files
  • Comments generated: 2

Comment thread src/Analyzers/MSTest.Analyzers/AvoidThreadSleepAndTaskWaitInTestsAnalyzer.cs Outdated
Comment thread src/Analyzers/MSTest.Analyzers/Resources.resx Outdated
- Fix XML doc rule ID (MSTEST0064 -> MSTEST0066) on the analyzer summary.
- Expand rule title to also mention Task<TResult>.Result, matching the analyzer's reported scope.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

Copy link
Copy Markdown
Member Author

Addressed the two Copilot review comments in e712ed4:

  • Fixed the XML doc summary to reference MSTEST0066 (was MSTEST0064).
  • Expanded the rule title to include 'Task.Result' so it matches the analyzer scope.

Not adopting the github-code-quality '.Select(...)' suggestion: it would force adding 'using System.Linq' for a single-line projection that doesn't improve readability, and the current foreach is consistent with how other analyzers in this codebase iterate attribute data.

Evangelink and others added 3 commits May 28, 2026 14:27
…verage

- Detect static Task.WaitAll / Task.WaitAny as blocking calls.
- Update title/description resources and regenerate xlf files.
- Add tests for ClassCleanup, AssemblyCleanup, GlobalTestInitialize, GlobalTestCleanup.
- Add test for custom attribute deriving from TestMethodAttribute.
- Add expression-bodied test method case.
- Add VB smoke tests (Thread.Sleep and Task(Of T).Result).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Conflicts:
#	src/Analyzers/MSTest.Analyzers/AnalyzerReleases.Unshipped.md
#	src/Analyzers/MSTest.Analyzers/Helpers/DiagnosticIds.cs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 28, 2026 12:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 19/19 changed files
  • Comments generated: 1

Comment thread src/Analyzers/MSTest.Analyzers/Helpers/DiagnosticIds.cs
Comment thread src/Analyzers/MSTest.Analyzers/AvoidThreadSleepAndTaskWaitInTestsAnalyzer.cs Dismissed
@Evangelink Evangelink changed the title Add MSTEST0066 analyzer to flag Thread.Sleep/Task.Wait/Task<T>.Result in tests Add MSTEST0067 analyzer to flag Thread.Sleep/Task.Wait/Task<T>.Result in tests May 28, 2026
@Evangelink Evangelink merged commit aa3e9e4 into main May 28, 2026
49 of 50 checks passed
@Evangelink Evangelink deleted the dev/amauryleve/mstest0066-avoid-thread-sleep branch May 28, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Detect Thread.Sleep(x) and Thread.Wait(x) inside test code to reduce flakiness

2 participants