Testing Platform overwriting/removing user console output

[thomhurst]

Using TUnit, I created this simple test suite:

public class Tests
{
    [Before(TestSession)]
    public static async Task BeforeTestSession()
    {
        Console.WriteLine($"BeforeTestSession started at {DateTime.Now}");
        await Task.Delay(TimeSpan.FromSeconds(5));
        Console.WriteLine($"BeforeTestSession ended at {DateTime.Now}");
    }
    
    [Before(Assembly)]
    public static async Task BeforeAssembly()
    {
        Console.WriteLine($"BeforeAssembly started at {DateTime.Now}");
        await Task.Delay(TimeSpan.FromSeconds(5));
        Console.WriteLine($"BeforeAssembly ended at {DateTime.Now}");
    }
    
    [Test]
    public void Test1()
    {
        Console.WriteLine($"Executing Test at {DateTime.Now}");
    }
}

If you just do a dotnet run on the CLI, you'll have to be quick, but when it starts to show "BeforeAssembly started", you'll see it render really quickly, but then it'll be removed and overwritten by the rendering of the [✓0/x0/↓0] ConsoleApp5.dll (net9.0|x64).

I guess this is related to the new interactive output and showing running tests etc. But overwriting user output means they may lose visibility of important information.

[Evangelink]

I'm off for now so just dumping my thoughts for when I return. We need to try if the behavior is the same when using IOutput to display messages.

[Evangelink]

I won't be able to investigate more right now but dumping more info of trying to repro:

The before and after on assembly are not displayed at all while the test session is.

[Evangelink]

@nohwnd What's your opinion and recommendations here?

[thomhurst]

The before and after on assembly are not displayed at all while the test session is.

From what I saw the Before assembly happens really quickly, but then immediately after TUnit will pick up the first test and send a test node update with InProgressState, which is probably causing the output terminal to update and re-render.

[nohwnd]

This is bound to happen when the progress is enabled, and we write to the screen from a different place, e.g from the test directly without capturing the output.

There is no way around unless we substitute the Console.Out with one that is respecting the lock we have in progress reporting.

#3491

Similarly this can happen when we run in child process and parent process (testhost controller writes to the screen).

[Youssef1313]

@nohwnd I didn't look into how the progress-aware terminal works, but I think at whatever point the position of writing is determined, to the end of writing, should be wrapped in lock (Console.Out).

In short, we can already lock on Console.Out which is implicitly the same lock as user writing to console. So, no need to substitute Console.Out with one that respects our lock. Instead we will respect the lock used by runtime's SyncTextWriter for the default console.