Erlang/OTP 26 write to stdout hangs in latin1 encoding on Windows

[lukaszsamson]

Describe the bug
After changing standard_io encoding to latin1 all writes to the output device hang.

To Reproduce

Run from windows command prompt

erl -noshell -eval "Path = os:find_executable('erl'), Port1 = erlang:open_port({spawn_executable, Path}, [binary, {args, ['-noshell', '-kernel', 'standard_io_encoding', 'latin1', '-eval', \"io:setopts(standard_io, [{encoding, latin1}]), io:format('duπa~n', []), erlang:halt()\"]}, exit_status]), receive {Port1, {data, A}} -> io:fwrite(A, []) after 2000 -> io:fwrite('nothing~n') end, erlang:halt()"

Result on windows

nothing

Expected behavior
Should print

du\x{3C0}a

it works correctly on OTP 25 both windows and *nix, and on OTP 26.0.2 on *nix

Affected versions
OTP 26.0.2

Additional context

This bug was tricky to track as it hangs only with a spawned process. If the same code is run directly from cmd shell it works correctly

erl -noshell -kernel standard_io_encoding latin1 -eval "io:setopts(standard_io, [{encoding, latin1}]), io:format('duπa~n', []), erlang:halt()"

prints

du\x{3C0}a

This bug breaks ElixirLS on windows elixir-lsp/elixir-ls#927

Possibly this is a regression of #7261 once fixed in 2bbeed3. The original script from that bug as well as a modified one hangs as well

-module(foo).
-compile(export_all).

run() ->
  io:put_chars("START\n"),

  open_port(
    {spawn_executable, os:find_executable("erl")},
    [exit_status, binary,
     {args, ["-noshell", "-kernel", "standard_io_encoding", "latin1", "-eval", "io:setopts(standard_io, [{encoding, latin1}]), io:write('teπt'), erlang:halt()."]}]
  ),

  loop().

loop() ->
  receive
    {_, {data, Data}} ->
      io:put_chars(Data),
      loop();

    {_, {exit_status, _}} ->
      erlang:halt()
  end.

Related to #7230, #7384

[garazdawi]

It is indeed #7261 that failed to solve the entire problem. Windows seems to treat input differently depending on whether it is a console, a file, or a pipe. The code in #7261 works when the input is a console or a file, but it does not work for a pipe...

I'm unsure if it is at all possible to achieve non-blocking I/O when STDIN is a pipe... I'll continue digging and see what I get figure out, worse case we may have to live with a blocking API and just make sure that when it happens the rest of the system does not dead-lock.

[garazdawi]

I think I've fixed the issue, but as I did I stumbled across other problems in this area that I need to look closer at before releasing a patch.

One interesting thing I found was that apparently -oldshell and -noshell were documented always be byte-oriented (See io:setopts/1,2 here: https://www.erlang.org/doc/apps/stdlib/unicode_usage.html#summary-of-options). Had I seen that documentation earlier we may have avoided introducing standard_io_encoding altogether, but alas I did not so now we have it...

[garazdawi]

The fix is implemented in #7473 together with some changes in how bytewise (latin1) encoding is handled in general. I've also included some documentation about how things are supposed to work and be used. Hopefully, it will make it easier for anyone else trying to do the same thing.