.NET 7 - Segmentation Fault on BeagleBone Black (arm32)

[Limb]

I'm getting a Segmentation Fault after installing .NET 7.0 SDK (v7.0.100) - Linux Arm32 on a BeagleBone Black running Debian 10.4. I tried both an existing upgrade from 6.0, as well as a fresh SD card image.

Running just the dotnet command works fine, but any other command (even dotnet --version) results in a segmentation fault.

debian@beaglebone:~$ dotnet

Usage: dotnet [options]
Usage: dotnet [path-to-application]

Options:
  -h|--help         Display help.
  --info            Display .NET information.
  --list-sdks       Display the installed SDKs.
  --list-runtimes   Display the installed runtimes.

path-to-application:
  The path to an application .dll file to execute.
debian@beaglebone:~$ dotnet --version
Segmentation fault

Running strace dotnet --version the command seems to output properly (however I am not a strace expert by any means but I am seeing a proper "write" command in the output):

clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=283537198}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=285314775}) = 0
cacheflush(0xb55e7948, 0xb55e7960, 0)   = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=290090158}) = 0
cacheflush(0xab7cab64, 0xab7cabd0, 0)   = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(69, "7.0.100", 77.0.100)                 = 7
write(69, "\n", 1
)                      = 1
unlink("/tmp/dotnet-diagnostic-13773-193303-socket") = 0
futex(0x216f248, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x216f208, FUTEX_WAKE_PRIVATE, 1) = 1
unlink("/tmp/clr-debug-pipe-13773-193303-in") = 0
unlink("/tmp/clr-debug-pipe-13773-193303-out") = 0
write(4, "\3", 1)                       = 1
clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=318814829}) = 0
munmap(0xb6ca3000, 225874)              = 0
exit_group(0)                           = ?
+++ exited with 0 +++

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I'm getting a Segmentation Fault after installing .NET 7.0 SDK (v7.0.100) - Linux Arm32 on a BeagleBone Black running Debian 10.4. I tried both an existing upgrade from 6.0, as well as a fresh SD card image.

Running just the dotnet command works fine, but any other command (even dotnet --version) results in a segmentation fault.

debian@beaglebone:~$ dotnet

Usage: dotnet [options]
Usage: dotnet [path-to-application]

Options:
  -h|--help         Display help.
  --info            Display .NET information.
  --list-sdks       Display the installed SDKs.
  --list-runtimes   Display the installed runtimes.

path-to-application:
  The path to an application .dll file to execute.
debian@beaglebone:~$ dotnet --version
Segmentation fault

Running strace dotnet --version the command seems to output properly (however I am not a strace expert by any means but I am seeing a proper "write" command in the output):

clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=283537198}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=285314775}) = 0
cacheflush(0xb55e7948, 0xb55e7960, 0)   = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=290090158}) = 0
cacheflush(0xab7cab64, 0xab7cabd0, 0)   = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(69, "7.0.100", 77.0.100)                 = 7
write(69, "\n", 1
)                      = 1
unlink("/tmp/dotnet-diagnostic-13773-193303-socket") = 0
futex(0x216f248, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x216f208, FUTEX_WAKE_PRIVATE, 1) = 1
unlink("/tmp/clr-debug-pipe-13773-193303-in") = 0
unlink("/tmp/clr-debug-pipe-13773-193303-out") = 0
write(4, "\3", 1)                       = 1
clock_gettime(CLOCK_MONOTONIC, {tv_sec=1948, tv_nsec=318814829}) = 0
munmap(0xb6ca3000, 225874)              = 0
exit_group(0)                           = ?
+++ exited with 0 +++

Author:	Limb
Assignees:	marcpopMSFT
Labels:	area-Host, untriaged
Milestone:	-

[marcpopMSFT]

Routing to runtime as a likely issue in the runtime or host itself.

[elinor-fung]

@Limb Would it be possible for you to get/share a core dump of the crash?

[Limb]

@elinor-fung Hopefully I created the core dump successfully, please see core

[elinor-fung]

Seems to be in Microsoft.TemplateEngine.Cli.dll:

[0x0]   0xb556c001!+   0xbebcc000   0xabb0b04b   
[0x1]   Microsoft_TemplateEngine_Cli + 0x5c04b!Microsoft_TemplateEngine_Cli+0x5c04b   0xbebcc000   0xabb09ca1   
[0x2]   Microsoft_TemplateEngine_Cli + 0x5aca1!Microsoft_TemplateEngine_Cli+0x5aca1   0xbebcc050   0xabaec809   
[0x3]   Microsoft_TemplateEngine_Cli + 0x3d809!Microsoft_TemplateEngine_Cli+0x3d809   0xbebcc068   0xae8333a9   
[0x4]   dotnet_ae760000 + 0xd33a9!dotnet_ae760000+0xd33a9   0xbebcc080   0xae82e42b   
[0x5]   dotnet_ae760000 + 0xce42b!dotnet_ae760000+0xce42b   0xbebcc0b8   0x0   

@dotnet/dotnet-diag I'm failing at getting managed stacks from the dump. Can someone tell me what I'm missing?
I'm running a docker container with linux arm32 and getting this:

(lldb) sosstatus
Target OS: LINUX Architecture: Arm ProcessId: 6323 (0x18B3)
#0 .NET Core runtime at 00000000B677F000 size 0046E2B9 index ad7c731feb7243bba7f5fa9b0f931444aaad01e0
    Runtime module path: /home/issues/libcoreclr.so
    Runtime module directory: /home/issues
    DAC: /home/issues/libmscordaccore.so


Current symbol store settings:
-> Cache: /root/.dotnet/symbolcache
-> Server: https://msdl.microsoft.com/download/symbols/ Timeout: 4 RetryCount: 0
GC memory usage for managed SOS components: 628,552 bytes
(lldb) clrstack
Failed to load data access module, 0x80004002
Can not load or initialize libmscordaccore.so. The target runtime may not be initialized.

For more information see https://go.microsoft.com/fwlink/?linkid=2135652
ClrStack  failed

[michaldobrodenka]

I think that dump might not be complete or doesn't contain managed info. I'm not expert on this, but I had problems with core dumps some time ago (also on arm32 ).

It usually means:

• disk space issue - did you have enough space on partition where you made dump?
• limit set to dump - maybe something like ulimit -S -c unlimited will help - not to limit dumps
• correct flags - to include all info including managed - I use echo 0xff > /proc/<PID>/coredump_filter in systemd for my process

but i may be wrong :)

[tommcdon]

https://github.com/orgs/dotnet/teams/dotnet-diag I'm failing at getting managed stacks from the dump. Can someone tell me what I'm missing?
I think that dump might not be complete or doesn't contain managed info. I'm not expert on this, but I had problems with core dumps some time ago (also on arm32 ).

Hi @elinor-fung and @michaldobrodenka! It does indeed look like the dump is incomplete. For more details on crash dump generation see here. I suggest collecting a full dump by setting DOTNET_DbgMiniDumpType=4. Hope this helps!

[elinor-fung]

Thanks, @michaldobrodenka and @tommcdon.

@Limb could you try collecting a full dump per #78657 (comment)?

[Limb]

@elinor-fung here is the core dump using DOTNET_DbgMiniDumpType=4

[Limb]

I will add that by chance I happened to run the dotnet --version command again after generating the dump and it worked, but executing the command again resulted in a segmentation fault. So this issue does seem to be inconsistent in generating a fault all the time.

debian@beaglebone:~$ dotnet --version
7.0.100
debian@beaglebone:~$ dotnet --version
Segmentation fault (core dumped)
debian@beaglebone:~$ dotnet --version
Segmentation fault (core dumped)
debian@beaglebone:~$ dotnet --version
Segmentation fault (core dumped)
debian@beaglebone:~$ dotnet --version
7.0.100