[BUG] r22 appears to break cmake compiler and OS detection on Windows

[grendello]

Description

Xamarin.Android has been using Android SDK's cmake for quite a while now and
until NDK r22 everything has worked as expected. With r22 the Unix (Linux and macOS) builds still work fine, however the Windows build breaks because
cmake is not able to properly detect the C/C++ compilers (please see the attached error and output logs). Not only that, it appears that the WIN32
cmake variable is not set at all, instead MINGW is set.

We normally use cmake from the SDK (version 3.10.2), but I also tried with cmake 3.18 shipped with VS2019, to the same effect. Our CMakeLists.txt, PR to switch XA to NDK r22, and a sample invocation line:

C:\Users\AzDevOps\android-toolchain\sdk\cmake\3.10.2.4988404\bin\cmake --debug-output -GNinja -DCMAKE_MAKE_PROGRAM=C:\Users\AzDevOps\android-toolchain\sdk\cmake\3.10.2.4988404\bin\ninja -DANDROID_STL="none" -DANDROID_CPP_FEATURES="no-rtti no-e
xceptions" -DANDROID_TOOLCHAIN=clang -DCMAKE_TOOLCHAIN_FILE=C:\Users\AzDevOps\android-toolchain\ndk\build\cmake\android.toolchain.cmake -DANDROID_NDK=C:\Users\AzDevOps\android-toolchain\ndk -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DJDK_INCLUDE="C:\Users\AzDevOps\android-toolchain\jdk-11\include C:\Users\AzDevOps\android-toolchain\jdk-11\include\win32" -DMONO_PATH=C:\a\2\s\external\mono -DCONFIGURATION=Release -DCMAKE_BUILD_TYPE=Debug -DANDROID_NATIVE_API_LEVEL=16 -DANDROID_PLATFORM=android-16 -DANDROID_ABI=armeabi-v7a -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=C:\a\2\s\bin\Release\lib\xamarin.android\xbuild\Xamarin\Android\lib\armeabi-v7a -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=C:\a\2\s\bin\Release\lib\xamarin.android\xbuild\Xamarin\Android\lib\armeabi-v7a C:\a\2\s\src\monodroid\

The same CMakeList.txt (and the same cmake version) works fine with r21 on Windows

Environment Details

Not all of these will be relevant to every bug, but please provide as much
information as you can.

• NDK Version: 22.0.7026061
• Build system: cmake
• Host OS: Windows 10 64bit
• ABI: all
• NDK API level: 16 or 21 (others not tested)
• Device API level: N/A

CMakeError.log.txt
CMakeOutput.log.txt

[grendello]

Just a note regarding the command above. I do know that r22 doesn't have support for API16 (at least the runtime libs for it are absent, clang wrapper scripts are still there), but it's not relevant in this case - the error appears to be unrelated at least. However, for API21 build cmake still mis-detects the system as MINGW. Please see the logs attached below - the libraries mentioned in the error log are used only if MINGW is detected.

CMakeError.log.txt
CMakeOutput.log.txt

[DanAlbert]

Looks like CMake hasn't set the -target for Clang?

Compiler: C:/Users/AzDevOps/android-toolchain/ndk/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe 
Build flags: -g;-DANDROID;-fdata-sections;-ffunction-sections;-funwind-tables;-fstack-protector-strong;-no-canonical-prefixes;-D_FORTIFY_SOURCE=2;-march=armv7-a;-mthumb;-Wformat;-Werror=format-security;
Id flags:  

The output was:
1
clang: warning: argument unused during compilation: '-mthumb' [-Wunused-command-line-argument]
error: unknown target CPU 'armv7-a'
note: valid target CPU values are: nocona, core2, penryn, bonnell, atom, silvermont, slm, goldmont, goldmont-plus, tremont, nehalem, corei7, westmere, sandybridge, corei7-avx, ivybridge, core-avx-i, haswell, core-avx2, broadwell, skylake, skylake-avx512, skx, cascadelake, cooperlake, cannonlake, icelake-client, icelake-server, tigerlake, knl, knm, k8, athlon64, athlon-fx, opteron, k8-sse3, athlon64-sse3, opteron-sse3, amdfam10, barcelona, btver1, btver2, bdver1, bdver2, bdver3, bdver4, znver1, znver2, x86-64

Our QA will have tested Windows before we shipped though. We probably need a repro case here.

at least the runtime libs for it are absent

$ ls android-ndk-r22/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/16/
crtbegin_dynamic.o
crtbegin_so.o
crtbegin_static.o
crtend_android.o
crtend_so.o
libandroid.so
libc++.a
libc.a
libcompiler_rt-extras.a
libc++.so
libc.so
libdl.a
libdl.so
libEGL.so
libGLESv1_CM.so
libGLESv2.so
libjnigraphics.so
liblog.so
libm.a
libm.so
libOpenMAXAL.so
libOpenSLES.so
libstdc++.a
libstdc++.so
libz.a
libz.so

[DanAlbert]

Assigning to hhb in case he has any ideas.

[grendello]

Looks like CMake hasn't set the -target for Clang?

It seems as if ARM support wasn't there at all, note this error:

error: unknown target CPU 'armv7-a'

And look at the list of supported CPUs, it contains only x86.

Compiler: C:/Users/AzDevOps/android-toolchain/ndk/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe 
Build flags: -g;-DANDROID;-fdata-sections;-ffunction-sections;-funwind-tables;-fstack-protector-strong;-no-canonical-prefixes;-D_FORTIFY_SOURCE=2;-march=armv7-a;-mthumb;-Wformat;-Werror=format-security;
Id flags:  

The output was:
1
clang: warning: argument unused during compilation: '-mthumb' [-Wunused-command-line-argument]
error: unknown target CPU 'armv7-a'
note: valid target CPU values are: nocona, core2, penryn, bonnell, atom, silvermont, slm, goldmont, goldmont-plus, tremont, nehalem, corei7, westmere, sandybridge, corei7-avx, ivybridge, core-avx-i, haswell, core-avx2, broadwell, skylake, skylake-avx512, skx, cascadelake, cooperlake, cannonlake, icelake-client, icelake-server, tigerlake, knl, knm, k8, athlon64, athlon-fx, opteron, k8-sse3, athlon64-sse3, opteron-sse3, amdfam10, barcelona, btver1, btver2, bdver1, bdver2, bdver3, bdver4, znver1, znver2, x86-64

Our QA will have tested Windows before we shipped though. We probably need a repro case here.

at least the runtime libs for it are absent

My bad, looked in a wrong directory (aarch64) :)

$ ls android-ndk-r22/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/16/
crtbegin_dynamic.o
crtbegin_so.o
crtbegin_static.o
crtend_android.o
crtend_so.o
libandroid.so
libc++.a
libc.a
libcompiler_rt-extras.a
libc++.so
libc.so
libdl.a
libdl.so
libEGL.so
libGLESv1_CM.so
libGLESv2.so
libjnigraphics.so
liblog.so
libm.a
libm.so
libOpenMAXAL.so
libOpenSLES.so
libstdc++.a
libstdc++.so
libz.a
libz.so

[grendello]

I think the error: unknown target CPU 'armv7-a' error is a red herring and that, ultimately, the problem is the misdetection of the system as MINGW. Looking further at the CMakeOutput.log attached in the OP, it gets the correct compiler options:

Detecting C compiler ABI info compiled with the following output:
Change Dir: C:/Users/Grendel/vc/xamarin-android/src/monodroid/obj/Debug/armeabi-v7a-Debug/CMakeFiles/CMakeTmp

Run Build Command:"C:\Users\Grendel\android-toolchain\sdk\cmake\3.10.2.4988404\bin\ninja" "cmTC_22a78"
[1/2] Building C object CMakeFiles/cmTC_22a78.dir/CMakeCCompilerABI.c.o
[2/2] Linking C executable cmTC_22a78
Android (6875598, based on r399163b) clang version 11.0.5 (https://android.googlesource.com/toolchain/llvm-project 87f1315dfbea7c137aa2e6d362dbb457e388158d)
Target: armv7-none-linux-android16
Thread model: posix
InstalledDir: C:\Users\Grendel\android-toolchain\ndk\toolchains\llvm\prebuilt\windows-x86_64\bin
Found candidate GCC installation: C:/Users/Grendel/android-toolchain/ndk/toolchains/llvm/prebuilt/windows-x86_64/lib/gcc/arm-linux-androideabi\4.9.x
Selected GCC installation: C:/Users/Grendel/android-toolchain/ndk/toolchains/llvm/prebuilt/windows-x86_64/lib/gcc/arm-linux-androideabi/4.9.x
Candidate multilib: thumb;@mthumb
Candidate multilib: armv7-a;@march=armv7-a
Candidate multilib: armv7-a/thumb;@march=armv7-a@mthumb
Candidate multilib: .;
Selected multilib: armv7-a/thumb;@march=armv7-a@mthumb

However, I added the following snippet to our CMakeLists.txt:

if(WIN32)
        message(STATUS "WIN32 set")
endif()

if(MINGW)
        message(STATUS "MINGW set")
endif()

And the output is:

-- MINGW set
   Called from: [1]     src/monodroid/CMakeLists.txt
CMake Error at CMakeLists.txt:93 (message):
  Please set the MINGW_DEPENDENCIES_ROOT_DIR variable on command line
  (-DMINGW_DEPENDENCIES_ROOT_DIR=PATH)

[fuchg3]

It seems cmake 3.19.2 could detect the right platform. That's my temporary workaround.

[hhb]

It seems cmake 3.19.2 could detect the right platform. That's my temporary workaround.

You probably are using cmake's NDK support instead of NDK toolchain file. In that case CMake >= 3.19 is required for NDK r22.

(I'm investing why MINGW is used there in this issue

[hhb]

Seems it is not trivial to build Xamarin.Android. Do you have a smaller repro case?

Also can you try to copy r21 android.toolchain.cmake to r22 to see how it does? One of the new changes in r22 is aosp/1413029. Maybe it affects compiler detection somehow?

[grendello]

Seems it is not trivial to build Xamarin.Android. Do you have a smaller repro case?

You don't really need to build it all. On Windows (from VS prompt), after cloning and restoring submodules, you need the following steps:

1. In the top source dir run msbuild /t:Prepare Xamarin.Android.sln
2. Change to the src/monodroid directory and run msbuild /bl (the /bl will create a detailed binary log, msbuild.binlog, which you can replay either with msbuild /v:diag msbuild.binlog or with https://msbuildlog.com/)
3. CMake generates its files in subdirectories of src/monodroid/obj/{Debug,Release}, you can rerun cmake in any of those with the options needed

Also can you try to copy r21 android.toolchain.cmake to r22 to see how it does? One of the new changes in r22 is aosp/1413029. Maybe it affects compiler detection somehow?

I don't use Windows locally, but I'll try to test that ASAP.

[fuchg3]

Seems it is not trivial to build Xamarin.Android. Do you have a smaller repro case?

Also can you try to copy r21 android.toolchain.cmake to r22 to see how it does? One of the new changes in r22 is aosp/1413029. Maybe it affects compiler detection somehow?

The problem seems fixed after replacing android.toolchain.cmake file.

[grendello]

@fuchg3 thanks, you saved me from a few hours of installing software :)