Compiling with Simulink API: Matlab *.so undefined reference to '***** @GLIBCXX_3.4.21''

[Seager1989]

Hi everyone,

I am trying to compile the OpenFAST with the Simulink API using the dev branch of the latest version on an HPC cluster with the system information as follows. The MATLAB 2020a is installed. I use the manually compiled GCC-6.3.0 since this is the version supported by the MATLAB 2020a.

Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-1062.9.1.el7.x86_64
Architecture: x86-64

After cloning the OpenFAST, I did run the following comman.

mkdir build
cd build
cmake .. -DBUILD_OPENFAST_SIMULINK_API=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX="/home/xd101/local/openfast_sim" -DBUILD_TESTING=ON
make -j 4

However, error appeared with key information listed below:

/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmex.so: undefined reference to `VTT for std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwmlutil.so: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::_Ios_Openmode)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwboost_log.so.1.70.0: undefined reference to `typeinfo for std::codecvt<char16_t, char, __mbstate_t>@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwfl.so: undefined reference to `std::ios_base::failure[abi:cxx11]::~failure()@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwflnetwork.so: undefined reference to `std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::erase(unsigned long, unsigned long)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwboost_regex.so.1.70.0: undefined reference to `std::__cxx11::messages<char> const& std::use_facet<std::__cxx11::messages<char> >(std::locale const&)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwi18n.so: undefined reference to `std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::_M_construct(unsigned long, wchar_t)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwservices.so: undefined reference to `std::out_of_range::out_of_range(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmex.so: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmex.so: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwflnetwork.so: undefined reference to `std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::rfind(wchar_t, unsigned long) const@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmex.so: undefined reference to `std::logic_error::logic_error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwboost_regex.so.1.70.0: undefined reference to `std::overflow_error::overflow_error(char const*)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwboost_log.so.1.70.0: undefined reference to `std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::~basic_string()@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmex.so: undefined reference to `operator delete(void*, unsigned long)@CXXABI_1.3.9'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwboost_log.so.1.70.0: undefined reference to `std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::_M_append(wchar_t const*, unsigned long)@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libCppMicroServices.so.3.3.5: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::find_last_not_of(char, unsigned long) const@GLIBCXX_3.4.21'
/cache/sw/packages/MATLAB/R2020a/bin/glnxa64/libmwservices.so: undefined reference to `std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::pbackfail(int)@GLIBCXX_3.4.21'
collect2: error: ld returned 1 exit status
make[2]: *** [modules/inflowwind/CMakeFiles/inflowwind_driver.dir/build.make:119: modules/inflowwind/inflowwind_driver] Error 1
make[1]: *** [CMakeFiles/Makefile2:145: modules/inflowwind/CMakeFiles/inflowwind_driver.dir/all] Error 2

I do not know if this a permission issue, code issue, or something else? Does anyone have any ideas on this problem? Thank you for any of your help.

[bjonkman]

This looks like a linking issue to me. Perhaps the build options for the mex function in Matlab 2020a are incompatible with what you used to build OpenFAST. The errors have references to cxx11, so maybe OpenFAST has to be configured differently.

Maybe this helps? https://www.neuron.yale.edu/phpBB/viewtopic.php?t=3880

[Seager1989]

This looks like a linking issue to me. Perhaps the build options for the mex function in Matlab 2020a are incompatible with what you used to build OpenFAST. The errors have references to cxx11, so maybe OpenFAST has to be configured differently.

Maybe this helps? https://www.neuron.yale.edu/phpBB/viewtopic.php?t=3880

Hi Bonnie,

The response from the link you provided seems interesting, but I do not know how to incorporate the CXXFLAGS='-D_GLIBCXX_USE_CXX11_ABI=0' option? I can not make it work with the OpenFAST compiling. Thank you

[bjonkman]

You'd have to change the options in the cmake files. Perhaps @rafmudaf or @andrew-platt could help with that.

[rafmudaf]

In CMake, you can pass a flag with -D in the command line. @Seager1989 read the CMake section of the OpenFAST docs for a reference and detailed description.

[rafmudaf]

In this case, CXXFLAGS is the CMake flag and the contents of it is the compiler flag.

[Seager1989]

Thank you!

1. I try to do the following setup.
   cmake .. -DBUILD_OPENFAST_SIMULINK_API=ON -DCMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=0' -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX="/home/xd101/local/openfast_sim" -DBUILD_TESTING=ON

This generate an error:

/home/xd101/Downloads/openfast_Simulink/openfast_sim/build2/unit_tests/pfunit/include/driver.F90:3:7:
    use pfunit_mod
       1
Fatal Error: Cannot read module file 'pfunit_mod.mod' opened at (1), because it was created by a different version of GNU Fortran
compilation terminated.
make[2]: *** [unit_tests/CMakeFiles/beamdyn_utest.dir/build.make:123: unit_tests/CMakeFiles/beamdyn_utest.dir/pfunit/include/driver.F90.o] Error 1

2. So, I turned off the BUILD_TESTING:
   cmake .. -DBUILD_OPENFAST_SIMULINK_API=ON -DCMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=0' -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX="/home/xd101/local/openfast_sim" -DBUILD_TESTING=OFF.

It works and I generated the S-function successfully.

However, I got errors as I run a simple model to call S-function.

TMax =

    20


 **************************************************************************************************
 OpenFAST

 Copyright (C)  National Renewable Energy Laboratory
 Copyright (C)  Envision Energy USA LTD

 This program is licensed under Apache License Version 2.0 and comes with ABSOLUTELY NO WARRANTY.
 See the "LICENSE" file distributed with this software for details.
 **************************************************************************************************

 OpenFAST-HEAD-HASH-NOTFOUND
 Compile Info:
  - Architecture: 64 bit
  - Precision: double
  - Date: Oct  8 2020
  - Time: 18:13:51
 Execution Info:
  - Date: 10/08/2020
  - Time: 20:06:34-0400

 OpenFAST input file heading:
     FAST Certification Test #18: NREL 5.0 MW Baseline Wind Turbine (Onshore)

 Running ElastoDyn.
 Running AeroDyn.
 Running BEM.
 Running InflowWind.

    Reading a 31x31 grid (145 m wide, 17.5 m to 162.5 m above ground) with a characteristic wind
    speed of 12 m/s. This full-field file was generated by TurbSim (v1.06.00, 21-Sep-2012) on
    07-Jan-2014 at 12:50:45.

    Processed 1442 time steps of 20-Hz full-field data (72.05 seconds).
 Running ServoDyn.

FAST_InitializeAll:SrvD_Init:ValidatePrimaryData:Yaw angle and rate are not commanded from Simulink model.
ValidatePrimaryData:HSS brake is not commanded from Simulink model.
  Time: 0 of 20 seconds.

Intel MKL ERROR: Parameter 4 was incorrect on entry to DGEMM .

Intel MKL ERROR: Parameter 4 was incorrect on entry to DGEMM .

Intel MKL ERROR: Parameter 10 was incorrect on entry to DGEMM .

...................................................many rows..............................................................

  OpenFAST completed.

Error using openloopre (line 7)
Error reported by S-function 'FAST_SFunc' in 'OpenLoop/FAST Nonlinear Wind Turbine/S-Function':
FAST_Solution:FAST_AdvanceStates:ED_ABM4:ED_AB4:ED_CalcContStateDeriv:LAPACK_DGETRF: illegal value in argument           -4.
FAST_AdvanceStates:SolveOption2a_Inp2BD:ED_CalcContStateDeriv:LAPACK_DGETRF: illegal value in argument           -4.

Error in run (line 91)
evalin('caller', strcat(script, ';')); - Show complete stack trace

This is a similar problem with the one mentioned by @andrew-platt in #531, where the Fortran compiler is connected with the Intel MKL. It is so strange, do you know what is the problem? Thank you

[rafmudaf]

Well, its just due to CMake finding MKL on your path before any other BLAS/LAPACK library. If you don't want to use MKL specifically, then you can specify the math library directly (docs). You could uninstall MKL or try using the BLA_VENDOR flag.

For what its worth, there's no hard requirement that MKL is used with the Intel Fortran compiler and open source math libraries are used with GNU Fortran. I regularly use MKL with gfortran.

As for that particular error, I do not know what the problem could be. I am curious to know whether it is also shown with other math libraries and/or precision settings.

[Seager1989]

I tried to re-compiling it but get the undefined reference error again. I set the -DCMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=0', but it does also not help. I do not why the previous compilation can be successful after I tried every possible setup. I tried the GCC7.3.0, which works to compile the OpenFAST with Simulink API but it can not compatible with MATLAB 2020a. I do not know what's the difference caused the outcomes.

Is there any other possible ways to fix this undefined reference error? Thank you.

[Seager1989]

I fixed the compiling problem by exporting the Gcc6.3.0-lib64 to the $Path. I compiled the libstdc++.so, but the system can not find it, which is fixed under the help of the GCC community GCC-debug#97369. Also, ensuring the libopenfastlib.so can find all dependencies.

Now, it seems the S-function is running but an new error is in there as follows.
`
Stack Trace (from fault):
[ 0] 0x00007fc361b66d11 /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+23264529 mkl_blas_avx2_xidamax_nonan+00000417
[ 1] 0x00007fc361a33a0c /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+22006284 mkl_blas_avx2_xidamax+00000060
[ 2] 0x00007fc360bff186 /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+07111046 mkl_blas_xidamax+00000086
[ 3] 0x00007fc360b831ad /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+06603181 mkl_blas_idamax+00000861
[ 4] 0x00007fc36132acdf /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+14630111 mkl_lapack_dgetf2_team+00000975
[ 5] 0x00007fc36132a84c /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+14628940 mkl_lapack_dgetrf_local_team+00002812
[ 6] 0x00007fc36132a00b /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+14626827 mkl_lapack_dgetrf_local_team+00000699
[ 7] 0x00007fc36132a00b /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+14626827 mkl_lapack_dgetrf_local_team+00000699
[ 8] 0x00007fc36132a00b /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+14626827 mkl_lapack_dgetrf_local_team+00000699
[ 9] 0x00007fc36132a00b /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+14626827 mkl_lapack_dgetrf_local_team+00000699
[ 10] 0x00007fc36132a00b /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+14626827 mkl_lapack_dgetrf_local_team+00000699
[ 11] 0x00007fc360a83888 /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/mkl.so+05556360 mkl_lapack_dgetrf+00001816
[ 12] 0x00007fc79d50f9f3 /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/../../sys/os/glnxa64/libiomp5.so+00973299 __kmp_invoke_microtask+00000147
[ 13] 0x00007fc79d4cff96 /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/../../sys/os/glnxa64/libiomp5.so+00712598
[ 14] 0x00007fc79d4cef81 /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/../../sys/os/glnxa64/libiomp5.so+00708481
[ 15] 0x00007fc79d50fe8b /cache/sw/packages/MATLAB/R2020a/bin/glnxa64/../../sys/os/glnxa64/libiomp5.so+00974475
[ 16] 0x00007fc7b4cd2e65 /lib64/libpthread.so.0+00032357
[ 17] 0x00007fc7b446388d /lib64/libc.so.6+01042573 clone+00000109

Abnormal termination:
Segmentation violation

Register State (from fault):
RAX = 0000000000000001 RBX = 00007fc1253f1798
RCX = 0000000000000000 RDX = 0000000000000001
RSP = 00007fc1253f0318 RBP = 0000000000000000
RSI = 000000004aaaaaa8 RDI = 00007fc1253f1798

R8 = 00007fe3a0cba0d0 R9 = 000000004aaaaaa8
R10 = 4b7649c000000000 R11 = 0000000000000000
R12 = 00007fe3a0cba0d0 R13 = 00007fc365ec5ac8
R14 = 00007fc365ec57d8 R15 = 00007fc1253f2048

RIP = 00007fc361b66d11 EFL = 0000000000010246

CS = 0033 FS = 0000 GS = 0000

Stack Trace (from fault):
** This crash report has been saved to disk as /home/xd101/matlab_crash_dump.9925-6 **
`
Also, *.SFunc.out can be generated with only the following statement: Predictions were generated on 11-Oct-2020 at 19:14:18 using OpenFAST, compiled as a DLL S-Function for Simulink as a 64-bit application using double precision at commit HEAD-HASH-NOTFOUND.
I can make sure that I compiled with openblas, but it seems there is an communication issue. I am not sure what is the exact error. Can you please help find what is the exact problem of this error? Thank you

[Seager1989]

Thank you for your help. With the help of #482 , overriding the shipped libraries of MATLAB by the below code solved this problem.

export BLAS_VERSION=libopenblas.so export LAPACK_VERSION=libopenblas.so matlab