fix: the undefined reference compile error when using cuDNN

[Murad1997]

When I try to compile the llm.c with the cuDNN enabled. I get the following error.

/usr/local/cuda/bin/nvcc --threads=0 -t=0 --use_fast_math -std=c++17 -O3 -DENABLE_CUDNN -DENABLE_BF16 train_gpt2.cu build/cudnn_att.o -lcublas -lcublasLt -lnvidia-ml -lcudnn -Icudnn-frontend/include  -o train_gpt2cu
/usr/bin/ld: build/cudnn_att.o: in function `cudnn_frontend::experimental::Sm100RmsNormSiluEngine::build()':
cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0x8e0): undefined reference to `nvrtcCreateProgram'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0xb90): undefined reference to `nvrtcCompileProgram'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0xbb0): undefined reference to `nvrtcGetProgramLogSize'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0xc01): undefined reference to `nvrtcGetProgramLog'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0xc32): undefined reference to `nvrtcDestroyProgram'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0xeca): undefined reference to `nvrtcGetCUBINSize'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0xf3d): undefined reference to `nvrtcGetCUBIN'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv[_ZN14cudnn_frontend12experimental22Sm100RmsNormSiluEngine5buildEv]+0x128c): undefined reference to `nvrtcDestroyProgram'
/usr/bin/ld: build/cudnn_att.o: in function `cudnn_frontend::experimental::compile_and_load_kernel(cudnn_frontend::experimental::KernelSpec const*, CUlib_st*&, CUkern_st*&, std::unique_ptr<char [], std::default_delete<char []> >&, unsigned long&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x11e): undefined reference to `nvrtcCreateProgram'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x378): undefined reference to `nvrtcCompileProgram'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x3ae): undefined reference to `nvrtcGetProgramLogSize'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x408): undefined reference to `nvrtcGetProgramLog'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x492): undefined reference to `nvrtcDestroyProgram'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x54c): undefined reference to `nvrtcGetCUBINSize'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x6be): undefined reference to `nvrtcGetCUBIN'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x9c2): undefined reference to `nvrtcGetPTXSize'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0xa7f): undefined reference to `nvrtcDestroyProgram'
/usr/bin/ld: cudnn_att.cpp:(.text._ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN14cudnn_frontend12experimental23compile_and_load_kernelEPKNS0_10KernelSpecERP8CUlib_stRP9CUkern_stRSt10unique_ptrIA_cSt14default_deleteISB_EERmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0xfdf): undefined reference to `nvrtcGetPTX'
collect2: error: ld returned 1 exit status
make: *** [Makefile:274: train_gpt2cu] Error 255

However by adding the lnvrtc flag to the NVCC_LDFLAGS fix the problem.