This repository was archived by the owner on Mar 20, 2023. It is now read-only.
Support for shared libraries in GPU execution (python launch support) #795
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment was marked as outdated.
This comment was marked as outdated.
Collaborator
Author
|
@olupton : to reproduce the linking issue: git pull
git checkout pramodk/exclude-global-vars
git submodule update -f --init --recursivetemporarily update hh.mod with call to random123: +++ b/coreneuron/mechanism/mech/modfile/hh.mod
@@ -112,6 +112,9 @@ UNITSOFF
sum = alpha + beta
ntau = 1/(q10*sum)
ninf = alpha/sum
+ VERBATIM
+ double xxx = nrnran123_dblpick(nullptr);
+ ENDVERBATIM
}and build: and this should give: [ 81%] Running nrnivmodl-core with halfgap.mod
[INFO] Running: make -j1 -f /gpfs/bbp.cscs.ch/home/kumbhar/workarena/systems/bbpv/repos/bbp/coreneuron/build_gpu_pr/share/coreneuron/nrnivmodl_core_makefile ROOT=/gpfs/bbp.cscs.ch/home/kumbhar/workarena/systems/bbpv/repos/bbp/coreneuron/build_gpu_pr MOD2CPP_BINARY=/gpfs/bbp.cscs.ch/home/kumbhar/workarena/systems/bbpv/repos/bbp/coreneuron/build_gpu_pr/bin/mod2c_core MODS_PATH=x86_64/corenrn/mod2c BUILD_TYPE=SHARED NRN_PRCELLSTATE=0
Default NMODL flags:
NVC++-W-1057-Static variables are not supported in acc routine - _ZN49_INTERNAL_27_x86_64_corenrn_mod2c_hh_cpp_4898369610coreneuron21_global_variables_ptrE (x86_64/corenrn/mod2c/hh.cpp: 364)
NVC++-W-1057-Static variables are not supported in acc routine - _ZN49_INTERNAL_27_x86_64_corenrn_mod2c_hh_cpp_4898369610coreneuron21_global_variables_ptrE (x86_64/corenrn/mod2c/hh.cpp: 374)
NVC++-W-1057-Static variables are not supported in acc routine - _ZN49_INTERNAL_27_x86_64_corenrn_mod2c_hh_cpp_4898369610coreneuron21_global_variables_ptrE (x86_64/corenrn/mod2c/hh.cpp: 440)
ptxas fatal : Unresolved extern function '_ZN10coreneuron17nrnran123_dblpickEPNS_15nrnran123_StateE'
NVC++-F-0155-Compiler failed to translate accelerator region (see -Minfo messages): Device compiler exited with error status code (x86_64/corenrn/mod2c/hh.cpp: 444)
NVC++/x86-64 Linux 22.2-0: compilation aborted
make[3]: *** [x86_64/corenrn/build/hh.o] Error 2
make[2]: *** [bin/x86_64/special-core] Error 2
make[1]: *** [coreneuron/CMakeFiles/nrniv-core.dir/all] Error 2 |
Contributor
|
Two posts from me about the compilation/linking issues:
we currently don't know how to produce a shared library containing both OpenACC and CUDA functions. |
This comment was marked as outdated.
This comment was marked as outdated.
b5083fd to
89baf7b
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
9b8fe22 to
df32d6f
Compare
olupton
reviewed
Apr 26, 2022
This comment was marked as outdated.
This comment was marked as outdated.
Contributor
|
I think the CI failures are mainly because of build system issues, and using (apparently) not widely supported arguments to |
This comment was marked as outdated.
This comment was marked as outdated.
pramodk
commented
Apr 26, 2022
Collaborator
Author
pramodk
left a comment
There was a problem hiding this comment.
I went through the changes quickly and I added my quick comments here. The only thing that stands out is random123 but that could be rediscussed once we have initial tests are working.
9ebd22d to
3989293
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
e9f8930 to
4af041d
Compare
This comment was marked as outdated.
This comment was marked as outdated.
4af041d to
246918e
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Contributor
|
neuronsimulator/nrn#1922 is needed to make some tests (e.g. |
Collaborator
Collaborator
pramodk
commented
Aug 25, 2022
Collaborator
Author
pramodk
left a comment
There was a problem hiding this comment.
LGTM!
Don't have anything specific to add here except that rpath question.
Collaborator
Collaborator
4e0e386 to
0e24755
Compare
Collaborator
Collaborator
Collaborator
pramodk
commented
Aug 28, 2022
Codecov Report
@@ Coverage Diff @@
## master #795 +/- ##
==========================================
- Coverage 58.49% 57.74% -0.76%
==========================================
Files 102 103 +1
Lines 9412 9459 +47
==========================================
- Hits 5506 5462 -44
- Misses 3906 3997 +91
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Summary
enables launching coreneuron on GPU via pyton
and they don't need to be copied on GPU
TODOs
See see Avoid use of global variables in generated code mod2c#78
See Support for SHARED build with PGI OpenACC build #141 (comment)
into link errors, see
Support for SHARED build with PGI OpenACC build #141 (comment)
@olupton to rescue here!
pointer when coreneuron is launched via python. See
Support for SHARED build with PGI OpenACC build #141 (comment)
python
call to cuLaunchKernel returned error 400: Invalid handleCloses #141.
Closes #599.
How to test this?
Build neuron master with this PR branch of coreneuron including updated mod2c submodule.
module load unstable gcc nvhpc cuda hpe-mpi cmake python-dev cmake .. -DCMAKE_INSTALL_PREFIX=`pwd`/install -DCORENRN_ENABLE_GPU=ON -DCORENRN_ENABLE_NMODL=OFF -DCORENRN_ENABLE_MPI=ON -DNRN_ENABLE_CORENEURON=ON -DNRN_ENABLE_INTERVIEWS=OFF -DNRN_ENABLE_TESTS=OFF make -j12 make installNo compile
ringtestwithnrnivmodl -coreneuronand launch ringtest on GPU usingpythonas well asspecial. See #141 (comment)Test System
CI_BRANCHES:NEURON_BRANCH=olupton/coreneuron-gpu-dynamic-loading,SPACK_BRANCH=olupton/coreneuron-gpu-dynamic