MPI-3

[friedmud]

We should probably start to at least test for MPI-3... even if, for now, all we do is switch some internal algorithms.

Here is a good list of the new capabilities we could look for:

https://htor.inf.ethz.ch/blog/index.php/2012/02/06/mpi-3-0-is-coming-an-overview-of-new-and-old-features/

High on my personal list are:

Ibarrier, Ibcast, Igather, Ialltoall, Ireduce, Iallreduce, mprobe, mrecv

Stretch goals with be one-sided communication and windowing - I don't have plans to use those in libMesh right now though.

It would be great to get a configure test for this stuff... so we can start to see what people have on their machines.

[pbauman]

We should probably start to at least test for MPI-3... even if, for now, all we do is switch some internal algorithms.

Agree 100%. Could we maybe log either here or in #1826 or somewhere what MPI library/version support MPI3? Mpich, Mvapich, and Intel-MPI are the most commonly used around here. I don't have any sense of version cutoffs etc.

[roystgnr]

Pinging @BalticPinguin for opinions, since #1822 is obviously relevant. If nearly everybody is on MPI-3 then perhaps we could make MPI-2 support non-default for a while, with something like configure --enable-mpi2 required to build with it and a MPI-2 support is deprecated; please reconfigure with --enable-mpi2 and email libmesh-users@ message printed for people who don't have MPI-3 support? That way we could avoid maintaining multiple code paths for too long but not leave anyone in the lurch.

Oh, and if you guys want to join me in reviewing #1822 ... between vacation and backlog of other things I've barely glanced at it yet, but ideally I'd like to drop support for MPI-2 only after we finish dropping support for MPI-1.

[BalticPinguin]

I like this idea as well.
If you agree on the testing-scheme suggested in #1822 , this can be extended straight forward to test for MPI-3 as well...

[friedmud]

@pbauman I honestly don't know how to figure out which versions support MPI-3. I tried to do some digging today and came up with nothing.

The best way might honestly just bee to try them out with a little test. At least for mpich and openmpi it wouldn't be TOO hard of a bisection to do!

Or maybe just email their mailing lists?

[BalticPinguin]

ok; probably it is good to shift the discussion from #1822 to this thread:
As I see it, the MPI-config currently is done as follows:

1. Try to find PETSc with MPI and take theirs without further checks
2. If this doesn't work, search mpi.h and take this
3. If it is not found, try whether the linker/compiler can find it and if it works, take it as it is.
4. configure without MPI.

According to my understanding,
--> in case of 1 and 3 we cannot assume to have access to the header-file explicitly. But if this works,
one can hope to also have mpirun or mpiexec available (can we assume this safely?) Than the
check mpirun --version (or similar; here further syntax-versions need to be found) gives us the version
--> In case of 2 we can grep for some MPI_VERSION-type string (OMPI_MAJOR_VERSION,
MPICH_VERSION etc.) to find the version.

But probably SO has a smarter idea to do it?

[friedmud]

The problem is figuring out which version of MPICH and OpenMPI actually support MPI-3 (or, as I suspect- what partial MPI-3 portions they support). I just spotted @roystgnr ‘s comments about MPI_VERSION and MPI_SUBVERSION - hopefully those are consistently used...

…

On Tue, Aug 21, 2018 at 12:33 AM Hubert Weißmann ***@***.***> wrote: ok; probably it is good to shift the discussion from #1822 <#1822> to this thread: As I see it, the MPI-config currently is done as follows: 1. Try to find PETSc with MPI and take theirs without further checks 2. If this doesn't work, search mpi.h and take this 3. If it is not found, try whether the linker/compiler can find it and if it works, take it as it is. 4. configure without MPI. According to my understanding, --> in case of 1 and 3 we cannot assume to have access to the header-file explicitly. But if this works, one can hope to also have mpirun or mpiexec available (can we assume this safely?) Than the check mpirun --version (or similar; here further syntax-versions need to be found) gives us the version --> In case of 2 we can grep for some MPI_VERSION-type string ( OMPI_MAJOR_VERSION, MPICH_VERSION etc.) to find the version. But probably SO has a smarter idea to do it? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1827 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA1JMfF7lrqA5lIV5h1yw-2dP_D6FDtnks5uS6nPgaJpZM4WEbTl> .

[pbauman]

@pbauman I honestly don't know how to figure out which versions support MPI-3. I tried to do some digging today and came up with nothing. The best way might honestly just bee to try them out with a little test. At least for mpich and openmpi it wouldn't be TOO hard of a bisection to do! Or maybe just email their mailing lists?

Thanks for taking a look @friedmud. Was just curious if maybe someone had catalogued it or there was institutional knowledge. Agreed we'll just need to test (and should try and parlay that into m4/configure test I think).

[roystgnr]

So it turns out that we (I) already dropped MPI-2 support on May 7, based on my confusion about when MPI fixed API const correctness rather than on any careful discussion.

#1822 (comment)

Note that this postdates our last libMesh release, though. My inclination would have been to fix it, test it, and put out one last libMesh-that-supports-MPI-2 release... but if we still haven't figured out what MPI versions to use for that testing? I think I have an ancient MPICH (1.5?) and an ancient OpenMPI (1.2.9?) that claim MPI_VERSION 2...

But there's no way we're getting tests with those into CI, and without CI tests with MPI-2 we're just going to break it again inadvertently at any time, so yeah, let's drop MPI-2 support. I'll go with whatever others decide on whether we should spend the time it takes to put out one last MPI-2-compatible release first.