Meeting minutes

[philbucher]

small issue to document things we discussed in meetings

@pooyan-dadvand

[philbucher]

Meeting 14.04.2021

• we agreed to start with the MPI testing environment and to make it work in the CI
• also we should check if we can directly use the DataCommunicator from Kratos as we will need those capabilities too
• Meanwhile I am working on the SubComm things in Kratos, they are almost all in master
• also I am splitting the MPI-things in the MappingApp into a separate binary (same as is done e.g. in the FluidDynApp)

[philbucher]

Preparation for next meeting:

• Using CoSiImIO in the same program but from different compilation units/different libs (e.g. from KratosCore and StructuralMechanics). I am currently not sure if it works, esp since Kratos has quite some trouble with static at the moment. This is the potentially problematic line:
  

  CoSimIO/co_sim_io/impl/co_sim_io_impl.hpp

  Line 33 in 401f53e

  static std::unordered_map<std::string, std::unique_ptr<Connection>> s_co_sim_connections;

  
  I haven't done much research yet, so not sure abt the details. Here are some thoughts though:
  • C++17 can have static inline for variables, maybe that could help
  • maybe using extern for all but one compilation unit where the CoSimIO is included could work
    Note the C and Python should not have this problem as there the static map is in a lib. But we should discuss it to be sure
• How to properly test this situation?
• MPI:
  • [MPI/Testing] adding test framework for impl MPI tests #172 adds the framework for MPI tests in C++
  • How should the interface look like for MPI? I would pass a MPI_Comm somehow. But how to do it in a way such that we don't have the #ifdef USING_MPI everywhere?
• port CodeLocation from Kratos for improved debugging? Probaly also requires KRATOS_TRY & KRATOS_CATCH to be useful
• Prefix all CoSimIO CMake options with CO_SIM_IO (or COSIMIO?)?

[pooyan-dadvand]

It is definitely a problem....

I think we may give up to be header-only but let's make research before.

[philbucher]

Meeting 26.04.2021

• Discussion whether or not to drop header-only. Reasons for dropping it are:
  • the registration (see above) when including from different dlls
  • the MPI extension, having it header only would probably require quite some #ifdef USING_MPI. If we have it not header only then we can have a separate extension (like e.g. the KratosMPICore
• I will set up tests when including from different dlls the CoSimIO testing framework. => also when libraries are compiled as shared/static. Note to test also different configurations/hierarchies of dependencies (A.so -> B.so / A.so -> B.so -> C.so / B.so -> A.so <- C.so)
• MPI: If possible keep the same interface for serial and MPI and use the Settings to distinguish btw serial/MPI
• Prefix all of the CoSimIO cmake vars with CO_SIM_IO_ to avoid conflicts (probably some need to be made backward compatible)

@pooyan-dadvand please add if I forgot sth

[pooyan-dadvand]

Adding that we also reviewed the status of the development

[philbucher]

@pooyan-dadvand I think I found out how pybind does the registration across modules. If I understand they use python to create a unique storage:
https://github.com/pybind/pybind11/blob/master/include/pybind11/detail/internals.h#L255-L312

[philbucher]

Meeting 03.05.2021

• updated status of developments
• discussing MPI interface
  • We agreed to create an MPI-Extension header (that gets included if CO_SIM_IO_USE_MPI is defined) so that we don't have to leak mpi-includes into the serial part
  • Advantage is that this way we can cleanly separate the parts that include MPI from that parts that don't => Very useful when separate binaries exist! (think Core/MPICore)
  • This also means to create a separate Connect-fct: ConnectMPI which in addition to the serial version accepts an MPI_Comm to be used by the CoSimIO
  • We also said that MPI must be initialized before calling ConnectMPI, otherwise we throw an error => CoSimIO should NOT initialize MPI, this must be done by the host!
• reviewed PR related to MPI in CI
• MPI C++ CI is done, next step is to do the same for C & Python and add some dummy tests

[philbucher]

Preparation for next meeting:

• [Mapping] deprecate mesh conformity check Kratos#8709
• [MPI] adding C++ interface #179
  • esp we have to link everything to MPI when using it, even the serial test, which I don't like but not sure if we can avoid it
  • I like that the MPI is completely separated :)
• [C] initial add of MPI testing infrastructure #180 => not really sth to discuss, seems to work well :) Probably once we add more tests we will see how well it works
• Python MPI Interface:
  • Have our own small module for init and finalization of MPI OR use mpi4py
  • pass the MPI_Comm from Python will be interesting to solve => we should know what the using applications do!
  • Probably useful to get the Comm from mpi4py => https://stackoverflow.com/questions/52657173/sharing-an-mpi-communicator-using-pybind11

[philbucher]

Meeting 07.05.2021

• MPI Interfaces ([MPI] adding C++ interface #179) were discussed and agreed upon
• Passing the MPI_Comm in Python is not yet clear, will be seen as we go along in the implementation
• support for Penta elements for mapping would be good! However this will require modifications in Kratos..

[philbucher]

Preparation for next meeting:

• Using CoSimIO in different TU / from other (linked) libraries #184
• [Mapping] deprecate mesh conformity check Kratos#8709
• Storage in ModelPart => leave as is but provide faster interfaces
• Save&Load for DataContainer & ModelPart #160 => directly use serializer from Kratos?
• Porting documentation to github pages #186
• Using intrusive_ptr for Node & Element #182 is ready
• Adding CodeLocation to Exception #187
• For FileComm: Serializing ModelPart so that we don't have to go through vtk (which doesn't support all the geometry types we need)
• Pooyans feedback for Documentation

[philbucher]

for async communication:
three options:

• sync (default)
• async_buffered: copy buffer to own data structure so that user can directly reuse the buffer
• async_unbuffered: requires wait. Most efficient but more difficult to use

CoSimIO::Info export_settings;
CoSimIO::Info export_info = CoSimIO::ExportData(export_settings, some_data);


CoSimIO::Info wait_info = CoSimIO::Wait(export_info);

CoSimIO::Info wait_all_info = CoSimIO::WaitAll();

Import to be sync for now. Async import only useful when doing stuff in the middle

[philbucher]

Serialization:
Port Serializer from Kratos and always serialize things for data exchange => also means to drop vtk for FileComm

[philbucher]

Could a raw array be directly serialized?
=> see the serialization in the DataContainer for RawMemory