Describe the bug
Currently, our flushing logic checks iterations if they are dirty, i.e. have any data that needs to be flushed, and flushes them only then. This does not work reliably in parallel since even an independent write must be flushed collectively.
To Reproduce
No need for a reproducer, the reason for the behavior is known.
Expected behavior
Since we are currently moving towards improving our support for more general Iteration::open()/Iteration::close() workflows, our flushing logic should flush exactly those Iterations that are currently open instead of trying to outsmart the user by just flushing what we think needs to be flushed.
Until then, an intermediate solution would be to expose sth like Iteration::markDirty().
Additional context
Found by @psychocoderHPC
Describe the bug
Currently, our flushing logic checks iterations if they are dirty, i.e. have any data that needs to be flushed, and flushes them only then. This does not work reliably in parallel since even an independent write must be flushed collectively.
To Reproduce
No need for a reproducer, the reason for the behavior is known.
Expected behavior
Since we are currently moving towards improving our support for more general
Iteration::open()/Iteration::close()workflows, our flushing logic should flush exactly those Iterations that are currently open instead of trying to outsmart the user by just flushing what we think needs to be flushed.Until then, an intermediate solution would be to expose sth like
Iteration::markDirty().Additional context
Found by @psychocoderHPC