Provide all available data in `engine.state.output` of workflows

[Nic-Ma]

Is your feature request related to a problem? Please describe.

Hi @ericspod @wyli @rijobro ,

The problem and idea is that:
currently, MONAI post transforms(independent with ignite) execute on the engine.state.output dict, the post transforms can't get some useful information in engine.state.batch, for example, the meta_dict of images or some transform information we put in the engine.state.batch. As @vfdev-5 and I discussed in the ignite ticket: pytorch/ignite#1923, I think we need to shallow-copy the data of engine.state.batch into engine.state.output, then post transforms and handlers can get all the available data from only 1 dict.
The data flow can run through all the components and no need to write 2 lambda funs (batch_transform and output_transform) in handlers, we can use keys to get expected data directly.
I didn't find any benefit to split the data flow into state.batch and state.output in MONAI.
For the first step, I plan to change this line as aligned with @vfdev-5 :
https://github.com/Project-MONAI/MONAI/blob/master/monai/engines/trainer.py#L160
into:

engine.state.output = dict(engine.state.batch)
engine.state.output.update({Keys.IMAGE: inputs, Keys.LABEL: targets})

Then we can use engine.state.output in post transforms and handlers as the overall data flow, and no need to get data from engine.state.batch anymore.

Do you guys have any comments?

Thanks in advance.

[vfdev-5]

Hi @Nic-Ma ,

I'm just wondering what is expected by doing this line : engine.state.output = dict(engine.state.batch) , the structure of batch is a sequence, so we can not construct a dict from it ?

Previously, I had a project where I needed also to apply data transformation inversion to the input batch and as far as I remember, this meta info was passed to output as you are suggesting with :

output = {
    "batch_loss": ...  # usual iteration output
    "meta": batch["meta"],
    "y": batch["mask"]
}

My only point here is that it can be dependent on the application. Otherwise, I have an impression that we have to put requirements on provided input data to be able to dispatch it correctly.

Thanks

[wyli]

I feel MONAI as a user of pytorch ignite should wait for the upstream's sufficient discussions and provide concrete use cases to help them make decisions... this proposal doesn't look like the intended way of using ignite, but I might be wrong here

[Nic-Ma]

Hi @vfdev-5 and @wyli ,

Thanks for the discussion.
This proposal expects the engine.state.batch to be a dict, which associated with dict transforms in MONAI.
I agree if ignite can help discuss this problem and make some change or give us suggestions, that would be great.
@vfdev-5 do you mean to make a CopyProperties handler to manually specify and copy expected data by users?

Thanks.