Auto3D Task Module 1 (DataAnalyzer) for #4743

[mingxin-zheng]

Description

Implemented a DataAnalyzer class to encapsulate data analysis module. As the beginning part of the auto3d/automl pipeline, the module shall find data and label from user inputs and generate a summary (dictionary) of data stats. The summary includes
- file names, list, number of files;
- dataset summary (basic information, image dimensions, number of classes, etc.);
- individual data information (spacing, image size, number and size of the regions, etc.).
The summary can be exported as a YAML file and a dictionary variable for use in Python

Example Usage:

from monai.apps.auto3d.data_analyzer import DataAnalyzer

datalist = {
    "testing": [{"image": "image_003.nii.gz"}],
    "training": [
        {"fold": 0, "image": "image_001.nii.gz", "label": "label_001.nii.gz"},
        {"fold": 0, "image": "image_002.nii.gz", "label": "label_002.nii.gz"},
        {"fold": 1, "image": "image_001.nii.gz", "label": "label_001.nii.gz"},
        {"fold": 1, "image": "image_004.nii.gz", "label": "label_004.nii.gz"},
    ],
}

dataroot = '/datasets' # the directory where you have the image files (in this example we're using nii.gz)
analyser = DataAnalyzer(datalist, dataroot)
datastat = analyser.get_all_case_stats() # it will also generate a data_stats.yaml that saves the stats

Status

Ready for Review
Reference issue #4743.

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• New tests added to cover the changes.
• Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[wyli]

thanks, there are some style issues reported, I think some of them are valid ones... https://deepsource.io/gh/Project-MONAI/MONAI/run/96404a6f-933e-4971-b866-f170efb1d4b9/python/

[mingxin-zheng]

Thanks @wyli . Still ramping up myself on the the MONAI style coding. I will fix those.

[Nic-Ma]

Hi @mingxin-zheng @dongyang0122 @wyli ,

I think the current pipeline can only work for 3D segmentation, should we call the module "auto3d_seg" or "auto_seg_3d"?

Thanks.

[Nic-Ma]

Hi @mingxin-zheng ,

Thanks for the contribution about this highlight feature.
I put some comments for the high level review, haven't checked the computation details.
May need @dongyang0122 @wyli to also help review it.

Thanks in advance.

[mingxin-zheng]

thanks, there are some style issues reported, I think some of them are valid ones... https://deepsource.io/gh/Project-MONAI/MONAI/run/96404a6f-933e-4971-b866-f170efb1d4b9/python/

Most style issues are fixed now except two pending (https://deepsource.io/gh/Project-MONAI/MONAI/run/0422e6b3-e5b3-4830-9cd3-f460969470b0/python/PYL-W0106)

[mingxin-zheng]

Hi @mingxin-zheng I put some doc page updates here mingxin-zheng#1 please add them to the PR so that the docstrings will be part of the doc website

Thanks @wyli ! I have merged it to the PR code.

[Nic-Ma]

Thanks for the quick update.
Looks good to me now.
@wyli Do you have any other comments? Especially the code folder structure.

Thanks.

[wyli]

cool, it looks good to me as well.

[wyli]

/build

[wyli]

/build

[wyli]

/build

[wyli]

/build