Skip to content

337 Add config tests#400

Merged
yiheng-wang-nv merged 45 commits into
Project-MONAI:devfrom
yiheng-wang-nv:add-config-tests
Jun 2, 2023
Merged

337 Add config tests#400
yiheng-wang-nv merged 45 commits into
Project-MONAI:devfrom
yiheng-wang-nv:add-config-tests

Conversation

@yiheng-wang-nv

@yiheng-wang-nv yiheng-wang-nv commented Apr 28, 2023

Copy link
Copy Markdown
Collaborator

Fixes #337 .

Description

This PR adds unit tests scripts, as well as test files on brats segmentation and spleen ct segmentation bundles. Tests are for both single GPU and multiple GPUs.
In addition, the PR also enhances the current Blossom CI pipelines. For PRs that do not need to use GPU or multiple GPUs, the pipeline will not wait for resources first.

Status

Ready

Please ensure all the checkboxes:

  • Codeformat tests passed locally by running ./runtests.sh --codeformat.
  • In-line docstrings updated.
  • Update version and changelog in metadata.json if changing an existing bundle.
  • Please ensure the naming rules in config files meet our requirements (please refer to: CONTRIBUTING.md).
  • Ensure versions of packages such as monai, pytorch and numpy are correct in metadata.json.
  • Descriptions should be consistent with the content, such as eval_metrics of the provided weights and TorchScript modules.
  • Files larger than 25MB are excluded and replaced by providing download links in large_file.yml.
  • Avoid using path that contains personal information within config files (such as use /home/your_name/ for "bundle_root").

@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

yiheng-wang-nv and others added 3 commits April 28, 2023 22:54
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

todo:

all test on evaluation, mgpu train, mgpu evaluation, inference, determinism train.

@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

yiheng-wang-nv and others added 2 commits May 5, 2023 11:03
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv yiheng-wang-nv marked this pull request as ready for review May 6, 2023 09:08
@yiheng-wang-nv yiheng-wang-nv requested review from KumoLiu, Nic-Ma and wyli May 6, 2023 09:09
Comment thread ci/utils.py Outdated
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

Comment thread models/spleen_ct_segmentation/configs/metadata.json Outdated
Comment thread ci/utils.py Outdated
Comment thread ci/config_test_cases.py Outdated
Comment thread ci/config_test_cases.py Outdated
Comment thread requirements.txt Outdated
@Nic-Ma

Nic-Ma commented May 6, 2023

Copy link
Copy Markdown
Collaborator

I would suggest to present your CI testing solution in the dev meeting for discussion first.
@wyli @ericspod What do you think?

Thanks.

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

1 similar comment
@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

yiheng-wang-nv and others added 2 commits June 1, 2023 15:16
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

9 similar comments
@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

@YanxuanLiu YanxuanLiu changed the title 337 Add config tests 337 Add config tests [skip gpu] Jun 2, 2023
@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

1 similar comment
@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

@YanxuanLiu YanxuanLiu changed the title 337 Add config tests [skip gpu] 337 Add config tests Jun 2, 2023
@YanxuanLiu

Copy link
Copy Markdown
Collaborator

/build

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv

Copy link
Copy Markdown
Collaborator Author

/build

@yiheng-wang-nv yiheng-wang-nv merged commit e05b097 into Project-MONAI:dev Jun 2, 2023
@yiheng-wang-nv yiheng-wang-nv deleted the add-config-tests branch June 2, 2023 09:38
yiheng-wang-nv added a commit to yiheng-wang-nv/model-zoo that referenced this pull request Jul 29, 2024
Fixes Project-MONAI#337 .

### Description
This PR adds unit tests scripts, as well as test files on brats
segmentation and spleen ct segmentation bundles. Tests are for both
single GPU and multiple GPUs.
In addition, the PR also enhances the current Blossom CI pipelines. For
PRs that do not need to use GPU or multiple GPUs, the pipeline will not
wait for resources first.

### Status
**Ready**

### Please ensure all the checkboxes:
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Codeformat tests passed locally by running `./runtests.sh
--codeformat`.
- [ ] In-line docstrings updated.
- [ ] Update `version` and `changelog` in `metadata.json` if changing an
existing bundle.
- [ ] Please ensure the naming rules in config files meet our
requirements (please refer to: `CONTRIBUTING.md`).
- [ ] Ensure versions of packages such as `monai`, `pytorch` and `numpy`
are correct in `metadata.json`.
- [ ] Descriptions should be consistent with the content, such as
`eval_metrics` of the provided weights and TorchScript modules.
- [ ] Files larger than 25MB are excluded and replaced by providing
download links in `large_file.yml`.
- [ ] Avoid using path that contains personal information within config
files (such as use `/home/your_name/` for `"bundle_root"`).

---------

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhance CI test to check more configs of each bundle

6 participants