Skip to content

cron tutorial notebooks#1783

Merged
wyli merged 17 commits into
Project-MONAI:masterfrom
rijobro:cron_tutorials
Mar 22, 2021
Merged

cron tutorial notebooks#1783
wyli merged 17 commits into
Project-MONAI:masterfrom
rijobro:cron_tutorials

Conversation

@rijobro

@rijobro rijobro commented Mar 16, 2021

Copy link
Copy Markdown
Contributor

Run tutorial notebooks as cron job. Not sure if github actions syntax is correct.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).

Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
@rijobro rijobro requested review from Nic-Ma, ericspod and wyli March 16, 2021 16:44
Comment thread .github/workflows/cron.yml Outdated
Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
@ericspod ericspod enabled auto-merge (squash) March 16, 2021 18:10

@wyli wyli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks I'll test it and get back to this PR

wyli added 4 commits March 16, 2021 20:02
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
This reverts commit dcc6656.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli

wyli commented Mar 16, 2021

Copy link
Copy Markdown
Contributor

the script's syntax is fine now, but some of the notebooks are not ready https://github.com/Project-MONAI/MONAI/runs/2125064870?check_suite_focus=true#step:6:154

@rijobro

rijobro commented Mar 16, 2021

Copy link
Copy Markdown
Contributor Author

Not sure how to resolve this, I don't see it locally: ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).

@wyli

wyli commented Mar 16, 2021

Copy link
Copy Markdown
Contributor

Not sure how to resolve this, I don't see it locally: ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).

I see, I'll try to set --shm-size of the docker container tomorrow

rijobro added 2 commits March 17, 2021 08:59
Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
Comment thread .github/workflows/cron.yml Outdated
@rijobro

rijobro commented Mar 17, 2021

Copy link
Copy Markdown
Contributor Author

Tests made it further this time, but still running out of memory: RuntimeError: DataLoader worker (pid 11596) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.

@rijobro

rijobro commented Mar 17, 2021

Copy link
Copy Markdown
Contributor Author

@wyli Committing from a github suggestion seems to make DCO fail...

rijobro added 2 commits March 17, 2021 09:51
Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
wyli and others added 5 commits March 17, 2021 12:27
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Richard Brown <33289025+rijobro@users.noreply.github.com>
This reverts commit f8e9d6f.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli

wyli commented Mar 17, 2021

Copy link
Copy Markdown
Contributor

now having a different issue with the tensorboard magic line: https://github.com/Project-MONAI/MONAI/runs/2130772126?check_suite_focus=true#step:6:298 any idea?

@rijobro

rijobro commented Mar 17, 2021

Copy link
Copy Markdown
Contributor Author

There's info online about cgi.escape being deprecated in favour of html.escape:

cgi.escape which was deprecated in Python 3.7 and removed in 3.8.

Not sure what that means for solving this problem. Perhaps using python3.7? Or maybe it would be solved with pip install -U <package>, but I'm not sure which package(s) need to be updated -- jupyter?

@wyli

wyli commented Mar 17, 2021

Copy link
Copy Markdown
Contributor

updating tensorboard resolves that issue. now we have a testing data issue https://github.com/Project-MONAI/MONAI/runs/2132482457?check_suite_focus=true#step:6:317 need to host the zip file for testing?

@rijobro rijobro added the enhancement New feature or request label Mar 18, 2021
@rijobro

rijobro commented Mar 19, 2021

Copy link
Copy Markdown
Contributor Author

@ericspod The current error comes from data in the covid notebook not being automatically download-able from Kaggle. Do you think we could host a copy on Zenodo or somewhere else?

@ericspod

Copy link
Copy Markdown
Member

Yeah that's fine, it's easier that way for sure.

auto-merge was automatically disabled March 19, 2021 15:27

Pull request was closed

@rijobro rijobro reopened this Mar 19, 2021
@wyli

wyli commented Mar 19, 2021

Copy link
Copy Markdown
Contributor

I feel we could merge this after addressing Project-MONAI/tutorials#144, so that these nightly tests would at least give us some useful information about all the tutorials...

@wyli wyli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, this is a nice follow-up on Project-MONAI/tutorials#101, nightly tutorial tests will be scheduled using this repo's runner https://github.com/Project-MONAI/MONAI/actions/runs/661446080/workflow

@wyli wyli enabled auto-merge (squash) March 22, 2021 16:19
@wyli wyli merged commit 06e1e99 into Project-MONAI:master Mar 22, 2021
@rijobro rijobro deleted the cron_tutorials branch March 22, 2021 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants