GradualWarmupScheduler

[wyli]

Is your feature request related to a problem? Please describe.
This is a feature request for GradualWarmupScheduler -- gradually increasing the learning rate described in "Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour" https://arxiv.org/abs/1706.02677

Describe the solution you'd like
a light weight module in https://github.com/Project-MONAI/MONAI/blob/dev/monai/optimizers/lr_scheduler.py

[vfdev-5]

@wyli we recently moved our optimizer param scheduler to the core from contrib (thus available for you as well). This module contains a lot of different patterns on how to schedule lr, momentum etc. https://pytorch.org/ignite/master/handlers.html#parameter-scheduler
Maybe, it can already cover this feature request. Please, let me know what do you think.

[wyli]

thanks @vfdev-5! I was not aware of these recent features. I'm labelling this as an upstream fix.

btw I feel the example of create_lr_scheduler_with_warmup could have a better start value? the current warmup_start_value=0.0 seems to have no model updates for the first iteration.

[wyli]

while we prefer the ignite verision if it's available, it is also addressed by #2545