Cocktail hotfixes#245
Merged
ravinkohli merged 6 commits intorefactor_development_regularization_cocktailsfrom Jun 3, 2021
Merged
Cocktail hotfixes#245ravinkohli merged 6 commits intorefactor_development_regularization_cocktailsfrom
ravinkohli merged 6 commits intorefactor_development_regularization_cocktailsfrom
Conversation
ArlindKadra
commented
Jun 2, 2021
- Updated the dropout implementation, since the dropout rates were not calculated correctly.
- Fixed the dropout implementation.
- Removed the flatten layer in NoHead.
- Updated the normalization procedure in the backbones since the addition of conditional skip connection broke things.
ravinkohli
reviewed
Jun 3, 2021
autoPyTorch/pipeline/components/setup/network_backbone/ShapedResNetBackbone.py
Show resolved
Hide resolved
ravinkohli
reviewed
Jun 3, 2021
ravinkohli
reviewed
Jun 3, 2021
ravinkohli
reviewed
Jun 3, 2021
autoPyTorch/pipeline/components/setup/network_backbone/ResNetBackbone.py
Show resolved
Hide resolved
ravinkohli
suggested changes
Jun 3, 2021
Contributor
ravinkohli
left a comment
There was a problem hiding this comment.
Hey, thanks for the PR. After merging this PR, we should see a considerable improvement over the current results. I have requested a few small changes. One of them is, maybe you can fix the skip connection bug in the ResBlock module which we discussed on mattermost.
ravinkohli
reviewed
Jun 3, 2021
| # we will never apply the start_norm for the first block, | ||
| # which is why we should account for this case. | ||
| if not self.config['use_skip_connection']: | ||
| layers.append(nn.BatchNorm1d(in_features)) |
Contributor
There was a problem hiding this comment.
should this not have a condition to check if we should use batch norm?
Author
There was a problem hiding this comment.
Ups, I missed it. Very good catch.
ArlindKadra
commented
Jun 3, 2021
|
|
||
| dropout_shape = [ | ||
| dropout / 1000 * self.config["max_dropout"] for dropout in dropout_shape | ||
| dropout for dropout in dropout_shape |
Author
There was a problem hiding this comment.
Now that I see it, this is actually the same list.
ravinkohli
approved these changes
Jun 3, 2021
Merged
ravinkohli
pushed a commit
that referenced
this pull request
Dec 8, 2021
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments
ravinkohli
pushed a commit
that referenced
this pull request
Dec 8, 2021
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
that referenced
this pull request
Dec 21, 2021
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
that referenced
this pull request
Jan 24, 2022
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
that referenced
this pull request
Jan 28, 2022
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
that referenced
this pull request
Feb 28, 2022
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
that referenced
this pull request
Feb 28, 2022
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
that referenced
this pull request
Mar 9, 2022
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
to ravinkohli/Auto-PyTorch
that referenced
this pull request
Apr 12, 2022
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
ravinkohli
pushed a commit
that referenced
this pull request
Jul 26, 2022
* Fixes for the development branch and regularization cocktails * Update implementation * Fix unit tests temporarily * Implementation update and bug fixes * Removing unecessary code * Addressing Ravin's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments [refactor] Address Shuhei's comments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.