A Systematic Survey of Regularization and Normalization in GANs

A Survey and Taxonomy of the Regularization and Normalization in GANs. Please refer to the details in recent review paper A Systematic Survey of Regularization and Normalization in GANs. Ziqiang Li, Xintian Wu, Muhammad Usman, Rentuo Tao, Pengfei Xia, Huanhuan Chen, and Bin Li.

If you find this survey is useful in your research, please consider citing:

@article{li2020regularization,
  title={Regularization and Normalization For Generative Adversarial Networks: A Survey},
  author={Li, Ziqiang and Tao, Rentuo and Xia, Pengfei and Chen, Huanhuan and Li, Bin},
  journal={arXiv preprint arXiv:2008.08930},
  year={2020}
}

A table of Regularization and Normalization in Generative adversarial Networks (GANs). This page will be periodically updated to include recent works. Let us know if your recent work is not in the list, we will be happy to include it!

List of Regularization and Normalization in GANs

The most important studies will be highlighted.

Perspective of "Real & Fake" in GANs

Year	Title	Type	Venue	Link	Code
2021	Are High-Frequency Components Beneficial for Training of Generative Adversarial Networks	Data Preprocessing	ArXiv	Link	Code
2020	Training generative adversarial networks with limited data	Data Augmentation	NeurIPS	Link	Code
2020	On Data Augmentation for GAN Training	Data Augmentation	TIP	Link	Code
2020	Differentiable augmentation for data-efficient gan training	Data Augmentation	NeurIPS	Link	Code
2020	Image Augmentations for GAN Training	Data Augmentation	NeurIPS	Link	Code
2021	Manifold-preserved GANs	Image Consistency Regularization	ArXiv	Link
2021	Manifold Matching via Deep Metric Learning for Generative Modeling	Image Consistency Regularization	ICCV	Link	Code
2021	SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains	Image Consistency Regularization	AAAI	Link	Code
2020	Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions	Image Consistency Regularization	CVPR	Link	Code
2018	MMGAN: Manifold-Matching Generative Adversarial Network	Image Consistency Regularization	ICPR	Link
2017	Mcgan: Mean and covariance feature matching gan	Image Consistency Regularization	ICML	Link
2016	Improved techniques for training gans	Image Consistency Regularization	NeurIPS	Link	Code
2021	Improved Consistency Regularization for GANs	Network Consistency Regularization	AAAI	Link
2020	Consistency Regularization for Generative Adversarial Networks	Network Consistency Regularization	ICLR	Link
2020	Augmented Cyclic Consistency Regularization for Unpaired Image-to-Image Translation	Network Consistency Regularization	ICPR	Link
2021	LT-GAN: Self-Supervised GAN with Latent Transformation Detection	Predictive Self-Supervised	WACV	Link
2020	FX-GAN: Self-Supervised GAN Learning via Feature Exchange	Predictive Self-Supervised	WACV	Link
2020	DeshuffleGAN: A Self-Supervised GAN to Improve Structure Learning	Predictive Self-Supervised	ICIP	Link	Code
2019	Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game	Predictive Self-Supervised	NeurIPS	Link	Code
2019	Self-supervised gans via auxiliary rotation loss	Predictive Self-Supervised	CVPR	Link	Code
2021	Training GANs with Stronger Augmentations via Contrastive Discriminator	Contrastive Self-Supervised	ICLR	Link	Code
2021	InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning	Contrastive Self-Supervised	WACV	Link	Code
2021	Data-Efficient Instance Generation from Instance Discrimination	Contrastive Self-Supervised	NeurIPS	Link	Code

Perspective of "Fitting distribution" in GANs

Year	Title	Type	Lipschitz continuity	Venue	Link	Code
2020	ADVERSARIAL LIPSCHITZ REGULARIZATION	Gradient Penalty	1	ICLR	Link	Code
2019	Towards Efficient and Unbiased Implementation of Lipschitz Continuity in GANs	Gradient Penalty	1	IJCAI	Link	Code
2018	Banach Wasserstein GAN	Gradient Penalty	1	NeurIPS	Link	Code
2017	Improved Training of Wasserstein GANs	Gradient Penalty	1	NeurIPS	Link	Code
2017	On Convergence and Stability of GANs	Gradient Penalty	1	ArXiv	Link	Code
2021	Towards Generalized Implementation of Wasserstein Distance in GANs	Gradient Penalty	k (k<=1)	AAAI	Link	Code
2018	On the regularization of Wasserstein GANs	Gradient Penalty	k (k<=1)	ICLR	Link	Code
2020	Direct Adversarial Training: An Adaptive Method to Penalize Lipschitz Continuity of the Discriminator	Gradient Penalty	0	ArXiv	Link	Code
2019	Improving Generalization and Stability of Generative Adversarial Networks	Gradient Penalty	0	ICLR	Link	Code
2019	Lipschitz Generative Adversarial Nets	Gradient Penalty	0	ICML	Link	Code
2018	Which Training Methods for GANs do actually Converge?	Gradient Penalty	0	ICML	Link	Code
2018	Wasserstein Divergence for GANs	Gradient Penalty	0	ECCV	Link	Code
2018	A Wasserstein GAN model with the total variational regularization	Gradient Penalty	0	ArXiv	Link
2018	GAN-QP: A Novel GAN Framework without Gradient Vanishing and Lipschitz Constraint	Gradient Penalty	0	ArXiv	Link	Code
2021	Gradient Normalization for Generative Adversarial Networks	Gradient Normalization	1	ICCV	Link	Code
2020	Spectral bounding: Strictly satisfying the 1-Lipschitz property for generative adversarial networks	Weight Normalization	k (k<=1)	PR	Link
2019	Large scale gan training for high fidelity natural image synthesis	Weight Normalization		ICLR	Link	Code
2019	Spectral Regularization for Combating Mode Collapse in GANs	Weight Normalization	1	ICCV	Link	Code
2018	Spectral normalization for generative adversarial networks	Weight Normalization	1	ICLR	Link	Code

Perspective of "Training Dynamics" in GANs

Year	Title	Type	Venue	Link	Code
2019	Towards a Better Understanding and Regularization of GAN Training Dynamics	Jacobian Regularization	UAI	Link	Code
2018	Which Training Methods for GANs do actually Converge?	Gradient Penalty	ICML	Link	Code
2017	The numerics of gans	Jacobian regularization	NeurIPS	Link	Code
2017	Gradient descent GAN optimization is locally stable	Jacobian regularization	NeurIPS	Link	Code

Others

Year	Title	Type	Venue	Link	Code
2017	On the effects of batch and weight normalization in generative adversarial networks	Unconditional Layer Normalization	ArXiv	Link	Code
2020	Attentive Normalization for Conditional Image Generation	conditional Layer Normalization	CVPR	Link	Code
2019	A style-based generator architecture for generative adversarial networks	conditional Layer Normalization	CVPR	Link	Code
2019	Semantic Image Synthesis with Spatially-Adaptive Normalization	conditional Layer Normalization	CVPR	Link	Code
2018	cGANs with projection discriminator	conditional Layer Normalization	ICLR	Link	Code
2019	Diversity-Sensitive Conditional GANs	Inverse Gradient Penalty	ICLR	Link	Code
2018	Is Generator Conditioning Causally Related to GAN Performance?	Inverse Gradient Penalty	ICML	Link	Code
2021	Regularizing Generative Adversarial Networks under Limited Data	Discriminator Deceleration	CVPR	Link	Code

Authors

This page is managed and maintained by:

• Ziqiang Li iceli@mail.ustc.edu.cn
• Yi Wu wuyi2021@mail.ustc.edu.cn
• Hong Sun hsun777@mail.ustc.edu.cn