What is Batch Normalization?
TL;DR
A normalization technique that stabilizes and accelerates neural network training. A standard technique in deep learning.
Batch Normalization: Definition & Explanation
Batch normalization is a technique that normalizes the inputs to each layer of a neural network (bringing them closer to mean 0, variance 1) to stabilize and speed up training. Proposed by Google's Ioffe and Szegedy in 2015, it has become a standard technique in deep learning. During training, normalization is performed per mini-batch, reducing internal covariate shift (fluctuations in input distribution across layers). This allows for larger learning rates and faster convergence. It also provides a regularization effect that helps prevent overfitting. It is used in many modern deep learning architectures including image recognition models (ResNet, EfficientNet) and Transformer variants.