ResNet
What ResNet Does
ResNet classifies images with a deep stack of convolutional blocks connected by shortcuts. Its residual connections make networks with tens or hundreds of layers substantially easier to optimize than comparable plain networks.
Deep Residual Learning for Image Recognition, by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, introduced the architecture in 2015. A ResNet ensemble won the ILSVRC 2015 classification task with 3.57% top-5 error. This was lower than a reported human baseline on that benchmark, but the figures come from different evaluation conditions and should not be treated as a general claim of superhuman vision.
Core Idea
A residual block learns a change and adds the original input back:
If the desired transformation is close to identity, the residual branch can learn values near zero instead of reconstructing the entire input. The addition also creates a direct path through the block for activations and gradients.
ResNet-50 Architecture
Source: He et al., 2015 — Deep Residual Learning for Image Recognition (CVPR)
Each stage is built from residual blocks. ResNet-50 and above use the bottleneck variant; ResNet-18 and ResNet-34 use the basic block:
ResNet Residual Blocks
Source: He et al., 2015 — Deep Residual Learning for Image Recognition (CVPR)
How It Works
1. Create a base feature map
For ImageNet, the ResNet-50 stem applies a 7 × 7 stride-2 convolution, batch normalization, ReLU, and stride-2 max pooling. A 224 × 224 input becomes a 56 × 56 × 64 feature map.
2. Transform through residual blocks
ResNet-18 and ResNet-34 use a basic block with two 3 × 3 convolutions. ResNet-50, -101, and -152 use a bottleneck:
- A 1 × 1 convolution reduces channels.
- A 3 × 3 convolution processes spatial context.
- A 1 × 1 convolution expands channels.
- The shortcut is added, then ReLU is applied in the original post-activation design.
Batch normalization follows each convolution before its activation. Reducing channels around the 3 × 3 layer controls computation; the bottleneck is not identical in expressive power to an unrestricted full-width block, but it offers an effective capacity–cost trade-off.
3. Match shortcut dimensions when needed
If input and output have the same shape, the shortcut is the identity and adds no parameters. When a stage downsamples or changes channels, a stride-2 1 × 1 projection can align the shortcut with the residual branch.
4. Build the ResNet-50 stages
| Stage | Repeated bottleneck | Repeats | Output |
|---|---|---|---|
| conv2_x | 64 → 64 → 256 | 3 | 56 × 56 × 256 |
| conv3_x | 128 → 128 → 512 | 4 | 28 × 28 × 512 |
| conv4_x | 256 → 256 → 1024 | 6 | 14 × 14 × 1024 |
| conv5_x | 512 → 512 → 2048 | 3 | 7 × 7 × 2048 |
Global average pooling converts the final tensor to 2,048 values, and a fully connected layer produces 1,000 class scores. ResNet-50 has about 25 million parameters, compared with roughly 138 million in VGG-16.
5. Train the network
The ImageNet recipe used SGD with momentum 0.9, weight decay , batch size 256, and a learning rate starting at 0.1 and divided by ten during a 90-epoch schedule. Random crops, horizontal flips, and color augmentation regularize training. He initialization is suited to ReLU networks.
The paper also trained a 1,202-layer residual network on CIFAR-10. It reached low training error but overfit relative to a 110-layer model, so it demonstrated optimization feasibility rather than universal benefit from extreme depth.
Why It Works
- Easier identity mappings: A block can preserve its input by driving its residual branch toward zero.
- Short gradient paths: The additive shortcut supplies routes that do not pass through every residual operation, reducing—but not categorically eliminating—vanishing-gradient problems.
- Controlled computation: Bottlenecks concentrate expensive spatial processing in a narrower channel space.
- Normalization: Batch normalization stabilizes activation scales and supports the original training recipe.
- Composable blocks: The same unit can be repeated to create multiple depths and reused as a backbone for detection and segmentation.
Limitations
- Shortcut connections improve optimization but do not guarantee that every deeper model is more accurate or easy to tune.
- Bottleneck convolutions remain expensive for resource-constrained devices.
- Batch normalization depends on batch statistics during training and adds state and synchronization concerns.
- A classification ResNet alone does not explicitly combine multi-resolution feature maps; systems such as feature pyramid networks add that capability.
- Depth, width, and input resolution are chosen separately rather than through a single scaling rule.
- Newer CNNs and vision transformers may outperform the original ResNets under some training budgets, while ResNet remains a strong baseline and backbone.
Related
- LeNet-5
- AlexNet
- VGGNet
- CNN
- Convolution
- Pooling
- Batch Normalization
- Activation Function
- ResNeXt (2017) — grouped convolutions in bottlenecks
- DenseNet (2017) — dense inter-layer connections
- EfficientNet (2019) — compound scaling
- ConvNeXt (2022) — a modernized convolutional backbone
- ResNet in Vision Transformers — analysis of residual pathways in transformers
Sources
- Deep Residual Learning for Image Recognition — Original ResNet paper by He et al. (CVPR 2016)
- ResNet — Wikipedia
- ILSVRC 2015 Results
- CS231n: Convolutional Neural Networks for Visual Recognition — Stanford course notes
- PyTorch ResNet pretrained models