GoogLeNet / Inception
What GoogLeNet Does
GoogLeNet, also called Inception-v1, classifies images by examining each stage at several spatial scales in parallel. It won the ILSVRC 2014 classification task with 6.67% top-5 error while using roughly 6.8 million parameters—about twenty times fewer than VGG-16.
The architecture was introduced by Christian Szegedy and collaborators in Going Deeper with Convolutions. Its goal was not just to add layers, but to increase depth and width while controlling computation and parameter growth.
Core Idea
An Inception module sends one input through four branches: a 1 × 1 convolution, reduced 3 × 3 and 5 × 5 convolution paths, and a pooling path. It concatenates all branch outputs across channels. The network can therefore preserve evidence gathered with different receptive fields at the same stage.
The core building block is the Inception module, which applies multiple filter sizes in parallel:
Inception Module with dimensionality reduction
Source: Szegedy et al., 2014 — Going Deeper with Convolutions (CVPR)
How It Works
1. Prepare features in the stem
A 224 × 224 RGB image first passes through a 7 × 7 convolution, pooling, a 1 × 1 convolution, a 3 × 3 convolution, and another pool. The result is a 28 × 28 × 192 feature tensor.
2. Process each tensor at multiple scales
Each of the network's nine Inception modules applies four parallel paths:
| Branch | Operations | Role |
|---|---|---|
| 1 | 1 × 1 convolution | Mix channels cheaply |
| 2 | 1 × 1 reduction → 3 × 3 convolution | Capture local spatial patterns |
| 3 | 1 × 1 reduction → 5 × 5 convolution | Capture broader patterns |
| 4 | 3 × 3 max pool → 1 × 1 projection | Preserve pooled context |
The 1 × 1 reductions shrink channel depth before expensive spatial filters. They save computation and add another learned nonlinear transformation; the exact saving depends on branch dimensions rather than being a universal fixed factor.
3. Stack modules and reduce resolution
| Stage | Modules or operation | Output |
|---|---|---|
| Stem | Convolutions and pooling | 28 × 28 × 192 |
| Inception 3a–3b | 2 modules | 28 × 28 × 480 |
| Pool | 3 × 3 / 2 | 14 × 14 × 480 |
| Inception 4a–4e | 5 modules | 14 × 14 × 832 |
| Pool | 3 × 3 / 2 | 7 × 7 × 832 |
| Inception 5a–5b | 2 modules | 7 × 7 × 1024 |
The original paper counts GoogLeNet as 22 layers deep when counting layers with parameters along the longest path.
4. Train with auxiliary heads
Two auxiliary classifiers branch from intermediate features during training. Their weighted losses add gradient signals and regularization, but the heads are removed for ordinary inference. Their benefit is not simply that every gradient bypasses the full network; later implementations found the regularization effect especially relevant.
5. Replace a large dense classifier
Global average pooling turns the 7 × 7 × 1024 tensor into 1,024 values by averaging each channel. Dropout with probability 0.4 and one final linear layer produce 1,000 class scores. This avoids the parameter-heavy fully connected stacks used by AlexNet and VGG.
The reported training setup used SGD with momentum 0.9, weight decay , batch size 128, extensive crop and color augmentation, and a decaying learning rate.
Why It Works
- Parallel receptive fields: Fine and broad patterns remain available without committing the whole stage to one kernel size.
- Learned bottlenecks: 1 × 1 convolutions control channel count before costly operations.
- Sparse-inspired structure: Dense branch operations approximate the paper's motivation of selecting useful local groupings while remaining efficient on available hardware.
- Compact head: Global average pooling greatly reduces classifier parameters and overfitting pressure.
- Balanced capacity: More depth and width are allocated inside a fixed computational budget. The 2014 result demonstrated that this design was effective; it does not prove parallel multi-scale extraction always outperforms every single-scale design.
Inception Family
| Version | Main change |
|---|---|
| Inception-v1 | Original branches and auxiliary classifiers |
| Inception-v2/v3 | Factorized convolutions, batch normalization, and later label smoothing |
| Inception-v4 | More regular Inception-A/B/C modules |
| Inception-ResNet | Inception branches combined with residual connections |
| Xception | Depthwise separable convolutions framed as an extreme Inception design |
Later variants factorized 5 × 5 filters into two 3 × 3 filters and filters into then , reducing computation while retaining an effective receptive field.
Limitations
- Branch widths, reductions, and filter allocations create many architectural hyperparameters.
- Heterogeneous parallel paths are harder to implement, inspect, and optimize than uniform sequential blocks.
- Auxiliary classifiers complicate training even though they are omitted at inference.
- Nominally low parameter count does not imply proportionally low compute; GoogLeNet still performs many convolutions.
- Later Inception versions, residual networks, and mobile-focused designs improved particular accuracy, training, or deployment trade-offs.
Related
- AlexNet
- VGGNet
- ResNet
- MobileNet
- EfficientNet
- CNN
- Convolution
- Pooling
- Inception-v3 (2015) — factorized convolutions and batch normalization
- Inception-ResNet (2016) — Inception with residual connections
- Xception (2017) — depthwise separable convolutions
- EfficientNet (2019) — compound scaling
Sources
- Going Deeper with Convolutions — Original GoogLeNet paper by Szegedy et al. (CVPR 2015)
- Rethinking the Inception Architecture for Computer Vision — Inception-v3 by Szegedy et al. (CVPR 2016)
- Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning — Szegedy et al. (AAAI 2017)
- Xception: Deep Learning with Depthwise Separable Convolutions — Chollet (CVPR 2017)
- GoogLeNet — Wikipedia
- ILSVRC 2014 Results
- PyTorch GoogLeNet pretrained models