GoogLeNet / Inception

What GoogLeNet Does

GoogLeNet, also called Inception-v1, classifies images by examining each stage at several spatial scales in parallel. It won the ILSVRC 2014 classification task with 6.67% top-5 error while using roughly 6.8 million parameters—about twenty times fewer than VGG-16.

The architecture was introduced by Christian Szegedy and collaborators in Going Deeper with Convolutions. Its goal was not just to add layers, but to increase depth and width while controlling computation and parameter growth.

Core Idea

An Inception module sends one input through four branches: a 1 × 1 convolution, reduced 3 × 3 and 5 × 5 convolution paths, and a pooling path. It concatenates all branch outputs across channels. The network can therefore preserve evidence gathered with different receptive fields at the same stage.

The core building block is the Inception module, which applies multiple filter sizes in parallel:

Inception Module

Inception Module with dimensionality reduction

Source: Szegedy et al., 2014 — Going Deeper with Convolutions (CVPR)

How It Works

1. Prepare features in the stem

A 224 × 224 RGB image first passes through a 7 × 7 convolution, pooling, a 1 × 1 convolution, a 3 × 3 convolution, and another pool. The result is a 28 × 28 × 192 feature tensor.

2. Process each tensor at multiple scales

Each of the network's nine Inception modules applies four parallel paths:

Branch Operations Role
1 1 × 1 convolution Mix channels cheaply
2 1 × 1 reduction → 3 × 3 convolution Capture local spatial patterns
3 1 × 1 reduction → 5 × 5 convolution Capture broader patterns
4 3 × 3 max pool → 1 × 1 projection Preserve pooled context

The 1 × 1 reductions shrink channel depth before expensive spatial filters. They save computation and add another learned nonlinear transformation; the exact saving depends on branch dimensions rather than being a universal fixed factor.

3. Stack modules and reduce resolution

Stage Modules or operation Output
Stem Convolutions and pooling 28 × 28 × 192
Inception 3a–3b 2 modules 28 × 28 × 480
Pool 3 × 3 / 2 14 × 14 × 480
Inception 4a–4e 5 modules 14 × 14 × 832
Pool 3 × 3 / 2 7 × 7 × 832
Inception 5a–5b 2 modules 7 × 7 × 1024

The original paper counts GoogLeNet as 22 layers deep when counting layers with parameters along the longest path.

4. Train with auxiliary heads

Two auxiliary classifiers branch from intermediate features during training. Their weighted losses add gradient signals and regularization, but the heads are removed for ordinary inference. Their benefit is not simply that every gradient bypasses the full network; later implementations found the regularization effect especially relevant.

5. Replace a large dense classifier

Global average pooling turns the 7 × 7 × 1024 tensor into 1,024 values by averaging each channel. Dropout with probability 0.4 and one final linear layer produce 1,000 class scores. This avoids the parameter-heavy fully connected stacks used by AlexNet and VGG.

The reported training setup used SGD with momentum 0.9, weight decay 2×10−42 \times 10^{-4}, batch size 128, extensive crop and color augmentation, and a decaying learning rate.

Why It Works

  • Parallel receptive fields: Fine and broad patterns remain available without committing the whole stage to one kernel size.
  • Learned bottlenecks: 1 × 1 convolutions control channel count before costly operations.
  • Sparse-inspired structure: Dense branch operations approximate the paper's motivation of selecting useful local groupings while remaining efficient on available hardware.
  • Compact head: Global average pooling greatly reduces classifier parameters and overfitting pressure.
  • Balanced capacity: More depth and width are allocated inside a fixed computational budget. The 2014 result demonstrated that this design was effective; it does not prove parallel multi-scale extraction always outperforms every single-scale design.

Inception Family

Version Main change
Inception-v1 Original branches and auxiliary classifiers
Inception-v2/v3 Factorized convolutions, batch normalization, and later label smoothing
Inception-v4 More regular Inception-A/B/C modules
Inception-ResNet Inception branches combined with residual connections
Xception Depthwise separable convolutions framed as an extreme Inception design

Later variants factorized 5 × 5 filters into two 3 × 3 filters and n×nn \times n filters into 1×n1 \times n then n×1n \times 1, reducing computation while retaining an effective receptive field.

Limitations

  • Branch widths, reductions, and filter allocations create many architectural hyperparameters.
  • Heterogeneous parallel paths are harder to implement, inspect, and optimize than uniform sequential blocks.
  • Auxiliary classifiers complicate training even though they are omitted at inference.
  • Nominally low parameter count does not imply proportionally low compute; GoogLeNet still performs many convolutions.
  • Later Inception versions, residual networks, and mobile-focused designs improved particular accuracy, training, or deployment trade-offs.

Related

Sources