Convolutional Neural Networks (CNN)

What a CNN Does

A Convolutional Neural Network (CNN) turns spatial data into increasingly useful feature maps. In an image, early layers may respond to edges, deeper layers combine those responses into textures and parts, and a task head converts the final representation into a classification, detection, or segmentation result.

CNNs are designed around two properties of images: nearby pixels are related, and the same visual pattern can appear in different locations.

Core Idea

A small learned kernel slides across the input and applies the same weights at every location. Stacking these local operations grows the area each output can see while preserving spatial organization.

For an input XX and kernel KK, a two-dimensional cross-correlation—the operation commonly called convolution in deep-learning libraries—is

Yi,j=∑u,v,cKu,v,c Xi+u,j+v,c+bY_{i,j} = \sum_{u,v,c} K_{u,v,c}\,X_{i+u,j+v,c}+b

Each kernel produces one output channel, or feature map, whose activations indicate where that learned pattern appears.

How It Works

1. Detect Local Patterns

Convolutional layers apply multiple kernels to local windows. Kernel size sets the window, stride sets how far it moves, and padding determines how borders affect output size. Sharing each kernel across locations uses far fewer parameters than connecting every pixel to every output.

2. Add Nonlinearity

An activation such as ReLU or GELU follows convolution. Without nonlinear activations, stacked convolutions would remain a single linear operation and could not build complex decision boundaries.

3. Expand the Receptive Field

The receptive field is the input region that can influence one activation. It grows as layers are stacked. Pooling or strided convolution can reduce spatial resolution, lower computation, and make later units summarize a wider area.

Many small kernels often provide a useful balance: they grow the receptive field gradually, introduce several nonlinear transformations, and use fewer parameters than one very large dense filter.

4. Build a Feature Hierarchy

Depth Typical representation
Early Edges, color transitions, corners
Middle Textures, motifs, simple shapes
Deep Object parts and task-specific concepts

This progression is learned from the task rather than manually programmed; individual channels are not guaranteed to have a simple human-readable meaning.

5. Produce the Task Output

Classification networks often use global average pooling followed by a linear or MLP head. Detection and segmentation models preserve or reconstruct spatial maps so they can predict boxes, classes, or labels at particular locations.

Why It Works

  • Local connectivity: Nearby pixels provide a strong prior for visual patterns.
  • Weight sharing: A detector learned in one region can be reused elsewhere, reducing parameters and improving data efficiency.
  • Translation equivariance: Shifting an input tends to shift its feature map, which is useful for locating patterns. Pooling and task heads may add partial translation invariance.
  • Hierarchical composition: Deeper layers combine simple local evidence into broader semantic features.
  • Efficient implementations: Convolution maps well to optimized parallel hardware.

Limitations

  • Global relationships require enough depth, downsampling, dilation, or an additional global mechanism.
  • The locality and translation assumptions may be a poor fit when absolute position or nonlocal interactions dominate.
  • Downsampling improves efficiency but can discard fine spatial detail, which matters for small objects and precise boundaries.
  • Standard convolution is translation-equivariant, not inherently invariant to rotation, scale, or viewpoint changes.
  • Large high-resolution networks can still require substantial compute and memory.

Related