Convolution
What Convolution Does
Convolution turns an image or feature tensor into maps of local pattern responses. A learned detector is applied at every spatial position, so one set of weights can find the same feature wherever it appears.
This preserves spatial organization while avoiding the parameter cost of connecting every input pixel to every output. In CNNs, convolution is the main operation used to build edges, textures, parts, and higher-level features.
Core Idea
A small tensor of weights, called a kernel, moves across the input. At each location, the kernel and the overlapping input patch are multiplied element by element and summed to produce one output value.
Deep-learning libraries usually implement discrete cross-correlation while calling it convolution. For input , kernel , and bias , one output channel is
Mathematical convolution flips the kernel before applying it. That distinction does not reduce a learned layer's capacity because training can learn the correspondingly flipped weights.
How It Works
Scan Local Regions
At each output location, the kernel covers a local window and computes a dot product. Moving the kernel across the input produces a feature map: high or low values indicate how strongly the local content aligns with the learned pattern.
The same weights are reused at every location. This weight sharing lets a layer detect a pattern independently of its position and keeps its parameter count independent of image width and height.
Combine Input and Output Channels
For an input with channels, each standard kernel spans all input channels. A bank of kernels produces feature maps, so the weight tensor has shape
An RGB layer therefore combines evidence across red, green, and blue. Later layers similarly combine all incoming feature channels unless grouped or depthwise convolution restricts those connections.
Control the Spatial Output
For one input dimension of size , kernel size , padding , stride , and dilation , the output size is
| Parameter | What it controls | Typical effect |
|---|---|---|
| Kernel size | Local pattern extent | Larger kernels use more context and parameters |
| Stride | Distance between kernel positions | A stride of 2 roughly halves spatial resolution |
| Padding | Treatment of image borders | “Same” padding can preserve resolution for stride 1 |
| Dilation | Spacing between kernel elements | Expands coverage without adding weights |
A dilation of 2 makes a 3 × 3 kernel sample across a 5 × 5 area. Padding prevents maps from shrinking too quickly, but padded border values are synthetic and can influence edge responses.
Stack Layers into a Hierarchy
One convolution sees only a local region. Stacking layers makes later activations depend on progressively larger input regions, while nonlinear activations between layers allow the network to represent more than one linear filter.
Early layers often respond to edges and color changes. Middle and deep layers combine those responses into textures, shapes, object parts, or other task-specific features.
Use Specialized Variants
- 1 × 1 convolution mixes channels independently at each position and can change channel count.
- Depthwise separable convolution performs one spatial filter per input channel, then uses a 1 × 1 convolution to mix channels. It can greatly reduce computation.
- Transposed convolution applies the transpose of the convolution operator to increase spatial resolution. It is not a mathematical inverse or deconvolution.
- Grouped convolution divides channels into groups, reducing connections and computation while preserving some channel mixing.
Why It Works for Vision
- Locality: Nearby pixels and features often form meaningful visual patterns.
- Weight sharing: Reusing a detector across locations drastically reduces the number of learned parameters.
- Translation equivariance: Shifting the input tends to shift the feature map by the same amount, before boundary and stride effects.
- Hierarchical composition: Repeated local operations build broad, complex features from simpler ones.
- Useful inductive bias: The architecture encodes assumptions about images that can improve data efficiency compared with unrestricted dense connections.
Design Trade-offs
- A local kernel cannot model distant interactions until depth, downsampling, or dilation gives it a sufficiently large receptive field.
- Stride reduces computation and expands context quickly, but discards spatial detail and can introduce aliasing.
- Dilation preserves resolution but may sample sparse grids and create gridding artifacts.
- Standard convolution mixes every input and output channel, which can dominate computation in wide layers.
- Translation equivariance does not imply invariance to rotation, scale, viewpoint, or deformation.
- Transposed convolution can create checkerboard artifacts when kernel size and stride interact poorly.