Receptive Field

What a Receptive Field Describes

The receptive field of an activation is the region of the original input that can influence it. It describes how much spatial context is available at that point in a network.

A small receptive field supports precise local features such as edges. A large one can combine distant evidence into object-level or scene-level context. If the field is smaller than the pattern a task requires, the activation cannot represent that pattern in full.

Core Idea

One 3 × 3 convolution sees a 3 × 3 window. Stacking another 3 × 3 layer lets each new activation combine neighboring windows, expanding its receptive field to 5 × 5 even though neither kernel is larger than 3 × 3.

Depth, stride, pooling, and dilation determine how quickly this field grows. Padding changes which real or padded values are visible near a boundary, but it does not by itself add real image context.

How It Works

Track Receptive Field and Sampling Jump

For layer ll, let rlr_l be receptive-field size and jlj_l be the spacing, measured in input pixels, between adjacent activations. Starting with r0=1r_0=1 and j0=1j_0=1:

jl=jl−1slj_l = j_{l-1}s_l
rl=rl−1+(kl−1)dljl−1r_l = r_{l-1} + (k_l-1)d_lj_{l-1}

Here klk_l, sls_l, and dld_l are kernel size, stride, and dilation. For rectangular operations, apply the recurrence independently to height and width.

With stride-1, dilation-1 convolutions, each 3 × 3 layer adds two pixels per dimension. After LL such layers,

rL=1+2L.r_L = 1 + 2L.

Five layers therefore have an 11 × 11 theoretical receptive field.

Grow It with Downsampling

A stride-2 convolution increases the sampling jump for every later layer. A 2 × 2 pooling layer with stride 2 has a similar effect: it adds local coverage, then makes subsequent kernels span input positions farther apart.

Operation Immediate field growth Effect on later growth
3 × 3 convolution, stride 1 2j2j Leaves jump unchanged
3 × 3 convolution, stride 2 2j2j Doubles the jump
2 × 2 pooling, stride 2 jj Doubles the jump

Downsampling expands context efficiently and reduces computation, but it also removes spatial samples. This creates a trade-off between broad context and localization detail.

Grow It with Dilation

Dilation inserts spacing between kernel elements. A kernel of size kk and dilation dd has effective size

keff=k+(k−1)(d−1).k_{eff} = k + (k-1)(d-1).

A 3 × 3 kernel with dilation 2 therefore covers a 5 × 5 extent while still using nine spatial weights per channel pair. Rates such as 1, 2, 4, and 8 can expand context rapidly without reducing feature-map resolution.

Dilated convolution is useful in systems such as semantic segmentation and audio models. Poorly chosen dilation schedules can skip regular input positions, producing gridding artifacts.

Distinguish Theoretical and Effective Fields

The theoretical receptive field is every input position that could influence an activation according to the architecture. It can be calculated from kernel sizes, strides, and dilations.

The effective receptive field is the region that has substantial influence after accounting for learned weights and the many paths through the network. Influence is often concentrated near the center rather than spread uniformly across the theoretical field.

An architecture can therefore cover an object on paper while relying much more strongly on a smaller central region in practice.

Why It Matters

  • Architecture fit: The field should cover the largest context needed for the prediction.
  • Feature scale: Early local fields preserve fine detail; deeper fields support parts, objects, and scene context.
  • Localization: Detection and segmentation need broad context without losing precise spatial information.
  • Compute planning: Stride, pooling, and dilation offer different context–resolution–cost trade-offs.
  • Model diagnosis: A prediction may fail because the relevant evidence never reaches the activation making the decision.
Task Typical context need Common strategy
Edge detection Small and local Few layers without downsampling
Object classification Broad or global Deep stages with downsampling
Semantic segmentation Broad context plus fine detail Dilation or encoder–decoder features
Anomaly detection Multiple defect scales Multi-scale feature extraction

Feature pyramids and U-Nets combine deep, context-rich features with shallower high-resolution features. Self-attention can connect distant positions directly, although practical vision transformers may still restrict attention to windows or patches.

Design Trade-offs

  • A large theoretical field does not guarantee that distant pixels meaningfully affect the output.
  • Aggressive downsampling grows the field quickly but can erase small objects, boundaries, and texture anomalies.
  • Dilation preserves resolution but samples sparsely and may produce gridding artifacts.
  • Very large kernels increase direct context but can be expensive unless grouped or depthwise designs control their cost.
  • Multi-scale fusion preserves detail and context at the cost of additional memory, compute, and architectural complexity.
  • Border activations may depend partly on padding rather than real input, so their usable context differs from central activations.

Related