The smallest idea in computer vision

Every detection model, segmentation model, and most image classifiers are built from one operation repeated millions of times: the convolution. It sounds technical, but the idea fits on a sticky note. Take a small grid of numbers, the kernel, lay it over a patch of the image, multiply each pixel by the number above it, and add the results up. That sum is one pixel of the output.

Then slide the kernel over and do it again, until it has visited every position.

What the kernel is looking for

The numbers in the kernel decide what the convolution responds to. The press above uses a Sobel edge kernel: negative weights on the left column, positive on the right, zeros in the middle. Over a flat area the left and right cancel out and the sum is zero. Over an edge, where dark meets bright, they don't, and the sum is large. Positive means dark to bright, negative means bright to dark.

In a trained network nobody writes these numbers by hand. The model learns thousands of kernels during training. The early ones end up looking a lot like edge and texture detectors; later layers combine them into parts, shapes, and eventually objects.

Stride and padding decide the output size

Two settings control where the kernel stops. Stride is how far it moves each step: stride 1 visits every position, stride 2 every other one, which halves the output size and the compute. Padding adds a border of zeros around the image so the kernel can also centre on the edge pixels.

The output size follows one formula: floor((input + 2 x padding - kernel) / stride) + 1. For an 8x8 image and a 3x3 kernel, that's 6x6 with no padding, 8x8 with padding of 1, and 3x3 with stride 2. Press the stride knob and the padding lever in the figure and the readout shows the formula with the real numbers.

Why it matters when you build vision systems

Understanding this one operation explains a lot of practical decisions:

  • Input resolution: doubling the image width and height roughly quadruples the convolutions, which is why I tune input size to the smallest that still finds the objects.
  • Small objects: aggressive striding early in a network throws away detail, which is why small parts or distant people need models and input sizes chosen for them.
  • Speed: convolutions are many small independent multiply-adds, which is exactly the work GPUs are built for.
  • Edge devices: fewer and cheaper convolutions are what make a model run on a small fanless box next to the camera.