Your eyes already do this

Hold a finger close to your face and close one eye, then the other. The finger jumps a lot. Do it with something across the room and it barely moves. That jump is disparity, and your brain turns it into depth. A stereo camera does exactly the same with two cameras a fixed distance apart.

The formula

If the cameras have a focal length f in pixels and sit B millimetres apart, a point that shifts by d pixels between the two images is at depth Z = f x B / d. With f = 700 px and B = 120 mm, an object 600 mm away shifts 140 pixels. At 2.4 m it shifts only 35.

That's the whole trick. The hard part in practice is finding the same point in both images reliably, which is what stereo matching algorithms and depth cameras spend their effort on.

Why stereo gets worse with distance

Disparity is measured in whole or fractional pixels, and every matcher is off by some amount. Because depth divides by disparity, a one-pixel error matters much more when the disparity is small. In the figure, one pixel of error means about 4 mm at 0.6 m, 17 mm at 1.2 m, and 69 mm at 2.4 m. The error grows roughly with the square of the distance.

Halve the baseline to 60 mm and every disparity halves too, so the error doubles: 138 mm at 2.4 m. A wider baseline gives better depth further away, at the cost of a bulkier rig and more trouble matching close objects.

Choosing a setup

  • Start from the working distance and the depth accuracy you need, then work out the baseline and resolution.
  • Use a wider baseline or a longer lens for far objects, a narrow one for close work.
  • Add texture with a pattern projector for plain surfaces, which stereo can't match on its own.
  • Calibrate carefully and recheck it: a tiny shift between the cameras ruins depth.
  • Consider learned depth models for dense 3D, but measure them against a known reference first.