The model gives a score, not an answer

A detection or classification model doesn't say "defect" or "fine". It gives a score between 0 and 1. Somebody has to decide where the line is: above it, flag the part; below it, let it through. That number, the confidence threshold, quietly decides how the whole system behaves on the floor.

Most demos leave it at the default of 0.5. In production it's one of the most important settings you choose.

Precision and recall in plain words

Recall is the share of real defects you catch. Precision is the share of flags that turn out to be real defects. In the figure, at a threshold of 0.50 the gate catches 9 of the 12 defects (recall 75%) and 9 of its 13 flags are real (precision 69%). Three defects slip through and four good parts get flagged.

Lower the threshold and the beam drops: more parts break it, so you catch more defects but flag more good parts. At 0.20 the gate catches all 12, but 11 good parts get flagged with them. Raise it to 0.90 and every flag is real, but only 2 of the 12 defects get caught.

Set it from what a mistake costs

There is no correct threshold in general. There's a correct threshold for your line, and it comes from comparing two costs: what a missed defect costs (a return, a recall, a safety incident) and what a false alarm costs (a person spends 20 seconds checking a good part).

For forklift safety, a miss can mean an injury, so you accept false alarms. For a cosmetic check where every flag stops the line, you accept some misses. Write those two costs down first and the threshold almost picks itself.

How I set it in practice

  • Collect a labelled test set from the real site, with real defects in it, not just the training data.
  • Run the model and sweep the threshold, recording misses and false alarms at each step.
  • Pick the point where the cost of misses plus the cost of false alarms is lowest.
  • Consider two thresholds: act automatically above a high one, send to a person between the two.
  • Re-check it after any change to cameras, lighting, products or the model. The right number moves.