A classifier may produce a probability score for every observation. To turn those scores into a selected list, someone must choose a threshold. A score above the threshold is classified as positive; a score below it is not.
That threshold is not a universal property of the model. It is a decision rule whose consequences depend on the candidate population, class balance and cost of different errors.
What changes when the threshold moves?
Lowering the threshold usually selects more observations. This can increase recall because the model captures more positive cases. But it can also increase false positives and reduce precision.
Raising the threshold usually produces a smaller, more selective group. Precision may improve, but recall can fall sharply because more genuine positives are left unselected.
The exact relationship depends on the score distribution. It should be measured rather than assumed.
A historical BSC example
In one Stage 1 out-of-fold evaluation, the threshold table looked like this:
| Threshold | Candidates | GOOD | BAD | GOOD rate |
|---|---|---|---|---|
| 0.45 | 932 | 296 | 636 | 31.8% |
| 0.50 | 657 | 224 | 433 | 34.1% |
| 0.55 | 416 | 162 | 254 | 38.9% |
| 0.60 | 242 | 107 | 135 | 44.2% |
| 0.65 | 125 | 54 | 71 | 43.2% |
| 0.70 | 46 | 23 | 23 | 50.0% |
| 0.75 | 13 | 9 | 4 | 69.2% |
The general pattern is a smaller candidate set as the threshold rises. The GOOD rate generally improves, though it is not perfectly monotonic. At the highest threshold, only 13 cases remain. That 69.2% rate is based on a small sample and should be treated cautiously.
This table describes one historical evaluation. It does not prove that 0.75 is the best threshold in a future period or for a different use case.
Pick the threshold for the actual decision
Before selecting a threshold, define the objective. Is the goal to keep a candidate list small? To find as many positive labels as possible? To maintain a minimum level of precision? To reduce false positives under a fixed review capacity?
Then choose the primary metric and constraint using development data. A precision-recall curve can show the trade-off across thresholds. A confusion matrix at the selected threshold shows the actual counts. A cost-sensitive objective may be appropriate when the consequences of false positives and false negatives differ, but the costs need to be specified honestly.
Do not choose a threshold simply because it produces the most attractive percentage. A threshold with 100% precision on five cases may be less reliable than a threshold with 50% precision on hundreds, depending on the task and evidence.
Re-evaluate after the model changes
Thresholds are tied to score distributions and calibration. A new feature set, model version, candidate population or market period can change those distributions. Reassess threshold behaviour using later validation data. If scores are treated as probabilities, check calibration as well; a threshold of 0.50 is not automatically meaningful if the model is poorly calibrated.
For a two-stage system, there are two thresholds to consider. Stage 1 determines the candidates Stage 2 is allowed to see. Stage 2 determines which of those candidates are selected. Evaluate the whole cascade, because a strict first-stage threshold can cap the maximum recall available to the second stage.
Make threshold selection reproducible
A threshold-selection report should state the optimisation rule, candidate thresholds, minimum sample requirements and primary metric. For example, a research process might choose the highest-recall threshold subject to a pre-specified precision floor and minimum selected count. The exact rule depends on the use case, but writing it down before comparing results reduces the chance of choosing a threshold simply because it produced the most attractive historical percentage.
The practical lesson
A threshold is a policy choice built on model scores, not a magic number. Define the objective, inspect precision and recall across thresholds, include counts and uncertainty, tune on development data and confirm the choice on a later untouched period.