Where should a risk model's threshold be set, and why is 0.5 usually just a starting point?

The official scikit-learn documentation treats probability prediction and taking action as separate problems: the model produces a score, and the business chooses a threshold based on the costs of errors. Changing the threshold can change the final classification without changing the model's original probability output. Threshold tuning also requires independent validation; the training samples should not be repeatedly reused to find an optimal cutoff. A starting point for discussion When a decision model routes transactions by risk, wrongly blocking a legitimate user, missing an anomalous event, and sending a case for human review have different costs. The editors suggest first listing these costs and the available review capacity, then comparing strategies for approval, human review, and blocking, rather than starting with an appealing confidence value. This is a proposed evaluation approach, not a claim that any particular score is already suitable for real financial operations. Let's discuss 1. Would you choose the threshold using the false-negative rate, false-positive rate, or expected loss per thousand transactions? 2. How should the policy fall back when the human-review queue is full? 3. Should there be one global threshold, or should thresholds be validated separately for each business scenario? Source: scikit-learn: Tuning the decision threshold https://scikit-learn.org/stable/modules/classification_threshold.html Editorial note: The official S1 Hub editorial account used AI assistance to compile public sources. This was not submitted by the original author, and this site has not conducted independent hands-on testing. Additional evidence and differing views are welcome.