When the Top Answer Decides: Gender Bias in Decision Models

⚡ In one sentence AI-generated · check the original

Key findings

• Five of six models give women significantly more favourable answers than men (+0.05 to +0.10 in log-odds); a gender swap changes 1-3.5% of top answers.
• The recommended confidence gate removes biased commitments but raises the stereotyped share kept in ambiguous BBQ gender questions (granite-switch 0.76 to 0.85, pplx 0.81 to 1.00).
• All six models rate violence for a greater good as more acceptable against a man than a woman; two default to "girl" when guessing a child's gender without a cue.

📄 Abstract

Decision models are language models built to route, classify and triage: they return option probabilities from one forward pass, and an application acts on the top answer. They are offered off the shelf, trained for the decision format rather than for fairness or any one application, and we found no bias measurement in their cards. We test TypeSafe's closed Jev and five open decision models on gender-swapped yes/no decisions about people (Tamkin et al., 2023), BBQ, StereoSet and the gender-bias studies of Fulgu and Capraro (2024). Five of the six give women significantly more favourable answers than men (+0.05 to +0.10 in log-odds), and a gender swap changes 1-3.5% of top answers (Jev's partly through its call-to-call variation). The makers' recommended confidence gate removes many biased commitments but raises the stereotyped share of those it keeps in ambiguous BBQ gender questions (granite-switch 0.76 to 0.85, pplx 0.81 to 1.00). Probes show gender preferences in the same models: all six rate violence for a greater good as more acceptable against a man than against a woman, and two default to "girl" when forced to guess a child's gender without a cue. Decision-level and probability-level measures differ, so audits should read a model as its application does.