Literal Lock-In: How Test Language Shifts Literal Attraction in Typed Decision Models on Bengali Idioms

⚡ In one sentence AI-generated · check the original

Key findings

• Inserting the idiom's own literal gloss into the options removes 65% to 96% of each arm's remaining Bengali errors.
• Every arm is 1.9 to 6.4 times more captured in Bengali than in English on the same paired entries; capability does not order the failure.
• On a semantics-free option list with chance accuracy, the best-calibrated arm still reports mean confidence 0.82.

📄 Abstract

Decision models, non-generative scorers that report a probability over caller-supplied options and emit no text, are increasingly deployed as routers, classifiers and guardrails. Because they generate nothing, rationales and chain-of-thought do not apply. We audit three: an encoder scorer, a decoder scorer and a closed API, entirely through interventions on their input, using expert-annotated Bengali idioms carrying a gold figurative and a gold literal gloss. Dropping an idiom's own literal gloss into the option list draws every arm toward it, taking 65% to 96% of each arm's remaining Bengali errors. A model-by-language matrix over one shared item set shows how far this depends on the test language: every arm is 1.9 to 6.4 times more captured in Bengali than in English on the same paired entries, the commercial model's apparent resistance is a property of its test language, and capability does not order the failure. Calibration degrades in all three arms, and on the open arms confidence inverts, so gating on it is worse than gating at random. On a semantics-free option list, where accuracy is at chance, the best-calibrated arm still reports mean confidence 0.82. Code and data: https://github.com/sakhadib/literal-lock-in.