If a recommender always picks the highest score, will its choices become less diverse?

Vowpal Wabbit's contextual bandit tutorial describes a key limitation: after the system selects an action, it can observe feedback only for that action, not the actual outcomes of every candidate. How to allocate opportunities between options already known to work well and exploration of new options is therefore central to recommendation decisions. Discussion angle If System1 scores products, content, or customer-service strategies, a high model score does not mean the long-term outcome has been validated. The editors suggest designing candidate scoring and the exploration policy separately, and retaining the candidate set, selection probabilities, and actual feedback for later analysis. This also does not make a decision API equivalent to an online learning algorithm. Beyond short-term click-through rate, possible objectives include satisfaction, repeated exposure, and unsubscribes. Discussion questions 1. How can a new product get a fair opportunity to be shown when it has no historical data? 2. Is a fixed exploration rate preferable, or should it change with feedback? 3. If a user did not click, did they dislike the item, or did they simply not notice it? Source: Vowpal Wabbit official Contextual Bandits tutorial https://vowpalwabbit.org/docs/vowpal_wabbit/python/latest/tutorials/python_Contextual_bandits_and_Vowpal_Wabbit.html Editorial note: This official S1 Hub editorial account used AI to help compile public materials. This is not a submission by the original authors, and S1 Hub has not independently tested the work described. Additional evidence and different perspectives are welcome.