Ever notice that multiple‑choice quizzes often feel strangely predictable?

A couple of weeks ago I had the BBC Crowd Science team over - producer Emily Knight & presenter Alex Lethbridge - to talk about probability in all its surprising forms.
We ended up diving deep into patterns, biases, and randomness itself.
Listen here: https://www.bbc.co.uk/sounds/play/w3ct6sv0
2/10

They wanted to answer a question from Griffith in Ghana about Unexpected Elements (https://www.bbc.co.uk/programmes/p016tmt2).
He’d noticed the quiz answers were often B or C, but rarely A—and wanted to know why.
3/10

While I can’t speak for the producers (they explain their reasoning on the programme), it seemed to me the issue was our species’ difficulty with randomness.
A major theme of my second book How to Expect the Unexpected:
https://uk.bookshop.org/p/books/how-to-expect-the-unexpected-the-science-of-making-predictions-and-the-art-of-knowing-when-not-to-kit-yates/7328813?ean=9781529408690
4/10

We’re bad at spotting randomness, but also bad at being random. Ideally answers A/B/C should be chosen with equal probability.
Any systematic bias lets sharp listeners like Griffith pick up patterns and beat the 1/3 odds.
5/10

One key bias is middle bias - favouring central options and avoiding extremes.
Behavioural scientists have shown that adding an 'ultra‑premium' option to an initial offering of 'premium' and 'basic' makes the premium option (now middle) more popular
https://doi.org/10.1086/208899
6/10

Similarly, educational psychologists find that students who don’t know the answer tend to pick the middle options:
https://doi.org/10.1111/j.1745-3984.2003.tb01099.x

The same happens in Battleship, on shelves, dropdown menus:
https://doi.org/10.1177/1745691615588092
7/10

And even in public toilets—middle cubicles can be up to 50% more likely to be chosen than outer ones: https://doi.org/10.1111/j.1467-9280.1995.tb00304.x

Was middle bias affecting the producers’ choices? Hard to say—but it's one of many ways randomness surprises us.
8/10

I also explored other counterintuitive probability puzzles—like the Birthday problem and Monty Hall—with Alex & Emily.
Hear the episode: https://www.bbc.co.uk/sounds/play/w3ct6sv0
Or pick up How to Expect the Unexpected: https://uk.bookshop.org/lists/kit-yates-books?&new-list-page=true
9/10

You can find a longer version of this thread in my latest substack article:
https://open.substack.com/pub/kityates/p/middle-bias-at-the-bbc?r=1cw8pl&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
10/10

@kityates

This is used in marketing and sales psychology. I believe it's known as 'anchoring'. The very high priced item/service makes the middle one seem more affordable.

@kityates
Fighting the biases is one of the reasons why setting multiple choice exam questions is really difficult. You think sitting an exam is bad - try writing one.

@kityates

One of my first mistakes as a new professor was not using a randomizer for my multiple-choice questions.

About halfway through the exam, one of the braver students came up to ask if I was trying to mess with their heads.

I had to make an announcement that they should just pick what they thought was the right answer, even if it seemed like they were getting an impossible pattern in their answers.

@kityates but they only had two b’s and one c to use, no a’s ; )

@kityates I recall a major one in some math program. In one algebra quiz, the wrong answers had exactly one deviation from the right answer, allowing a quick solution by skipping the calculation.

For the general case, I suspect this is based on making wrong answers only a variation of the right answer.

@kityates
Enjoyable thread.

Yes, middle option is a bias, in creating and using surveys. which is why we use randomized order where possible, but Likert scales can't be random, and forced choice is a poor solution.

There are myriad other issues with survey design as well - from answer length to the monstrous weirdness of impure continua. And then you haven't even touched on (mis)leading or ungrounded questions.

Survey design is complex and insanely difficult