pr.probability - Estimate rate of real correct/wrong from 4 answers quiz.

I recently read that one in ten students think the first man on the moon was Buzz Lightyear, a Toy story cartoon. I'm not here to discuss the data in itself, rather, this reading got me into a problem I like just out of my interest for probability and statistics.

Suppose you have a quiz asking who was the first man on the moon, together with four answers:

1. Benjamin Franklin


2. Neil Armstrong


3. George Harrison


4. Buzz Lightyear

Now, if you give this test to a random number generator and you say they are students, the answers will be 25 % each. Clearly, you cannot declare 1 in 4 students believes Buzz Lightyear was the first man on the moon, and you also cannot declare 1 in 4 students knew the correct answer. It was just probability.

When you take such a test, you have these different cases:

1. people who know the correct answer and mark it


2. people who assume an incorrect answer as correct and mark it and


3. people who have no clue, and mark at random.

The results of the tests contain therefore a bias resulting from the "guessers", but hardly we can say that all people answering "Buzz Lightyear" really believed he was the first man on the moon. Most likely, some of them confused it with Buzz Aldrin and belong to case 2. Some others had no clue and threw a random choice. Same for the correct case: not all those who answered Armstrong really knew it. Some just guessed correctly.

Do you have any reference (or proposed solution) on this specific case to estimate the rate of really correct answers vs. random chance ?