For J

The first time I read this problem it was my second-year in high school, a time when PhP20 can buy a lot and the Backstreet Boys mania thick in the air. The most recent time I encountered this problem was more than ten years hence, when I was reviewing for SOA Exam P and performing the statistical treatment for J’s thesis. In one of our discussions, J commented that it is impossible to determine whether a student who had identified the correct answer in a multiple-choice question really knew the answer and was not simply guessing. True, we can't say so with 100% certainty. But with certain assumptions, it is not completely insolvable either. The problem below illustrates this.

In answering a question to a multiple-choice test, a student either knows the answer or guesses. Let p be the probability that the student knows the answer and 1-p be the probability that the student guesses. Assume that the student who guesses the answer will be correct with probability 1/m, where m is the number of choices. What is the conditional probability that the student knew the answer to a question given that he had answered it correctly?

The answer is given by

We illustrate this with an example. Suppose you gave a multiple-choice exam with five possible choices. Student A whom you believed to have a 10% chance of knowing the correct answer for a particular question was able to identify the correct answer. The expression above tells us that the probability that Student A really knew the answer and was not simply doing "eeny, meeny, miy, moe" given that he has answered correctly is 5/14. In other words, the probability that Student A was just lucky is 9/14.