How to write multiple choice exam questions: secret tips from an expert

Have you ever wondered who writes those pesky exam questions set to trip you up? And HOW they go about doing it? I asked multiple choice exam question expert Michael Lindsey to let us into some of his secrets…pay attention now: you may be asked questions later…here’s Michael.

Writing exam questions: it’s a question of getting it right

How to write multiple choice exam questions: secret tips from an expert

Over the years the humble multiple choice exam question has come in for a bit of criticism.

“And what do you do for a living?”

This is the sort of question people ask at social occasions. A perfectly legitimate ice breaker, but one that kind of fills me with dread.

The response I get to my reply is, at best, a very polite and unconvincing “Oh, that’s interesting.” Or at worst, you can see the eyes glaze over and swivel around the room desperately searching for someone more exciting to talk to.

So what bizarre job do I have that can kill conversation at ten paces? An occupation that can suspend general chit chat more effectively than an SAS officer’s stun grenade?

Well, I write examination questions for a living. Mostly in the field of financial services. OK, so it’s not that bad really, but it does tend to flummox people as it’s not exactly run of the mill.

Almost everyone in their lives will have taken an exam at some point, and someone will have sat down beforehand and wrote those questions. And on occasion that person is me.

It’s a job I got into by chance, but based on my 20 years of financial services experience working for an insurance company. For my sins I spent the next 20 years writing exam questions. Multiple choice questions to be precise. The sort of question where you are posed a problem and normally given four possible answers and have to deduce which of them is the correct one.

Over the years the humble multiple choice exam question has come in for a bit of criticism.

“Oh, they’re easy. It’s multiple guessing. You can usually discount two answers, so it’s 50:50.”

These are the sort of comments I’ve often heard. This type of testing people’s competence has long been savaged compared to more traditional “open answer” type questions, such as short answer or essay style.

—————————————————————————————————————————————————————-
Enjoying this article?
For more articles and tutorials about training and related topics, check out this category here on HTWB.
—————————————————————————————————————————————————————-

This comparison debate is probably best left for another time, and like most things in life, multiple choice has its pros and its cons and its suitability depends on circumstances.

Suffice it to say, any well prepared candidate who has come away from a multiple choice exam complaining about the questions is more likely to have suffered from some poorly constructed questions rather than an inherently flawed testing method.

So what are the main issues when writing multiple choice questions?

Well, firstly who do you get to write them? At first glance, you might think the answer is obvious. If you want questions on say physics, you get a physics expert.

But this is often the wrong approach. Being a subject expert doesn’t necessarily make you an expert on writing multiple choice exam questions on that subject.

Often a better approach is to get someone who is an expert (or has had training) on writing questions, but who also has a background knowledge on the subject being tested.

And then get the subject expert, or indeed a panel of them, to vet and edit the drafted questions. This bilateral and collaborative approach often works well.

Having sorted out who is going to write the questions, it might be helpful to look at some of the jargon that this process involves.

In the world of multiple choice exam questions, the question posed is called the stem, the correct answer is called the key and the (usually) three wrong answers are called the distractors.

One of the main skills in writing these questions is to make sure that, not only is the key definitively correct, but that the three distractors are definitively, but plausibly wrong.

Let’s look at a simple example.

What is the capital of Scotland?

  • A, Glasgow
  • B, Edinburgh
  • C, Birmingham
  • D, Perth

In this question, B is definitively right and the other three are definitively wrong, but C is not really plausible because it’s not even in Scotland. So it is a weak distractor as the candidate will only be choosing from three realistic options.

Another example, sillier and more exaggerated, makes the same point.

Which one of the following is a fruit?

  • A, Potato
  • B, Carrot
  • C, Apple
  • D, Shoe

Getting the wrong answers definitively wrong but with maximum plausibility is one of the main skills needed when writing multiple choice exam questions. The more plausible, the more robust the question. That’s why the exam writer needs at least background knowledge on the subject.

Consider this for a moment…using a relevant text book I could probably write a question on brain surgery with one perfectly correct answer and three wrong answers. But with no real knowledge on the subject, I suspect my three wrong answers would stick out like a sore brain!

Other grammar and reasoning skills are needed too.

The ability to avoid asking ambiguous questions is essential. You might have meant one thing, but if it’s written poorly, or you’ve overlooked a technical issue, it could be interpreted in a different way. Another common problem is accidentally giving two right answers. Remember, the three wrong answers have to be definitively wrong. No ifs, no buts.

—————————————————————————————————————————————————————-
Enjoying this article?
For more articles and tutorials about training and related topics, check out this category here on HTWB.
—————————————————————————————————————————————————————-

A further, less obvious issue is the subject of negative stems. Many exam boards like to limit or completely avoid using these. Questions like “Which one of the following is NOT an example of…” are often outlawed.

How to write multiple choice exam questions: secret tips from an expert

Michael Lindsey

I suspect that the main reason for this is that candidates’ brains tend to work by considering issues in a positive format not a negative format. The brain will instinctively assess what does apply rather than what doesn’t, and in a high pressure, high stakes, exam, candidates can easily get muddled.

Worse still is the cardinal sin of asking a ‘double negative’ question. A negative stem combined with a negative distractor. For example:

Which one of the following rules does NOT apply when discovering a fire?

  • A, Do not collect your valuables before evacuating the building.

The brain is expected to do backward somersaults and will probably hit cerebral meltdown!

So, remember, if you ever have to sit an exam, someone will have taken time (and hopefully great care) writing the questions.

Let’s hope that person did a good job so that the exam truly, fairly and efficiently tests your knowledge on the subject.

If you have any questions about multiple choice exam questions just jot them down in the comment section below or call Michael on +44 (0) 1525 381 921 or +44 (0) 7578 377 182.  

 

Comments

comments

Thoughts

*

css.php