MCQ – going all in

I think it was at ResearchEd Durrington, that Peps McCrea talked about making bets. If I remember correctly he was suggesting that, having digested both research and experience, there was a point at which you needed to make a bet, and go all in. If you don’t do this then you end up trying to do a bit of everything, which isn’t as effective.

A recent thread on CogSciSci about the use of multiple-choice questions (MCQs) was asking for thoughts on MCQ design and particularly the problem of distractors – exposing children to wrong answers, and the danger that these get learned instead of (or as well as) the right answers.

I’m no expert here but I’ve read some of the literature. There is an extensive, recent review by Andrew Butler and some interesting work on learning from errors by Janet Metcalfe. Both those links come from Pooja Agarwal, and there is lots more useful and really accessible information on her Retrieval Practice website.

I’m also old enough to have both sat and taught national exams with lots of MC components, as well as being around for their more recent resurgence.

These are my bets.

If the purpose is mainly retrieval practice then MCQs should be kept fairly easy, with the number of alternative answers adjusted to try to make retrieval successful but effortful e.g.
The part of the electromagnetic spectrum that has the longest wavelength is:

  1. infra-red,
  2. radio waves.

The part of the electromagnetic spectrum that has the longest wavelength is:

  1. infra-red,
  2. radio waves,
  3. gamma radiation,
  4. visible light,
  5. microwaves.

BUT I think short response is better than MCQ if children can handle the demand. I’m aware that some literature suggests MCQs might be better but I think that might be because the success rate on the short response questions was low i.e. the students couldn’t handle the demand.

If the aim is to hammer misconceptions then MCQs might be best down at just two answers (unless there is more than one thing to get confused) e.g.

Weight is measured in:

  1. kg,
  2. N.

When two 10 ohm resistors are connected in parallel the total resistance is

  1. less than 10 ohms,
  2. more than 10 ohms.

I think feedback is needed without delay, and for misconceptions I think it should be immediately after a single question.

 

If you want something closer to the thinking required for short and extended responses, then MCQs might benefit from more complexity –

Q3– here you need to choose more than one answer – but I would see this sort of question as a starting point, leading to “why did you pick that?” oral questioning, and then maybe on to an extended response. Or you could do a cool poster to elaborate on this!!!

And if you are actually preparing for an MC exam – there are some fabulous questions out there from IB and A-Level Physics – then the final preparation should reflect the nature of the exam questions. But again I would see this sort of question mostly as a starting point. As long as you are successful in the endless campaign to show working, then popping up wrong answers to the MCQ below, on a visualiser, could be really valuable use of lesson time.

20181101_095107

I am pleased that my dislike of 30-odd years for complex answers seems to be vindicated. However, obviously if students have these in a national exam then they need to learn the exam technique. I shudder whenever I see roman numerals in a question…

20181101_100337

So, I think I’m coming to the conclusion that there are two purposes for MCQs.
One is to nail down declarative knowledge. Here the distractors are a potential problem so keeping the number of answers lowish, and aiming for a highish success rate, and providing swift feedback (and absolutely immediate for misconceptions) would be my bet. I think most of the research will have been on this first purpose so it’s about finding the best ‘fit’ with what that says.
The other purpose is to provide a scaffold for practising application (procedural knowledge and inferences from declarative knowledge) and here MCQs and ‘normal’ questions are similar in the way it’s best to use them in teaching. MCQs have the advantage of being easy to mark, though 🙂

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s