Liftology 16: Knowledge and mini-scenarios

There’s a belief that we should see if people know the material first. However, this has been shown not to be useful. It’s easy, but it’s not helpful. Why? What can you do instead? Let’s look at knowledge and mini-scenarios.

Now, it’s received wisdom that you need to make sure folks know the material before you have them apply it. Until they have it, certainly, they can’t apply it. So it would seem to make sense. However, the received wisdom, it turns out, is wrong. 

What research discovered is that if you ask low-level knowledge questions,  you can find out if learners have the low-level knowledge. Of course, you won’t know if they can do the high-level things. If you ask both low-level and high-level, you can see if they have the knowledge, and whether they can apply it. What was insightful was that if you asked just high-level questions, they could answer high-level questions without having to ask the low-level questions. In short, the low-level ‘knowledge test’ questions didn’t add any value!

The reason why, as a proposal, is easy. To answer the high-level questions, they have to retrieve the low-level knowledge, and use it. If they don’t have the low-level knowledge, they won’t be able to solve the high-level problems. So they’re still having to retrieve the low-level knowledge, it’s just not asking that as a preliminary step. Now, you could suggest that you may be better off diagnosing a lack of specific low-level knowledge, but you can do that via well-designed high-level questions. And, really, you mostly care that they are ultimately able to answer the high-level questions, e.g. applying the knowledge to solve problems. 

Now, many times high-level questions are more costly to ask, in that you may be talking branching scenarios or even simulations. However, mini-scenarios: situation, choices of action, and consequences and feedback on the topic are just better written multiple choice questions! We can focus on writing better questions rather than asking knowledge test. (And, frankly, too often that’s not asking about critical knowledge and instead just seeing if you recall some arbitrary term from the content.)  We can do better, and should. LIFTs give us the ability to ask multiple choice questions, as well as generative activities like short answers or making a vid. We can and should use them to do a better job of preparing learners for learning.