Liftology 20: The role of asking for impact reviews

When we create learning interventions, e.g. instruction, our ultimate goal is (or should be) to have an impact. It’s not just to meet someone’s request, but to actually move a needle. But how do we know if that’s happened? We can look at external indicators, which requires collaboration with the information holders. And that’s ideal. But we can also ask about what impacts are being seen. Yes, it’s less direct, but it’s also valuable. 

We should always be looking for confirmation. Just one data source can be misleading, and triangulating data has a long history as a more trustworthy approach. So, while empirical data is best, anecdotal data, particularly in aggregate, is another source. It is also useful to address the individual.

When we ask learners about the impact they’re seeing, we’re accomplishing two things. For one, we’re looking at what impact they’re seeing. That tells us at least subjectively what’s happening. It’s ideal if this data isn’t used as a basis for reward or punishment, but is only used to help learners improve. That helps us get the most accurate data, as they don’t have to make up results that are better than actual. Individually, it records the impact of the learning and their application thereof. Collectively, it tells a story about the value of the initiative.

The other effect of asking about impact  is their own recognition of change. When they see how they’re doing, it’s feedback that helps them recognize improvement, or a need for same. Learners see their progress, and what is keeping them from doing better. While individual feedback is ideal, a model response is still helpful for calibration.

We can also go further, if possible. We can and should ask those around the learner whether they’re seeing impact from the intervention. While this could be peers or even customers, the easy solution is their supervisor/manager. That person can be identified and arranged to be contacted. Then their assessment of impact provides a second source of information. 

While it’s still ideal to also get objective information of performance, that’s not always possible. While subjective, asking about impact still serves multiple purposes, and any data is better than no data. As you progress along the learning journey, as you near the end, ask learners what impact they’re having. If none, evaluate your intervention. If positive, celebrate your accomplishment.