Posted by: bbannan | May 5, 2009

Testing and Design Thinking – Rachel

I just read an article in T&D’s magazine titled Leveling the Field – How to get the most success out of the Level 2 testing stage. Level 2 refers to the 2nd stage of Kirkpatrick’s evaluation testing whether “the participants learned what they set out to learn?” What I found interesting about the this article is that it provided the following checklist for evaluating learning:

* Know what you want people to learn and why it is important
* Measuring the effectiveness of your design and delivery is more important than measuring whether the learners are “good students.”
* Since you are measuring what matters, emphasize what matters in your design.
* Learn from your Level 2 evaluation by tweaking your design and delivery and tracking changes in results.
* Measure more than once. See what people retain over time, and track differences between learners or learning groups.

As I read the article I was struck by how relevant these tips are to design thinking. I believe the most important bullet is the first one which states “Know what you want people to learn and why it is important.” The instructional designer (or the design thinker) must be able to answer what is the goal for learning, how is it important and relevant to the person or organizational learning needs. The second bullet stresses the importance of “measuring the effectiveness of your design.” So instead of measuring how smart are your learners, the instructional designer should be testing out how effective the design allows for the transfer of knowledge. This idea is very much different than “testing” the ability of the learners. The final bullet suggests to “measure more than once [to] see what people retain over time, and track differences between learners or learning groups.” This concept reminds me of the iterative process for conducting usability testing for your design. Common themes will appear and it will be apparent what works and does work for the design. I think by following these tips, one can expect a positive learning experience for the learners and a better return in investment.



  1. Rachel,

    I agree with you that the bullets listed relates to design thinking and would also provide some useful feedback for the instructional designer to improve the training module.

    Just today I was asked to help with developing a training course. We have developed similar training courses before and I inquired about the effectiveness of the prior training. My coworkers responded with they liked the training and that it was the best training they ever had. My response to them was you cannot effectively evaluate training by merely asking a few multiple choice questions. I wanted to know if they had a reinforcer, someone who also works in the environment who can evaluate the learners based on what they learned in the training. I was a little frustrating for me to get them to understand the importance of measuring more than once. I am concerned about the effectiveness of our current training and I am brainstorming ways in which to educate my teammates on design thinking process.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: