Evaluation Enhances Blended and Online Learning
EnVision Performance Solutions and our elearning collaborator, Illumina Interactive, recently competed in the LINGOs (Learning in Non-Government Organizations) Global Giveback 2 Competition. Having developed dozens of online/elearning courses in the past few years, we jumped on this opportunity to “push the envelope” and integrate a variety of approaches.
For the competition we worked with Management Sciences for Health (MSH) in Cambridge, MA, to develop Coaching for Results, a highly interactive six-module, web-based course that introduces the value of coaching and provides learners with the ability to use MSH’s five-step coaching model. Learners have nine weeks to complete the course, which is moderated by an online instructor.
We integrated a number of evaluation approaches in the course designed to engage learners, help them learn, and provide feedback to both learners and the course instructor. As with instructional design for face-to-face instructor-led learning, we based evaluations on the learning objectives and spent some time in the planning phase to ensure evaluations were functional and relevant. Here are some of the approaches we used.
Quiz-type questions may be the most common type of evaluative components in online courses. We used them to reinforce learning as well as a level 2 evaluation of learners’ knowledge. For example, after being introduced to the differences between coaching and managing, learners are presented with a series of seven situations for which they must determine if the situation is an example of “coaching” or not. This series is not scored and feedback provides an explanation of the correct response.
Surveys can help engage learners and provide the course instructor with a before and after comparison of learners’ perceived skills and knowledge, a combination level 3 and 2 evaluation. At the beginning of the course learners respond to a series of statements that best describe how frequently they do, or are aware of, coaching-related actions and knowledge. The same statements appear at the end of the course. Statements include “I handle emotional responses and disagreements in a coaching conversation effectively” (level 3) and “I know my coaching strengths and weaknesses” (level 2).
Discussion forums can help learners process what they are learning and share their thoughts and ideas. Throughout the course, we leveraged Moodle’s discussion forum to ask a variety of level 2- and level 3-type questions.
For example, after viewing a “photo novella” vignette of a manager (who is also a doctor) coaching his employee, learners are asked, “What did the doctor do well?” In this level 2 activity, learners apply the basic knowledge they’ve learned in the course. The instructor may comment on discussions to reinforce correct responses and correct or redirect incorrect responses.
Another discussion asks, “What gets in the way? What are the challenges you face?” This provides the opportunity for learners to consider coaching in their real-life situations and begin to see how they can apply the skills they are learning to their own workplace, a level 3 evaluation activity.
Polls offer a fun and interactive way to engage learners while obtaining information about their level of skill or knowledge. Learners respond to the poll, and then see how other course participants responded.
We used a poll asking learners to identify their actual coaching opportunities by selecting from a list of “coaching cues.” This approach also helps set context for the learners, enabling them to identify ways they can readily apply what they learn throughout the course. The instructor can also use the poll information in a follow-up questionnaire after the course.
Learning partners, in which learners pair up to help each other learn, are a critical component of our course. Near the beginning of the course, we ask participants to complete a learning partner agreement and send it to the course instructor who follows up if the agreement is not sent in. Throughout the course, learners are instructed to complete a learning partner activity that includes:
- Discussing key points in the course
- Sharing ideas of how to apply what they learned to a specific work situation
- Practicing skills with each other before using them with employees
In follow-up discussion forum questions learners share key points from their learning partner experiences. The course instructor, following prompts in an instructor guide, monitors the discussions and offers guidance. Learners have the option of continuing their partnership after the course.
Those were some approaches we used, and we’d love to hear your thoughts and questions as well as your own tips for integrating evaluations into online learning. Oh, and the LINGOs competition I mentioned at the beginning? Yes, we did win! At this year’s eLearning Guild Learning Solutions Conference and Expo, EnVision Performance Solutions and Illumina Interactive were awarded first place in the corporate developer category for Coaching for Results. We continue to push the envelope to develop exciting, engaging, and effective courses.
