Polishing Your Gems – And Showing the Value of Your Training Efforts
Have you ever gone on a treasure hunt, or maybe seen one in the movies? The seeker searches for a treasure chest—usually attending to challenges along the way. Imagine large rolling rocks and a few poison darts, all difficult to control! Eventually the seeker locates the elusive treasure chest, but not all its contents are gleaming. There is some culling to do, and some polishing, before being ready to present the gems to the funders of the treasure hunt.
Back in the real world, learning & development professionals are given a mission to add value to the business, often through a request for “training.” On the L&D professionals’ “treasure search” they often attend to challenges along the way. Imagine the multiple competing priorities, regulatory controls, and insufficient resources that can impact successful learning.
Once the treasure is “found” (the learners have learned something!), what culling and polishing needs to be done so you can clearly see your treasure, before you present your findings to the funders and other stakeholders in your organization? Sometimes it can be challenging to find those precious gems and polish them so we can see how they glitter!
Earlier this month, I presented a workshop at the regional ATD conference. The topic, Polishing Your Gems, focused on creating an evaluation plan for a learning solution. Joined by client Matt Matosic from the Boston Public Health Commission (BPHC), we worked through a case study summarizing BPHC’s course on Hospital-Based Patient Decontamination and discussed options for evaluating its success. Participants then had an opportunity to begin developing their own evaluation plans, using an Evaluation and Measurement Planning Checklist we provided.
But where to start? Here are the core questions we suggested in our workshop to help folks determine which treasures to seek, as they begin their own evaluation planning.
Business issues questions
- Why is this training (or other intervention) being requested?
- What is driving the request or identification of this issue?
- How will solving this problem support the organization’s “business” goals?
- What do you need to be able to show as an outcome of your efforts?
- Who will see the results? (Consider: L&D management, business line management, senior management, learners themselves, instructional designer, instructor, funders, and other stakeholders)
- What will each stakeholder do with the information?
You might recognize that many of these questions get at Kirkpatrick’s level 4 (business impact).
Job performance questions
- What changes in job performance are needed to support meeting the business goals?
- What changes in job performance can reasonably be addressed via a learning solution?
- What workplace supports (such as managers, mentors, or peers; job aids or other performance support tools; periodic updates or web meetings) are realistically available to support the change in job performance?
These questions relate to Kirkpatrick’s level 3 (behavior change on the job).
Polishing away
Answers to these questions will help you define the learning objectives, set up workplace support systems to ensure learning continues and is reinforced on the job, and develop evaluation tools at all the Kirkpatrick levels.
For more information on our Evaluation and Measurement Planning Checklist, contact solutions@envision-performance.com.
Heating Things Up: Pilots Provide Useful Feedback
Practicing a classroom course polishes and improves it. Typically, practice comes with a pilot, during which instructional designers and trainers iron out classroom kinks.
Two pilots (an abbreviated pre-pilot and a full pilot) were employed in a class entitled Operationalizing Emergency Plans: Incident Command in Action, on which EnVision consulted for a public agency. The example shows how a pilot helps the design process when it works well.
This four-hour, immersive simulation was designed for nurses, administrators, and emergency medical technicians in healthcare environments such as long-term care facilities, hospitals, community health centers, and public health commissions. During the simulation, the learners make decisions to help their organization transition from day-to-day operations to a true-to-life emergency situation; a fictitious, extreme heat wave hits the area with no relief at night, producing health and medical complications for the learners to handle.
The learners are divided into teams of various medical and healthcare agencies seated at separate tables, with each learner playing a specific Incident Command System role. The scenario begins three days before the heat wave hits and progresses from the early stages to mid- and late-stages when serious repercussions occur, including power loss, a medical surge, and mass fatalities. The simulation ends with action reporting and improvement planning.
In the class, each team determines which actions to take. EnVision’s client utilized facilitators from different community agencies to help shape discussion at each table. In addition, each table had “injects” sporadically arrive with information that could impact each table’s decisions.
“During the class, the learners explore what it takes for organizations to be ‘operationally ready’ for emergencies, and to experience the Incident Command System in action,” explained Marilyn Kobus, an EnVision team member who worked on the project. Kobus supported the client subject matter expert/course designer-developer by offering design consultation, providing project management services, and contributing to the course instructor guide.
Following the pilot, the learners shared positive feedback. First and foremost, they felt more prepared for a large-scale emergency. The immersive nature of the class enabled the learners to envision what professionals in the different job roles might actually be thinking during an emergency. One learner called the class “a complete revelation.”
In addition, the learners liked the choice of a heat wave as a learning scenario versus a more typical New England event, like a blizzard. Their main challenge was playing “catch-up” following the fast-paced first module.
The instructional designers and trainers learned also, especially in the pre-pilot. Delivered to students in the Public Health Program at Tufts University School of Medicine, the pre-pilot uncovered necessary changes in the course timing, content, and organization of materials. The team also decided to add pre-work for the pilot session.
From the design team’s perspective, the pilot largely hit the bull’s eye. The team planned for minor changes for the actual course, such as having a participant assume the role of team scribe and modifying the pre-work.
Kobus was pleased with the pilot’s success, and believed the design team’s focus on gathering input from community stakeholders to create the scenario played a big role in it. “I have not seen a pilot with so many moving parts run as smoothly as this one did. Despite the fact that there were multiple players, detailed scenarios for the simulation, and unique “injects”…despite that complexity, I thought it was an outstanding pilot. Learners were fully engaged from the start with high energy, and said they gained great skills and information to take back to the job.”
Team Learning Opens Minds — and Office Doors
Too often, we go through our workdays head down, nose to the grindstone, and office door shut (or cubicle door metaphorically so). A coworker may need help with something, but we get lost in a deadline, meeting prep, client appointment and are oblivious. Perhaps once in a while, we stop to touch base with our colleagues. Yet in this electronic age, so much of our networking gets done online, rather than face-to-face. And our coworker still has her question that never gets answered.
At EnVision, we value teamwork, communication, and the figurative open office doors very highly. The “EnVisioning our Future” (EOF) sessions we offer are natural extensions of this philosophy.
Twice each year, we hold a day-long EOF for the EnVision team. The subject matter runs the gamut from a new business book’s theory to examples of elearning . Our most recent EOF focused on software application tips and shortcuts. The day always provides a great opportunity to learn from and help each other.
At our “Sharing Application Tips” EOF, several EnVision members presented all kinds of software tips — from embedding objects in a PowerPoint slide to assessing our document’s reading level — all designed to help each of us be more productive and efficient. As each team member presented her software tips, those tips were enthusiastically appreciated by the team.
”Sometimes, the simplest things make a huge difference in the way we work,” said Ginny Maglio, an EnVision team member. “The EOF professional development day validates that we never stop learning and can always learn from each other.”
At this EOF, as with past sessions, our team members realized three benefits beyond the subject matter knowledge they acquired. Team sessions, like our EOF days, can empower your team members to:
1. Help each other while also gaining knowledge in another area. In our recent EOF, one team member demonstrated how to create an info graphic using PowerPoint, while another showed us how to create a table of contents in Word.
2. Learn about—and from—other team members’ projects. Because we each work independently or in small teams, we may not know about other team members’ projects and competencies. The EOF sessions introduce team members to each other’s skills and capabilities, and also show participants how different parts of the organization work to achieve clients’ objectives.
3. Grow relationships with each other. Networking, sharing ideas, and just plain “kicking back” with your coworkers facilitates bonding, making you more comfortable in your role and more likely to approach a coworker with a question. And who knows? You may make a new friend out of it, too.
So, in the rush to meet deadlines, don’t forget to hold regular learning sessions. This results in not only a better-informed team, but also a collaborative, harmonious one.
Ride the Wave of Change – and Navigate Successfully
Change is hard and causes unease, anxiety, and at times, frustration. Every parent who has dropped a child off at kindergarten (or college) knows this!
While change can be overwhelming or even scary, it is often necessary. Professional change brings its own set of challenges. How can you best navigate change at work?
EnVision needed to adapt when we were required to move our SharePoint site over to a new hosting company. We were notified by the “old” hosting company via e-mail and given less than a month to move over our files, as our original service would then be discontinued.
I was a bit taken aback after learning of the tight timetable. After taking a few deep breaths, I notified Tom, our IT guy, and Danielle, our technical/administrative whiz. I couldn’t do this without them.
Tom managed the functionality of the server – back-up and hosting. Danielle’s responsibilities, which required about 8-10 hours of work, included revising the new site’s look to mimic the old SharePoint; setting up appropriate user access for each team member; creating usernames and passwords; and providing crackerjack technical support.
Danielle met some challenges initially. “I went in thinking it’s going to be exactly like the old SharePoint….It is a little bit different,” explained Danielle. “It wasn’t as intuitive.”
The new service provider didn’t offer a support document or user guide. Plus, Danielle discovered that when she added folders, there was no way to return her to the main SharePoint page, so she got creative and built a workaround by adding a “home” link.
The end users (EnVision team members) encountered obstacles, too. Some of the functionality worked best with a particular browser; one team member had difficulty logging in; another had trouble opening and saving documents. Thanks to Danielle and her support, these issues resolved pretty quickly.
Lessons learned from this SharePoint experience can be applied to any change. While initially I felt a bit overwhelmed, I dug into the new software and eventually figured it out.
Three things that helped me were:
1. Planning extra time into my schedule to adjust to the new software and minimize impact to our own clients.
2. Asking for help when needed. I was fortunate to have Danielle and Tom to turn to, and relied on them. Having Danielle as my capable “point person” to train my team was a huge benefit. I will remember to choose my “point person” carefully for all future changes; it can make the difference between implementation success and failure.
3. Keeping a positive attitude. Knowing that my team and I would eventually figure out the new SharePoint helped me through the change.
I’m happy to say we successfully moved our files and team over to the new SharePoint. Weeks later, it seems like we’ve always used this service provider…and we like the new site even better than the old. Time to embrace the next change!
Helping Learners to Digest – even Savor – Your eLearning
When you eat at a restaurant, the food arrives in a specific order. First, there’s the bread (for those who still eat carbs!). Next, you may get a first course, a salad or soup.
Then the waiter brings the main course, the heart of the meal. This takes longer to eat than the bread or first course, but is usually the most satisfying part of the meal. It’s also the most vital part – if you don’t eat the main course, you could be hungry when you leave the restaurant.
Finally, you can order dessert and coffee. Of course, dessert isn’t necessary – it’s “nice to have.”
But what if all of your courses were served at the same time? You wouldn’t know which food to start eating, or where to focus your attention. You’d have dessert at the start of the meal, whether or not you wanted it. And, you’d probably feel overwhelmed and irritated, and might lose interest in the main course.
The same principle holds true for elearning. If you are given too much material simultaneously, your mind will be overloaded, just like your stomach would be with too much food at once.
Here are some pointers to reducing cognitive overload in your elearning:
- Create a course that reads logically. People read top to bottom, left to right. So avoid having information begin on the right-hand side. This prevents your learners from having to work too hard and get frustrated.
- Use white space liberally. Placing paragraphs and graphics too close together creates a disorganized appearance and will only confuse the learner. Instead, break up one screen into two or more screens.
- Add color wisely. In an elearningindustry.com post, Christopher Pappas advises using just two or three colors per course or module; otherwise, the course may appear disorganized. When adding color, don’t forget that some learners may be color blind, and find another way to provide visual contrast. You can even find online applications that analyze color contrast.
- Be smart about animation. There are a lot of great tech tricks out there, but just because you can do something doesn’t mean that you should. Any formatting tool should support the learning objectives and/or learning styles; when in doubt, take it out!
- Focus on the need-to-know; trim the nice-to-know material. Research backs this up: Studies have shown that extra details can actually detract from learning. If you must incorporate the optional information, put it in a clickable “tips” box on the applicable screen or in a course resource list.
- Use audio rather than text to explain graphics. This allows the learner to focus visually on the graphic while taking in the description aurally, which maximizes learning. Of course, text should always be an option for sight impaired learners.
Implement these tips, and they will help your learners to be satisfied by – and not overwhelmed by – your elearning solution. And, they might even look forward to returning for another “meal” sometime!
Re-envisioning the Meeting, Part 2 – Meet, Close, and Follow-up
In part 1 of Re-envisioning the Meeting, we looked at how to prepare for and open a meeting. Now let’s look at facilitation, meeting conclusion, and post-meeting activities.
Facilitate Actively
The facilitator must consciously strive to keep participants on task to achieve the meeting’s goals. If you’re talking about an upcoming training launch, for example, don’t spend time generating ideas for your next team development day. Participants may have tangents that they would like to explore, due to their job role or specific interest. It is the facilitator’s job to keep them focused. Using a “parking lot” can capture important topics for discussion at a later time.
While attendees may take their own notes, the meeting facilitator should as well (or assign someone to take notes), so everyone can refer back to them later. As action items are assigned during the meeting, write down each assignment, responsible party, and due date. Similarly, for key decisions that are made, document the decision and date. For recurring team meetings, you can build on these lists and check off assignments as they are completed.
Close Graciously
Summarize key points and assignments at the end of the meeting. For recurring meetings, remind attendees when the next meeting will be. And of course, thank everyone before you adjourn. Showing your appreciation for everyone’s time and participation helps motivate them to continue contributing in future meetings.
Follow Up Promptly
After the meeting, be sure to e-mail the attendees and other interested parties the meeting notes, or at least any major decisions that came from the meeting. Include the action items that need to be completed and their due dates.
Although not done often enough, it is definitely wise to ask for participant feedback to help the next meeting run even smoother. One idea: Pose a statement or two, such as “This meeting met my needs” or “I know what is expected of me prior to our next team meeting.” Ask attendees to write their level of agreement with the statement, on a scale of 1-5 where 1 is strongly disagree and 5 is highly agree, and hand in their responses. If you see a trend, you can follow up with the group to learn how to improve the meeting next time. Another idea: Ask an open-ended question, such as: “What worked really well in today’s meeting you’d like to continue?” or “What is one thing we should do differently in future meetings?”
Follow these tips, and your meeting will not only be more meaningful, but can result in greater productivity, too!
Re-envisioning the Meeting, Part 1
“Wow…what a GREAT, productive meeting!”
When was the last time you heard someone say that? Hopefully the other day …. or has it been a while?
Meetings have, sometimes fairly, earned the reputation of being timewasters at best, scheduled
snooze-fests at worst. Too often, employees suffer through meetings until they are released from captivity, free to resume “real work” again.
Yet, meetings are actually “the laboratories of real, measureable teamwork,” according to an article by Kristine Kern of The Table Group (Inc.com). They can in fact be very productive, if planned and run well.
At EnVision, we offer a proprietary and customizable course called “Making Meetings Meaningful.” The cornerstone of the course is that a meeting’s success is not only determined by what occurs during the meeting, but also what happens both before and after. To conduct an effective meeting that really furthers participants’ work, you should:
Prepare Thoughtfully
Before you gather people together, know the objective for your meeting. What do you want to achieve? Be sure to only invite those people who will be interested in and/or impacted by the meeting’s content. If certain people don’t need to actually participate in the meeting, feel free to give them a pass and instead update them afterward.
Plan an agenda to keep the meeting focused and avoid tangents. And, estimate the time it will take to discuss each topic and share your timetable, so attendees know that the meeting, no matter how beneficial, won’t go on forever.
Start Strongly
From the beginning, set the tone that the meeting will be efficiently run. First, begin on time. If someone arrives late, acknowledge him and continue with the planned agenda. Backtracking will only frustrate those who came on time.
Share the meeting’s objective and agenda with the group so they know what to focus on. And, if needed, set ground rules for the meeting. If your company is one that claims meetings as “smartphone free” zones, remind everyone to turn the sound off their phones and put them away.
Designate a “time keeper” as a back-up for ensuring that the meeting ends on time.
An organized start will set your meeting off on the right foot.
Next time: We’ll cover meeting facilitation, meeting close, and follow-up.
Short Can be More Than Just Sweet!
What do stained glass windows, intricate marble carvings, and richly symbolic murals have in common?
Give up?
They are all easily seen from just one spot in the Library of Congress.
Why do I mention this?
I recently visited Washington, DC and toured the Library of Congress. If you haven’t been to the Library, I highly recommend it. It is jam-packed with, well, books. But also with all manner of media. And the art and architecture! The building was built to safely house books, but it also shows off art and architecture created by over 40 American artists.
Our tour guide spent much of our hour together describing just a portion of the art and architecture we were seeing in a relatively small part of this edifice. There were masterpieces filling every square inch of wall, ceiling, floor, pillar, and even staircase. Each item’s artist, creative origin, and beauty could have merited an hour on its own. After five minutes I was on mental overload!
I can tell you the feelings I am left with after visiting the Library: awe, pride, and patriotism. But I don’t think I can tell you who the artists were or the significance of each piece of art. Maybe that was the idea — that the most important take-away was my overall impression, not the myriad of artistic details.
Well, I do remember the Library is supposedly going to acquire all Tweets. And that Jefferson’s library burnt down and they are trying to re-acquire all missing volumes so they can recreate it. But the experience reminded me that less, so much of the time, is more. So, I’m making this post really short. Just trying to make one point.
Here it is: Keep your training focused. Lots of content doesn’t necessarily translate to effective learning.
I might need a reminder myself the time next I write a post, but here’s to giving it a try!
How Many “Two by Fours” Does Your Curriculum Use?
By Kathy Harvey-Ellis
For my day job, I work in marketing, for EnVision and another company. After this winter, however, I also feel like I work in home improvement – of our own home. I am my own general contractor. From the repair of a supporting beam in our garage, to minor water damage in our house, to our nascent attempts to renovate our long-suffering kitchen, the projects have taken center stage in my life.
I call contractors. I talk with them, and I make notes. I hire people. They bring materials, and begin to work. While I know generally what they will do, I don’t always know their project plan or understand specifically how it will address what I need done. After they leave, I carefully examine the finished work, trying to evaluate its success without knowing exactly what I’m looking for. Have they consulted the most knowledgeable people on their staff? Have they used quality materials? Really, I’m just hoping for the best.
If only I had a construction plan, a blueprint. Then, I would know what they are preparing to do, and if their plans match what I need done.
For a successful “construction job” or learning solution, instructional designers need a curriculum blueprint, too. Their blueprint, a design document, guides the team to a solution leading to improved performance.
The curriculum blueprint will tell the reader the Who, What, and When of the learning plan. Let’s start with Who.
Who
My garage repair project involves a project manager, structural engineer, construction foreman, and construction workers, each with specific and different responsibilities. Similarly, the curriculum plan describes who has responsibilities in creating and executing the learning design, and what exactly their tasks are. These people include the instructional designer, subject matter expert (SME), project manager, reviewer, and more.
Early on, the document also describes the “homeowner” or target audience– the learners. What are their roles within the company, and what is their experience level? Which factors motivate them to perform well, and which approaches to learning do they prefer? This information helps lead the design team to choose certain modalities and activities to “construct” the curriculum.
What
Like a construction project’s scope of work, the “what” portion of the curriculum blueprint tells the team all of the work that needs to be done to ensure the learners are taught effectively.
The blueprint document addresses the blend of learning modalities that the curriculum includes, whether those be elearning or classroom training (in-person or virtual), on-the-job training or hands-on lab time. The evaluation strategy may be described here as well.
Once the learning blend is established, the blueprint will incorporate the curriculum development, including a list of the physical deliverables generated. Slides, instructor notes, and learning manuals all fall under this category.
While these components are critical pieces of information in the blueprint, the bulk of the document is an outline of the curriculum’s modules, comprising the topics, delivery modalities, and learning activities that support the learning objectives.
When
The outline includes estimated learner time for each topic, allowing the team to plan for delivery of the training. The blueprint also spells out the order of the modules and any prerequisites.
The curriculum blueprint provides an organization with a logical framework to construct a learning solution. Similar to how a construction company supplies a scope of work and builds a structure, such as a garage, the curriculum blueprint designs and frames a training program. “A curriculum blueprint provides a clear overview of what will be included in the training, how the training will flow, and in what manner the training will be delivered,” explained Irene Stern Frielich, EnVision’s President. “Having a roadmap helps guide the developer, though she may decide to make a different design decision as new options arise during the development phase.”
The Pick of the Crop
Have you ever gone berry picking? Usually, you fill a container and pay by its size. Those juicy berries look so appealing you want to fit as many as possible into the container, right? Would you squish them in to get more? Or would you select the ripest berries to ensure excellent quality and highest satisfaction?
So, what happens if you try to pack too much content into your learning event? While perhaps not as messy as overstuffing berries, it’s not a pretty sight either. As training professionals, we are often caught between (a) advocating for the appropriate amount of time needed to result in effective learning and (b) the time that management is willing to make available for training. This often happens because organizations are running pretty lean, and missing employees for a few hours or days can impact production or the customer experience. Sometimes there is the added opportunity cost of billable time or direct cost for overnights that result in hotel and meal expenses. If effective learning requires six hours but the training “container” only holds two hours, how can we reconcile this? Do we stuff more in or pick the best, given the size of the container?
Here are three ways that instructional designers can address time constraints while helping to ensure the learners are prepared to do their jobs effectively.
#1. Have an honest conversation with stakeholders and share with them how people learn best.
Reinforce that learners need to be exposed to various methods for taking in information and they consolidate their knowledge by practicing skills in activities such as role plays, re-enactments, diagram creation, or troubleshooting. Part of the process is to receive feedback along the way and debrief at the end. This process cannot be rushed.
#2. If it is absolutely necessary to shorten the training prioritize the learning objectives and design training that focuses on supporting the most important behavior changes at work. We recommend the resulting lessons be ones that will help the learners perform the most frequent tasks that are most critical and could result in greatest risk if done incorrectly.
If, let’s say, only two hours are allocated for what would normally be a six-hour course, the instructional designer should work with the stakeholders to determine the highest-priority skills for the learners to perform effectively. This applies for any training, including performance management, compliance, and product sales and service.
#3. Keep in mind that learning can be maximized, and time away from work minimized, with the right design and blend of learning modalities.
Creating a blended learning approach, rather than relying solely on traditional classroom training, for example, can help streamline the learning process. It also lowers time away from work since travel time may be reduced and, often, the overall course “seat time” can be trimmed down.
We recommend that training include ways for the learners to immediately apply skills learned in class on the job. One such method is a practice known as “action learning.” This provides a structured way for the learner to solve a real-work problem on the job, then reflect on his or her learnings.
For a course on process management, EnVision recently incorporated an action learning approach, which proved helpful to the learners. “Action learning was very valuable, because we went from theory to practice,” said one learner. “The time for reflection and processing between sessions was valuable,” said another.
Increasingly, companies rely on training tools that can be accessed “on demand” to incorporate training into the workday. “On-demand training, such as virtual reference guides and ‘bite-sized’ training modules, can give you the ability to offer immediate and effective training to employees when they need it the most.” (elearningindustry.com).
Try using the above approaches to align with your stakeholders and focus your training, given the resources or “berry containers” you have. Throughout the process, keep in mind the learning objectives vital to improving employee performance. And help your learners enjoy fewer, but more satisfying, berries!
The Course Pilot and a Wedding? More Alike than You Think.
The bridesmaids donned light green dresses with intricate lace overlays. Before the ceremony, the ketubah (marriage contract) was signed by the groom. The groomsmen and bridal party enjoyed sushi and potato pancakes with smoked salmon in separate rooms before the wedding. At the appointed time, both groups entered the ballroom, awaiting the arrival of the bride on the arms of her parents. Each part of the beautiful wedding is executed perfectly, but how? Is it practiced for weeks on end?
No, of course not. Rather, it is choreographed according to a plan, by the couple and aided by a wedding planner. It is designed according to the couple’s traditions, which many of the attendees may know by heart from attending other nuptials.
After celebrating at this very wedding, it struck me that a well-coordinated wedding resembles the successful pilot of a course. Just like a wedding, the pilot needs to have each facet planned and each element timed carefully.
While the wedding plan focuses on where the bride, groom, officiant, and wedding party stand and what they say and do, the pilot plan describes what the instructor, debriefer, business stakeholder, and learners will say and do during the pilot. Prior to the course pilot, the pilot choreographer discusses the responsibilities with each of those groups just as, before the nuptial ceremony, the wedding participants learn what their roles will be. As soon as guests begin arriving at the wedding, the wedding planner begins to quietly guide the festivities from behind the scenes. Similarly, once the pilot begins, the pilot choreographer works in the background to help ensure the pilot runs smoothly.
A course pilot serves several objectives, one of which is to get a sense of the length of each portion of the training. If a facilitator goes off on a tangent, there may not be enough time for the skill-based portion of the class. Should the instructor take too many questions from learners, there may be no remaining minutes to summarize takeaways for the learners.
Similarly, wedding planners must have every element of the couple’s special day planned down to the minute. The bridal party must start their walk down the aisle at the appointed time. Each dinner course must be served on schedule, so the dancing can take place as planned. The band must plan their break carefully. Every element of the event is carefully orchestrated behind the scenes so that the celebrants don’t notice it.
In addition to the timing of events, the order of modules and learning topics can be finalized with the knowledge gained during the course pilot. Sometimes, the original plan may not correlate to the logical order of learning. Likewise, the bride and groom may choose to change the order of events if there is a timing issue during the wedding rehearsal. Perhaps they decide a particular reading or song would lengthen the ceremony and cut into the reception time, so they elect to take it out.
Of course, there are some factors for course pilots that don’t arise in wedding planning. Periodic check-ins with learners, with or without feedback forms, can be very effective for a pilot. Is the class going too fast or too slow for the learners? Are they engaged in the material? Are they bored? All of this information helps the instructional designer.
The course pilot offers many tools to plan the learning solution. It helps create what the instructional designer hopes is a successful course – just like a well-planned wedding rehearsal leads to a well-choreographed wedding, enjoyed by all.
When Learning Becomes Too Much: Seven Ways to Reduce Cognitive Overload
Have you ever been in a class where everyone’s eyes are glazed over? By the end of the first half-day, participants stop, well, participating? Perhaps the course just isn’t engaging them. Or, perhaps (cue music: dut, dut, dut) they are experiencing cognitive overload.
So, what is cognitive overload?
Cognitive overload is an inundation of short-term memory or working memory. This type of memory is easily overwhelmed because it can only hold between two and four “chunks” of information. (Halls, Jonathan. “Memory and Cognition in Learning.” InfoLine: Tips, Tools, & Intelligence for Training, May 2014). Sometimes cognitive overload cannot be avoided; however, it is considered “extraneous” when the learning design or method causes it.
And, why should I care?
Learning can be adversely impacted by stress, including stress caused by cognitive overload. The amygdala is the section of the brain involved with emotions, and it is affected by negative experience or emotions. “When learners experience stress they go into a hyper-stimulated state known as the affective filter, which inhibits information passing through the amygdala to the information processing parts of the brain.” (InfoLine).
An overzealous subject matter expert (SME) could also induce cognitive overload. While SMEs may want to provide as much information as possible, it is the instructional designer’s job to choose the most critical content to achieve the learning objectives.
Inattention is considered a hallmark symptom of cognitive overload. The “glazed eye look” represents an overwhelmed state. If a learner is unable to perform the activities or fails the test, this could be a sign that he or she is having difficulty absorbing the information.
Now, what can I do about it?
There are many ways to combat cognitive overload, and we’ll introduce seven of them here.
- Take breaks. A well-timed intermission allows participants to consolidate their learning.
- Allow for a lot of movement, either within the context of the training, or during a scheduled break time. Physical activity allows for more blood and oxygen to flow to the brain, preparing it for more learning time.
- Keep the training focused on the learning objectives to eliminate superfluous content.
- Arrange material in a logical pattern so that learners may build on their prior knowledge.
- Avoid having the learners multitask, and discourage it when you see it in your learners. Despite the stereotypical view of multitaskers as super-competent individuals, human beings are not made to multitask. When we do, an overload occurs between working memory and long-term memory.
- Repeat fundamental material and key points. This may sound counterintuitive to avoiding cognitive overload, but cerebral overwhelm actually results from too much new material. Once a learner repeats a process enough, it can become automatic, freeing up the working memory (theelearningcoach.com).
- Don’t put the meat in the middle. Unfortunately, many programs are structured just this way. Research tells us that the human brain recalls the beginning and ending better than the middle, so introduce crucial material at the beginning of the course, or save it until the end.
What two tips could you implement this week? Did you notice a change in your learners?
Commit to following even some of these pointers, and your learners will retain and implement more of what they’ve learned well after the training is over.
Blended Learning: Look Before You Leap
Blended learning tools provide an instructional designer with many options, but more choices can also be confusing. How do you determine if the blended solution meets your learning objectives? How do you know which modalities to pick? And finally, how do you know that your mix will work within the culture of your organization?
To start creating a blended solution, you first need to know what your goals are and what you want your employees to learn. Is the material strictly knowledge-based, such as company policies or history, or is it skill-based, such as learning how to use a new software program? Or, will the learners need to create something entirely from scratch or evaluate a product, process, or individual? These different types of learning are represented in Bloom’s taxonomy as different levels of the cognitive domain and include knowledge, comprehension, application, and other learning levels.
For EnVision’s clients, we recommend examining the learners when putting together a specific blend. Where do the learners work, and how easy is it for them to travel? Do they work in similar or different job roles? We also advise instructional designers to look at an organization’s budget and efficiency requirements.
Regarding the organizational culture, consider which training methods have worked well in the past, and management’s attitude toward training. Finally, it is important to consider whether or not the material requires peer interaction and facilitator feedback, which are best addressed with a classroom environment. Factual content can be learned via independent study, either reading or elearning.
Julie Young, now an EnVision consultant, oversaw development and implementation of a blended solution on talent management for a global biopharmaceutical company where she worked. The company wanted to support new employees and educate them in the company’s philosophy in talent management, as well as teach practical skills in delivering performance reviews and engaging reports in career development conversations.
The client wanted the learners to master both knowledge and skill-based domains. However, they also wanted to educate the new employees without investing in a series of lengthy classes, said Young. All potential learners happened to work at the same campus.
Because the company philosophy piece was knowledge-based, Young, together with the EnVision consultant partnering with her, decided to implement an e-learning module to address it. They determined that material such as background information and company philosophy would be addressed effectively by elearning, because these learning chunks remained consistent and did not require hands-on practice.
However, the client also wished this group of new employee managers to learn how to write and deliver a performance review, as well as talk with a direct report appropriately about compensation. For these skills, the new employees needed hands-on practice that only a classroom environment could provide.
Young, who delivered the training, states that the company wanted to be efficient with class time, especially given the fact that the number of learners would periodically vary according to the rate at which they were hired. The final product was a four-hour, half-day classroom session of hands-on skill building. “This was a much more efficient way of bringing people up to speed and onboarding,” said Young.
Alison McIsaac, an EnVision consultant, took both learning efficiency and effectiveness into account for her learning blend. As the project manager for a statistics course for a global pharmaceutical company, McIsaac helped to design and pilot the course. While classroom training comprised the bulk of the learning solution, independent reading and computer-based training also augmented the solution.
The learners included professionals from quality control, manufacturing, engineering, and other roles throughout multiple worksites. Prior to attending the 3-day classroom training, learners read a detailed process document and took a 30-minute computer-based training which introduced concepts, provided interactive learning with questions and answers, and assessed knowledge with a quiz. This prerequisite learning enabled the classroom training to focus on applying the concepts to practice scenarios that simulated the on-the-job experience.
As with Young’s project, learning efficiency proved to be an important factor in creating the blend. McIsaac noted that elearning reduces the amount of classroom training, which benefits both learners and employers, since a company loses money every day an employee is pulled out of his/her job. “For every hour you spend in the classroom, you want to maximize that time,” said McIsaac.
When basic concepts are self-taught initially, it also makes the classroom training more engaging for the learners, since there is less lecture and more practice. Interestingly, this particular blend also may prove more effective, because the instructional designer “plants a seed” with the elearning on which the classroom learning later expands, rather than overloading learners at the outset, said McIsaac. “The learners are much more likely to absorb small amounts at a given time,” she explained.
While a powerful option in the instructional designer’s toolkit, blended learning serves an organization best when it follows thorough analysis. The result will be a targeted, appropriate learning solution designed to grow the skills of the organization’s employees.
Through EnVision’s years of experience in planning, designing, developing, and implementing blended solutions, we have prepared a checklist and worksheets to support L&D managers and instructional designers in creating robust, well-constructed blended solutions.
If you would like a complete copy of our blended learning considerations and checklists, please e-mail solutions@envision-performance.com.
Can the Myers-Briggs Help You Achieve a Happy Medium?
Colleen and Stella both work at a medical device company and develop a catalog to market the products. Though they often complement each other well, recently they have reached a stalemate on this year’s catalog drop.
Colleen, a product manager, wants to make sure the catalog hits the printer well in advance of her conservative deadline, that each of her products gets equal billing and promotion, and that the branding and colors in the catalog look similar to last year’s catalog, since that version generated a lot of sales. Colleen has given Stella, the marketing director, specific, detailed notes on what she is looking for on the project, with interim targets ahead of the ultimate completion date.
However, Stella has an entirely different approach. She knows the company just launched a new medical device that is fairly unique and innovative, and has the potential to become a successful product. While she wants to please Colleen, she also wants to ensure the branding is tweaked to feature this new product, and doesn’t mind missing the deadline to allow for these changes. Stella runs the catalog by many colleagues, including several she doesn’t need to. Additionally, Colleen is irritated by Stella’s constantly finding improvements in the catalog and drafting several versions, delaying its completion.
Who is right…Colleen or Stella, and why? The answer is, neither one. Their personality characteristics influence what is most important to each of them, causing the stalemate.
This scenario shows that the most challenging part of each workday can lie in interacting with colleagues; how do we reach consensus with people whose viewpoints are so different from our own? To come to agreement with someone, it helps to understand his/her perspective and approach to the world. The Myers-Briggs Type Indicator (MBTI), the subject of EnVision’s most recent team development day in June, provides tools to illuminate another’s perspectives.
The MBTI consists of four scales, each consisting of a pair of opposite behavioral tendencies. While everyone uses all of the behaviors from time to time, each person has preferences that are illustrated with the MBTI indicator. These show how an individual most commonly interacts with his/her environment.
The first scale shows how individuals get their energy; Extraverts (E) get their energy from the outer world, whereas Introverts (I) prefer inward reflection. The Myers-Briggs also captures preferences on information gathering; people who typically rely on Sensing (S) get their information in “real time,” from their senses, whereas those who rely on iNtuition (N) find patterns in information and make mental leaps to the future.
In the third scale describing decision-making style, people who tend toward Thinking (T) analyze a decision objectively and logically. Someone who exemplifies Feeling (F) makes decisions on how results impact other people, and strives to preserve harmony between others. Lastly, those who are commonly Judging (J) like to achieve closure and resolve decisions quickly, whereas people who favor Perceiving (P) prefer to gather more information and not rush a decision.
The indicator yields a four-letter “Type” that has its own description and way of viewing the world. There are 16 possible personality types with the Myers-Briggs.
If Colleen and Stella completed the Myers-Briggs, they would learn that they, unsurprisingly, have very different profiles. Colleen is a classic ISTJ (Introversion, Sensing, Thinking, Judging,). She decides logically what should be done and works toward her goal systematically. She is orderly and organized, sometimes to a fault, and takes pleasure in this.
Stella, on the other hand, interacts with the world as an ENFP (Extraversion, iNtuition, Feeling, Perceiving). She’s imaginative and connects ideas quickly – such as the promise of the new product and the potential to promote this in the catalog. Stella also wants approval from others. She doesn’t prioritize the deadline over continuing to work on the catalog; as a “P,” she is fairly relaxed about meeting her deadline.
Colleen and Stella then took the MBTI, which resulted in greater understanding of each other’s point of view. Colleen agreed to incorporate Stella’s valuable ideas, even though they produced a different catalog than last year. She also allowed more flexibility on her deadline, while Stella in turn cut down on the number of catalog iterations. They also specifically tailored their discussions in meetings, especially when hitting a communication roadblock, something Kathy Maloney, an EnVision consultant, advocates.
During EnVision’s team development day, Maloney and EnVision consultant Ginny Maglio kicked off the session by showing the attendees a photograph of a busy New York City intersection, asking them to state what the picture brought to their minds. Whether people remembered a specific item from the picture or took away a certain feeling from it, their observations reflected their MBTI type.
Moving into the core of the session, Maglio and Maloney led the group in real-world, job-applicable exercises that incorporated organization, problem solving, and decision making. They handed the attendees a scenario of starting a new L&D project, a program on diversity, and split the attendees into two groups. They then asked two groups how they would plan the project.
During this exercise, EnVision’s team learned which elements they prioritized, where they placed emphasis, and how this was reflected in their own MBTI type. For example, someone who has a preference for Sensing and Thinking would want to know why the client believes they have a need for diversity training, what data is available to support this, and which performance improvements the client is seeking in its employees, explains Maglio. These are all task-oriented focuses. An iNtuition/Perceiving type would focus not only on the content of the curriculum, but also on how each team member can have an impact on the learning solution and give members choices about what roles/responsibilities they want to assume, said Maloney.
In the second half of the exercise, each team determined how to complete the project. Maglio and Maloney chose these exercises to mimic real-life work with clients, and also encourage team members to be aware of their MBTI profile to best work with their colleagues. “[The MBTI] is an understanding of yourself and others to work more effectively together,” said Maglio.
In addition, the Myers-Briggs can be used to problem solve. If MBTI types are missing from a work team, members can adopt those preferences, using their past experience. This ensures the problem is approached from all perspectives, leading to a more comprehensive solution.
Most often, company leaders introduce the MBTI when there is a specific need, such as an interpersonal issue within a team, said Maloney, like the example with Colleen and Stella. “To me, there needs to be a business need, and an interpersonal need,” said Maloney. “It’s a great tool, but not the only tool.”
Using the MBTI to put a team together would be a misuse of the indicator, said Maloney. Because the Myers-Briggs does not test ability or knowledge, it should not be used to assess employees’ abilities or performance. Frequently, people may refer to the Myers-Briggs as a “test,” though it is not.
While limitations to the MBTI exist, its framework based on the personality theories of Swiss psychiatrist Carl Jung still endures today. The indicator remains one of the most widely used psychological tools worldwide. Add it to your arsenal of tools to have your employees understand each other better, like Colleen and Stella, and grow more productive.
Create Learner Value with the Virtual Classroom
Independent elearning offers convenience for the learner and ease of delivery for the client. Yet, sometimes there is no substitute for discussion with one’s peers, feedback from the instructor, and hands-on activities to cement learning. Can the advantages of elearning and interaction of classroom learning both be addressed in a different modality?
Meet the virtual classroom. With this modality, learners sit where they like, with their own computers, while experiencing the give-and-take of a classroom setting. The virtual classroom enables learners to participate remotely, but in a real-time, synchronous manner, unlike elearning, in which participants learn asynchronously or independently.
In a virtual classroom, learners access training via Adobe Connect, Blackboard Collaborate, or another platform. When they sign on, they may see PowerPoint slides and/or a white board, and sometimes a video of the instructor speaking. In addition to the instructor, there is a technical assistant working “behind the scenes” to ensure the technology and interaction between the instructor and class run smoothly.
Learners actively participate in the class in a number of ways, states Marilyn Kobus, an instructional designer with EnVision. They can “talk” amongst themselves via an online chat feature, or use a text tool to write on one of the instructor’s slides, visible to all the learners. Just as in a traditional classroom, the learners can talk directly with the instructor and with each other. Depending on the size of the group, participants may speak up directly or click on an icon on their desktop to “raise their hand.” The instructor can see the raised hands, and then call on learners to speak.
The instructor can even divide the participants into breakout groups, where they work on an activity, speak amongst themselves, and capture notes on their small group’s white board or notes pod. They can then return to the larger group with their notes to present their findings —just like in a traditional classroom.
A virtual classroom does not equal a webinar. The two differ both in their size and the manner in which the learning is designed. Hundreds of people may attend a webinar, whereas a virtual classroom ideally accommodates 12 to 16 participants.
The virtual classroom allows for interaction with the instructor and peers, as well as practice and hands-on learning, whereas a webinar is a nearly 100% lecture format. “If you really want people to learn skills, you’re not going to get that in a webinar….there’s no practice,” explains Kobus. “Webinars are good for sharing information.”
Virtual classrooms offer many practical benefits to learners and their organizations. Professionals obviously don’t travel to take or deliver the class, so costs decrease. Virtual training may be easier for learners to work into their busy schedules, since content is broken into smaller chunks spread out over time, rather than covered in one or two full days as in a traditional classroom. These smaller learning sessions usually include robust prework assignments, as well as inter-session assignments that enable learners to put into practice what they are learning. This approach makes the best use of in-class time with an instructor while fostering application of learning in action. The opportunity to take what is learned in the classroom, immediately practice it on the job and then return to troubleshoot is a luxury that is not often available in a typical classroom-based workshop, according to Kobus.
Another benefit is that managers and instructors may opt to record a virtual classroom session and reuse it or make it available as a refresher, though it will lack the interactivity and spontaneity of the first iteration (from “Interactive Classrooms,” Rachel Griffith-Boyes and Mark Aberdour, September 2013). Recordings could also augment existing training as a review or job reference.
Despite all the benefits, there are crucial points the instructional designer must consider before embarking on a virtual classroom as a learning modality. First, the L&D professional must believe the class can benefit from learning in a group setting, versus independently. If the curriculum includes prework, the instructor should be sure to discuss the prework and expand on the learning in class. Also, the course must be as interactive as possible, whether the tools used be polls, chats, discussions, or group activities to deepen skill development.
The faculty member must also exhibit different skills in a virtual setting than when teaching in a traditional classroom. He or she needs to be particularly energetic and responsive; there is no student body language to observe, so the instructor must actively encourage a balance of participation in the classroom. Because of the technology barrier, the instructor may need to work harder to keep learners engaged; for example, the instructor can call on learners frequently.
Also, the instructor must be a master juggler. In any given moment, there could be learners with their hands raised while others are chatting, or there could be one or several learners at a computer, and the instructor needs to remember the location of each one. Basically, the instructor needs to multi-task and execute seamless choreography with the technical assistant, all without losing command of the classroom.
Technology challenges, of course, cannot be predicted. Between audio and software applications, things can and will go wrong. In “Interactive Classrooms,” Griffith-Boyes and Aberdour advise the reader to “always have a back-up plan.” Some technology issues, however, can be reduced or addressed quickly with proficiency in the software program, says Kobus.
EnVision recently adapted a classroom curriculum on an ethics-related topic to a virtual classroom setting. The curriculum consists of six modules covering a framework on handling ethical issues, and is geared toward social workers, nurses, and mental health professionals.
This learning solution encompasses both independent learning and virtual classroom time, which allows the learners to get grounded independently in key concepts before the session and then practice the material they learned during class, according to Kobus. The recent pilot of the first module ran smoothly, and design work continues on the remaining modules.
The virtual classroom allows for an engaging, cost-effective experience in an interactive format. Remember that independent learning, no matter how convenient, does not enable learners to discuss and practice the way a virtual classroom does. If you’d like to learn more about developing a virtual curriculum for your company, contact EnVision.
Creating Your Own Sweet Adventure
“When it comes to chocolate, resistance is futile.” – Regina Brett
Like many people, I have a weakness for chocolate. For a while, I had wanted to take a chocolate walking tour in Boston, enjoying the city while sampling delicacies from various specialty shops. This would be a tasty way to be a tourist in my own backyard, I figured.
So, like any diligent scholar of chocolate, I did my research. I soon found a very convenient option – a chocolate walking tour located right in the Back Bay. This sounded ideal, until I saw the tickets were $40-$50 each.
After this bit of sticker shock, I decided to plan my own Boston chocolate walking tour, putting my creativity and organizational skills to work. Via Google, I located several shops on Newbury and Boylston streets, and off my husband and I went for the weekend.
Our outing included The Melting Pot, where we indulged in chocolate fondue; L.A. Burdick’s, a delicious shop on Clarendon Street which sold its signature chocolate mice; and Max Brenner’s, both a chocolate bar and restaurant, where we sampled cocoa powder-dusted waffle fries.
By far, the most unusual chocolate we encountered (no, we didn’t sample) hailed from the Gourmet Boutique on Huntington Avenue. Camel milk chocolate from Dubai, anyone?? In total, we sampled sweets from 10 chocolatiers, learning more about chocolate than we ever had before.
My husband and I agreed that L.A. Burdick was our favorite. The chocolates were simply delicious and creamy, and the interior of each integrated flawlessly with the chocolate exterior. Based in Walpole, NH since 1992, owner Larry Burdick learned the chocolate-making craft in both France and Switzerland.
In visiting the shops, I noticed different attitudes among the storekeepers. Some held us spellbound, describing the origin of their business, how they learned the craft under master chocolatiers, and the features of their trademark candies. You could see and feel their passion for their craft. No matter how busy they were, they took the time to speak with us and share their love for chocolate.
At other stores, the owners did not tell us about their specialties, but rather complained about the hours they worked. Despite working to create and distribute a product that makes most people happy, these people didn’t seem content with their work.
In spite of a few overworked chocolatiers, we greatly enjoyed the tour. I also found parts of the outing that correlated to my work.
With a chocolate tour or with instructional design, choosing a theme concentrates the focus, adding to the enjoyment of the learning experience. For example, had I tried to intersperse portions of the Freedom Trail with our chocolate tour, I don’t think I would have enjoyed the experience as much. Part of the fun was knowing that these two days were devoted to chocolate.
I also saw that enthusiasm really makes a difference, whether it is training or chocolate-making. As a consumer, I found myself impressed by the storekeepers who showed excitement for their chocolate-making, and it positively influenced my opinion of the store. Similarly, learners absorb the material when the design—and instructor—is engaging; it is difficult for them to maintain interest in a dull class.
Finally, I was reminded that learners often rely on take-aways to master new material. In a corporate training class, this could be a job aid or cheat sheet to help translate classroom learning to on-the-job success. For our tour, of course, the takeaways were the chocolate samples we were unable to sample during the tour. We’re going through our take-aways, little by little, remembering the fun we had that weekend and what we learned.
Ingenious Ways to Improve Learner Retention
As instructional designers, we focus on improving employee performance through knowledge and an increased skill set. We strive for learners to retain what they learn and implement it in their jobs.
Yet, with so many competing demands at work and home, retaining what we learn can be easier said than done. For the instructional designer, any guide or “rules” to increase learner retention could really help.
Most recently, I have found such a guide in the book Made to Stick by Chip Heath and Dan Heath, which describes six sticky principles to incorporate in our ideas and educational material, to make them more memorable and meaningful. I felt so strongly about this book that I made it the cornerstone of EnVision’s 15th EnVisioning our Future team workshop, held this past December.
The first principle we learned at the workshop was simple, or summarizing the essential core of your idea. Nike’s slogan, Just Do It, is a good example of this. Alternatively, the second principle happens to be unexpected, which means surprising people to get their attention. By invoking the senses or enhancing a scenario with details, one uses the concrete principle of stickiness. Appealing to emotion or inserting credible facts/testimonials will also help learners retain material. Finally, introducing a story and carrying it through a training workshop, book, or any other creative endeavor can also help concepts to stick.
After learning about these basic principles in the workshop, we broke into teams to analyze an instructional design case study, finding ways to improve our chosen course using these elements of “stickiness.” In my group, we found ourselves brainstorming original instructional design methods for a financial management course at a major Boston hospital.
While sound financial management is clearly important, the challenge lies in making this material engaging for the learner so s/he retains it. Between three of us, we thought of several ideas to make financial management “stick.” As an example of the concrete principle, an old-fashioned weighted scale could represent the importance of having accounts balance. An easy-to-remember slogan (“haste makes waste” relates to financial management) provides an overarching theme for a learner’s performance improvement and illustrates the simple concept.
Finally, our team recommended that the story principle could help show the learners the degree to which hospitals rely on sound financial management. The trainer could share a story of a hospital that budgeted well, and one that did not, and the consequences of each scenario, while following this story throughout the training. The relevance of a real-life scenario brings the material alive for the learners, placing it in a larger perspective. “…people care more about individuals (or individual hospitals, to paraphrase) than they do about abstractions,” write the Heath brothers in the epilogue of Made to Stick.
I found Made to Stick a novel way of looking at my field of instructional design. Even better, these principles can be applied to the fields of marketing, writing, and more. The Heath brothers have given us all a valuable framework to “raise the bar” and provide more value-add to our learners.
Forging a Path in the Snow
While many New Englanders bemoan this harsh winter, I find I embrace most of it; well, maybe not the snow removal part. Since one of my favorite pastimes is snowshoeing, winter – especially snowy winters like this one—enables me to indulge in it.
What keeps me attaching these cumbersome apparatuses to my feet, and trudging through the cold air and snow, time and again? When I enter the wilds of my local conservation land, I leave the modern world and daily annoyances behind. In forging a path, I experience the sense of wonder that comes to me from immersion in nature.
I enjoy marking my footprint in the clean, fresh snow. When several of these footprints are strung together, they make a path. I don’t always recognize the direction of my path. Sometimes, the journey is familiar; other times, I get lost. Yet, I always feel fairly relaxed, knowing I can simply retrace my snowy steps, use landmarks as a guide, and return where I began, ready to forge a brand new path. I may even make new discoveries along the way.
The instructional design process correlates to snowshoeing. My journey starts from the moment I put on my snowshoes, much like the client kick-off meeting launches the instructional design journey. Out in nature, I scan the horizon for a place to go, just like the design document and its learning objectives lead to a destination for the learners.
When I forge my path, some parts are easy going and others difficult, and I may get lost in designing a course, too. Yet, if I turn around, and retrace my steps, I can get back to where I veered off course. I can then think about where I am, look around, and choose a different trail that leads to the same destination, the learning objective. Perhaps the new path will be more clear-cut. There is a lot of freedom in knowing I can always change my path.
When we get lost or stalled as learning professionals, it’s important not to give up on the learning objectives. We can, however, revisit our journey to see where we drifted off course. We hope our new path leads to discoveries for our learners, too.
Get to the Head of the Class in Writing Learning Assessments
A college midterm for Introduction to Psychology. The Certified Public Accountant (CPA) exam. The MCAS (the Massachusetts Comprehensive Assessment System – if you’re a parent to a school-aged child, you know it!)
What do these three items have in common? Yes, we commonly refer to them as “tests” or “exams,” but like the MCAS acronym suggests, they are assessments, designed to determine how well learners have mastered given material. Whether the student learns the material as an accountant or while still in grade school, assessment questions must connect to the learning objectives and reflect an appropriate level of difficulty.
With these tenets in mind, I co-led a training session on Writing Learning Assessments to 40 training professionals at a large pharmaceutical client site back in September. I was excited to deliver the information to this experienced group, several of whom joined the session virtually.
We built the class around the learners defining best practices. First and foremost, it is crucial to ensure that the assessment questions are directly tied to learning objectives, to see how well the employees meet these goals. The learners will master the material better if they apply their knowledge in a realistic scenario, rather than merely parrot back what they have learned. Also, I advised the instructional designers to have someone “test their test” to evaluate the assessment’s timing, difficulty level, clarity of questions, and more. I also recommended considering the test-taking audience; sometimes a question may not apply to all learners.
Predictable answers can doom an assessment. From my experience, I have noticed that many correct answers are often either the most detailed choice, very obvious, or the infamous “all of the above.” Test writers should review their assessments for these types of answers. A high score on a test needs to reflect how well the learner masters the material, not how well he or she strategizes the assessment experience.
Some of the training professionals I met that day knew these tips. However, I shared one best practice which threw many of them a curve: During which stage of ADDIE (Assess / Design / Develop / Implement / Evaluate) should an instructional designer begin creating test questions? The answer is below! **Hint: instructional design professionals need to start thinking of assessment questions early, as this drives course content as well as provides ample time to review and test the questions.
This group of training professionals also needed to pay special attention to the difficulty level of questions, since they work in the pharmaceutical industry. We learned that the FDA includes training in their audits of life sciences companies, and that they are increasingly assessing the difficulty level of test questions to ensure that the employees truly master the material. At the same time, test writers must write questions that learners can comprehend, so this process is a balancing act.
I led several activities to solidify the instructional designers’ knowledge. After reading sample questions, teams rewrote them to increase clarity and effectiveness. They also wrote original questions based on a case study provided by the client. Finally, we directed the test writers to determine an action plan that they could implement during their workday.
What did I observe from leading this workshop? When I read many of the groups’ test questions, I was surprised how many asked learners to repeat facts, rather than apply knowledge. This omission reminded me of learning math in school; while it is important to know basic math facts, you truly master a math concept when you solve a word problem. Math facts don’t test a learner’s true ability as much as his/her memory. And for memory assistance at work, of course, there are job aids. The most competent workers will be those who can navigate a real-life challenge quickly and easily, illustrating their proficiency. Walking out of the workshop, the trainers understood this better than they had coming in.
** Answer: “Design.” However, if you answered “Assess” you are off to a good start, and should exercise caution in writing the questions until you have an approved set of learning objectives. The majority of the test writers in the workshop selected “Develop.”
Making New Discoveries Next Door
Recently, I was privileged to enjoy a stroll with Free Tours by Foot (www.freetoursbyfoot.com/boston-tours). I really love walking tours, and decided to visit Harvard Square, a place I had been many times before.
I soon gained a new perspective of this historic area favored by out-of-towners. I got a good view of the Widener Library, a large brick building in the Beaux-Arts style complete with two imposing columns and a large staircase (examiner.com). Walking by Lowell House, I heard the replica of the famous Danilov bells ringing. I marveled at the intricate stained glass windows in Sanders Theater, the site of speeches by Winston Churchill and Martin Luther King Jr. (www.fas.harvard.edu/~memhall/sanders.html). And I chuckled when I spotted the Dewey, Cheetem and Howe sign, the nameplate representing the fictional law firm at the corner of Brattle and JFK streets. All of these landmarks appeared new to me, though I had walked by them countless times before.
This experience made me realize how often we pass things by without seeing them at all. It also caused me to reflect on my work life, and what one can discover by looking at situations with new eyes. While we may not expand our work horizons with a walking tour, we can stretch our minds by brainstorming with colleagues, or by participating in a professional development session or networking event. We can simply open our minds and become willing to see and do something new and different, rather than stick with the tried and true because “we’ve always done it this way.”
This willingness and effort to gain a new perspective helps us in our professional lives. As instructional designers or trainers, we could gain a new way of presenting material to learners, such as a student self-assessment, an educational game, or unique role play. We could discover a new way to make a meeting engaging. As emotional intelligence continues to contribute to success in the workplace, finding a new way to approach a challenging topic with a colleague opens the line of communication, enabling us to collaborate. Most importantly, being open to innovation gives us the freedom to find a solution to an issue, whether the challenge be in employee performance, cost overruns, or employee turnover.
Marcel Proust wrote, “The real voyage of discovery consists not in seeking new landscapes, but in having new eyes.” Let your new eyes lead you to all your best ideas in 2014. And when you have a chance, check out Free Tours by Foot. I hope the tour inspires you, also!