4. What do you want out of the relationship?
In our previous posts we shared client and vendor insights about scoping out a project, defining success, and choosing a vendor. Today, we examine what we each want out of the relationship – being aligned on this is an important factor in a fruitful project and an ongoing relationship.
Trust is important for any good working relationship. As the client, I start to gauge trust during the vendor selection process – I listen carefully to their responses for things such as:
- Examples they provide that demonstrate instructional soundness and alignment with best practices
- Confidence describing their successes and issues they had to address – every project has a problem/challenge requiring clarification
I like to have another internal resource in the meeting when possible, either an SME or another L&D professional, to provide another perspective. And I also listen to my gut.
If I’ve already worked with a vendor, I also consider our past working relationship. Did it include good communication disciplines and consistency delivering quality products? I look for reliability, too—that they did what they said they would do. The biggest benefit of working with a previous vendor is a quicker start up time.
If I am working with a new vendor, I spend a little more time upfront to orient them to our communication process. While I hope for smooth sailing, check-in conversations are an opportunity for feedback and seeing where I could be of help. I want to remove internal roadblocks for the vendor if needed. I request status updates regularly and expect to receive reasonable and knowledgeable responses.
I don’t expect perfection; we are all human. I value dependability and keeping commitments, which build mutual respect. The vendor should be able to expect the same from me.
As the vendor, I want a lot of the same things out of this relationship. Trust and open communications are important because they provide the basis for all our work together. If there are difficult issues that arise, trust and open communications help both the client and I to manage the issues most effectively. I usually ask a lot of questions before and at the beginning of a project to clarify that we are on the same page. A process is also important to me and I am sometimes concerned when my client doesn’t seek a discipline around communications and project management. In those cases, I try to set up a plan they can work with. It’s important for me as a vendor to be willing and able to flex with the client’s needs.
Every client relationship is different. I want my team to be able to positively impact the client and not just do busy work for them. A recent example comes to mind in which we were invited to bid on a large, interesting and complex project for a government agency. In the agency’s question-and-answer period, they often responded to questions vaguely and with very little context. Without knowing the client, their needs, and how they operate, we didn’t feel we could reliably address their needs. There was no opportunity to sit down and have a real conversation with them. And, honestly, it made us question the challenges that could arise during the project, should we be awarded the bid. We decided not to proceed further in the bidding process.
In the end, I believe that building trust is a mutual endeavor and one in which I will take the lead. As David Maister, et. al., write in The Trusted Advisor, “…trust must be earned and deserved. You must do something to give the other people the evidence on which they can base their decision on whether to trust you. You must be willing to give in order to get.”
Keys to success
CLIENT: Start gauging trust during the interview process, spend time at the start of a project agreeing with the vendor on a communications approach, and keep an open line of communication throughout the project.
VENDOR: I mirror Veronica’s summary of the client’s keys to success: Start building and assessing trust during the interview process, take time to agree on a communications approach from the start, and keep an open line of communication throughout the project.
Veronica Clements, formerly a Learning and Development leader, with 25 years’ experience engaging vendors.
Irene Stern Frielich, President of EnVision Performance Solutions, with 20 years as a vendor providing solutions to clients.
3. How do you identify and select the best vendor for your project?
In our last two posts we shared client and vendor insights about scoping out a project. In this month’s installment, we offer suggestions for selecting the best vendor for a project.
I begin my search for a vendor by checking with peers and colleagues, both in my company and with professional organizations to which I belong. I might also check our corporate-approved vendor list. Vendors referred to me by my peers and colleagues are usually my first choice.
Usually I identify three of the highest recommended vendors and contact them to set up a time to discuss availability for the project, pricing, and the services they offer.
If possible, I like to meet with the prospective vendors in person. If that’s not possible, a video conference is the next best thing. During the meeting, I share with them the project scope, measures for success, and our timeline. Some typical things I request are:
- A demonstration of a course or walkthrough of sample training materials (for example, a leader’s guide or an elearning storyboard/prototype).
- Work samples and CVs from the consultants who would be assigned to the project.
- Confirmation that those consultants are available to work within the project deadlines.
- References I can call.
Here are some interview questions I like to ask:
- Tell me about a project that went well. What do you attribute that to?
- What project did you learn the most from and why?
- Tell me about a project that was behind due to a client’s issue. What happened?
- Do you have concrete examples of effective communication strategies with clients in a challenging situation?
I listen for specific responses that illustrate how they execute their processes, work with clients, and manage conflict. I take notes to refer back to and also ask probing questions for more details as needed. Based on their responses, I hope to confirm their know-how along with the value they can bring to my project.
As a vendor, it is important to have relationships with a large number of individuals, companies, and organizations so that you are “on the list” as a potential vendor. The most effective way I’ve done that is through networking and staying in touch with prospects as well as past and current clients, reminding them about the scope of services we offer and updating them about our newer, innovative services. Other potential avenues are through procurement databases, leveraging our certified woman-owned business status, and through speaking and writing opportunities. I also ask clients to share with their colleagues how they like working with EnVision. We uncover many opportunities through word of mouth.
For clients newer to the world of engaging vendors, I’ll also offer guidance for selecting a vendor. For example, I’ll suggest they check references, with specific questions about ease of working relationship, scope of capability, and other things that are important to this client. And, I suggest keeping in mind that just because one vendor was great for a particular type of project, it doesn’t mean they’ll be great for this particular project. So, interviewing the vendor and talking to people who worked with them is really important.
I prepare well for the interview. I have work samples ready to show that are related to the type of work the client is seeking. And, based on previous conversations with the client, I anticipate the types of questions they might ask and prepare stories about our recent successes as well as challenges, and how we handled them.
Keys to success
CLIENT: Ask behavioral questions and review work samples and examples the vendor provides to support their proven success with similar projects. Find out about the communication methods they use for projects.
VENDOR: Network, offer guidance to clients navigating the vendor engagement process, and be prepared for whatever might come up during an interview.
Next time: Considering what else you want out of the client/vendor relationship.
Veronica Clements, formerly a Learning and Development leader, with 25 years’ experience engaging vendors.
Irene Stern Frielich, President of EnVision Performance Solutions, with 20 years as a vendor providing solutions to clients.
2. Define success: How do you define what success will look like?
In our last post we discussed problem definition. As part of that we delve into defining success.
As the client, an important part of preparing a request for proposal or discussing my needs with a prospective vendor is to determine results I want to see at the project’s conclusion. In other words, what will success look like for the client and the vendor?
In the example from part 1 in this series, a new process is introduced and the employees need to be ready to use the new procedure starting in three months. Based on the project goals and discussions with the stakeholders, I would prepare target outcomes for the learners and the managers, such as:
- 95% of participants pass the final learning assessment with a score of 80% or better.
- Managers spot check key steps while employees are working and provide reinforcement a few weeks following the course.
- Employees are following the new process, improving accuracy to 99%.
In this example, our result for passing the final learning assessment measures learning outcome. The quality check by managers supports application of new skills on the job. There is an existing department process in place to measure the improvement to accuracy, specifically to improve accuracy to 99%.
Sometimes, as an internal resource, I don’t always see all elements of the big picture. Therefore, I frequently benefit from hearing the vendor’s additional suggestions as well as confirmation of our success measures.
I, the vendor, really like it when a client has defined success. But that isn’t always the case and, in those instances, I ask more questions! For example, I often ask something as open-ended as: What will success look like? Or: At the end of this project, how will we both know if my team was able to provide what you are looking for?
Once we have some success metrics, my next step is to ensure the metrics are something we’d be able to influence. For example, if the client expects us to ensure their managers spot check key steps as a measure of success, we need to be engaged in creating a solution to support managers in doing that. But, often, there are “environmental” – or non-training-related influencers – on the success of training. So, we’ll need to agree on our role as vendor in this engagement. However, if we are simply being asked to create a training program on a process for employees performing that procedure, I would let the client know that only the first measure (95% of participants pass the final learning assessment with a score of 80% or better) is fully applicable to our work.
Keys to success
CLIENT: Define measures of success for the project, obtain input from prospective vendors, and clarify responsibilities for each success measure.
VENDOR: Define the client’s “success” expectations and be clear about which of these the vendor can address.
Next time: Selecting a vendor.
Veronica Clements, formerly a Learning and Development leader, with 25 years’ experience engaging vendors.
Irene Stern Frielich, President of EnVision Performance Solutions, with 20 years as a vendor providing solutions to clients.
What happens when you are asked to take on more work than you can handle– either due to capacity or capability? Let’s say your manager just laid a huge project on you – to develop a new training curriculum for a group that has been having some challenges. Because you don’t have the resources you’ll need to get the work done by the date it is expected, you consider hiring a vendor.
In a series of posts, we propose five guidelines to help you, as the program manager, select a vendor and lay the groundwork for a successful relationship. At the same time, we share tips for the vendor about positioning yourself to become a strong partner with the client. We have been in your role, we gladly share our experience in this series, and would be happy to talk with you about our ideas and hear yours.
The first step for the client is to define the problem.
1. How do you define the problem and related supports and challenges?
Who has ever had a short-term project thrust on them seemingly out of nowhere? Or maybe I should ask who hasn’t?! I get those requests more often than I would like to admit! Imagine yourself with a new project you are quickly trying to scope, knowing you need to find an L&D vendor to work with. Where do you start?
As the client, I begin by defining my problem. Here’s an example of a project definition: A new process is being introduced. If the employees don’t follow the new process, errors and rework would result, causing frustration to internal customers. I determine that I need a course developed about the new process and I expect 200 employees to be trained in the next three months. In preparation, I identify related supports for the project, such as available subject matter experts (SMEs) and pertinent existing content and documentation. Next, I identify any potential challenges, such as the process that is still being updated or the target training launch date just 3 months away.
After review, I determine I don’t have adequate internal resources to work on this project, so I ask vendors to prepare a proposal to address my problem and the challenges I’ve listed. It is important for me to gather basic needs assessment information from the stakeholders before finalizing a request for proposal. This helps me to clarify course goals, project resources, and high-level timelines.
The next thing I do is define success, which we will discuss in the next blog post.
When I, as the vendor , receive a request for proposal or am discussing a new project with a client, I ask a lot of questions. I recognize that my client might be deeply immersed in her challenge and may find value from the objectivity I bring. Based on the project outlined, I’ll ask more about context. For example, why was the process changed? What problem was that intended to solve? If it sounds like the client conducted a high-level needs assessment, I might add: What were the results of your needs assessment? What were the surprises? What challenges were identified? When I as the vendor ask these types of probing questions, I gain some insights and get more clarity about results the client is expecting. Sometimes the conversation raises additional questions the client may need to research further.
I’ll also ask about existing related documentation, such as procedures, presentations, and process flows, as well as a list of SMEs available to work on this project. If there is little existing documentation, we’ll need the expert to spend time “brain dumping” for us and we’ll make that expectation clear from the start. I’ll also ask more questions about the audience – why are they not following the new process? What, besides training, might help them? What has already been tried and how/why did it help or not? Basically, I want to ensure that if we do prepare a proposal and ultimately engage in the work, we are able to have a positive impact.
I will also ask questions about the client’s internal capabilities and capacity. For example, if this will be an instructor-led classroom course, are there skilled trainers to deliver the training? What level of detail is needed in the instructor guide we create? Or, if it is an online course: What authoring tool does the client’s organization use? Is there an internal resource who will make post-launch course updates? This will help me determine the tools we will use and our approach in documenting the production notes and programmer explanations.
Keys to success
CLIENT: Complete basic information gathering to define and document your problem with related supports and challenges.
VENDOR: Ask questions to clarify the issue, gain context, and uncover additional challenges and supports in order to write a proposal with clear project scope.
Next time: Defining success.
After a wonderful few days spent with family and friends, I feel reenergized and ready to meet 2019 head on! How about you?
For many of us, the new year brings challenges, like reprioritizing our time so we’re focused on the right things or learning the next great tool or process or figuring out how to do more with less at work. And if we manage others, we may also need to help them prioritize, learn, and grow themselves.
Here are a few ideas to help you gear up for 2019.
- Find or be a mentor. If you’re in the earlier part of your career or exploring a new area and don’t have a mentor, find one. By having a mentor, you’ll be able to learn and be connected to others in a way you wouldn’t be able to on your own. Plus, if there comes a time when you need professional advice, a mentor is a great person to turn to.
If you’re a seasoned professional, then consider mentoring someone else. As a mentor, you can positively impact someone else by passing on your own insights, and you might even get some gratification out of being a mentor!
- Take a class. Whether you want to learn about QuickBooks or quality control or tai kwon do, there’s a class out there for you. See what your company or professional associations or community education groups offer. Locate online learning opportunities such as lynda.com or free MOOCs (massive open online course) offered by HarvardX. I actually learned how to calibrate my oven by melting sugar in it when I took HarvardX’s online food and chemistry course a couple years back. You may even decide to branch out and audit an academic class at a university, perhaps your alma mater.
- Network! Find a professional association to join and commit to attending 3 events in the next 6 months, or reach out to individuals you’d like to get to know better and invite them to coffee (with a focus in mind, of course). While many of us promise ourselves we’ll network, too often this pledge gets tossed aside in the hustle and bustle of everyday life. Job opportunities, new projects, and great ideas or connections can come from networking, and it’s also fulfilling to help someone else out. Networking creates new relationships and helps deepen established relationships to create a fabric of professional support — one that is invaluable for everyone.
- Set goals. Goals aren’t just for an annual performance review at work. Set your annual career learning goals and also set some personal targets. My personal goal for 2019 is to complete a first draft of a book I am writing. It’s easier to stay on track if you break down the goals into manageable tasks and evaluate them each week or two. Establishing goals that are clear and specific is best. For my book writing, I have a goal of completing three half-day writing sessions per month.
- Keep an open mind. Sometimes new opportunities arise when you least expect them and sometimes things just don’t go as easily as you expected, so goals need to be flexible. I’m not sure how realistic my 2019 book-writing goal is, so I plan to reassess my progress every two or three months and adjust my goals or focus as needed. Maybe I’ll need to write more frequently or give myself another few months to complete my draft, or perhaps I’ll need to spend less time watching the Great British Baking Show to fit in more writing!
Hope you have a great start to 2019!
Imagine that you work as a manager, heading up a department of six people, and tomorrow Alex, one of your reports, plans to attend training all day. He let you know that he’d be attending this class on project management, and that he needs someone to cover for him while he’s away. Knee-deep in work yourself, you ask one of Alex’s colleagues to cover his crucial items.
But do you know what this training is really about? And do you know what you should expect Alex to be able to do differently as a result of attending this class? You get frustrated when Alex completes projects quickly, but misses key steps he probably shouldn’t at this point in his career. Will this performance need get addressed at this training session? You’re now wondering if you should have paid greater attention to managing his performance and determining his training needs.
Instructional designers and subject matter experts can put all of their expertise and effort into creating an effective course for learners. However, if the employee’s manager doesn’t involve herself or support the effort, the knowledge and skills learned may not “stick.”
Mary L. Broad, in Beyond Transfer of Training, describes managers as stakeholders in the performance improvement process. “There are many stakeholders in complex organizations who have strong interests in effective performance and who can provide necessary support for the performance. For all interventions to develop or improve performance, primary stakeholders include: supervisors and team leaders, and—for strategically important interventions—top executives and managers…” ¹
How should the manager get involved in training for his reports? Let’s consider how a training program resembles a successful sale—both offer important solutions to the end user. As a salesperson pursues closing a sale, the manager aims to improve her team’s performance. This doesn’t happen instantly, however, just like a major sale doesn’t. There are several actions managers can, and should, perform before, during and after to support learning and help ensure it is successful.
Introduction: Prime the Learner
A salesperson begins his job by getting to know the prospect—uncovering their needs, introducing services to meet those needs, and establishing a good relationship in the pre-sale stage. Managers should similarly take advantage of the pre-training time, referred to as “priming,” to set the stage for the employee to learn.
Before training can occur, what items does the manager tackle? In the priming stage, she “helps to identify the performance need” that the training will address, according to Broad. It’s important that manager familiarize herself with the training program, including its learning objectives. Beyond that, a lot of the “introduction” tasks include logistics, such as giving the employee time to attend the class and complete any pre-work ahead of time.
It is the manager’s role to talk with the employee before training with three goals in mind: to share the value of learning, hear what the employee hopes to gain from the training, and talk about performance expectations once the training is complete. Understandably, a big part of what the manager does is support and encourage the employee, which will help the training go smoothly.
Priming often involves a partnership between the manager and training function. Before training begins, the person responsible for delivering the training may share a checklist of questions with the manager, so she can discuss the course material with the employee.
The Sale: Close and Provide Support
A sale is made! When the sale closes, the customer learns how to use the product or service, often with the aid of customer service. The salesperson may contribute in ways that enhance the customer’s experience, such as offering specific suggestions related to the customer’s situation. Similarly, the manager contributes in ways that make it easier for the employee to learn. The manager ensures that a colleague covers the employee’s work in his absence. The manager should also provide any needed materials, like lab equipment for technical training, for the employee to use and practice on.
Post-Sale: Follow Up and Cement the Learning
After the product or service has been purchased, a salesperson follows up with the customer to make sure the solution worked as promised and to see if any other tools or support are needed. Similarly, the manager sits down with the employee after training for a follow-up meeting. In this conversation, the manager aims to find out what the employee learned, which items he may have struggled with, and the top two or three things he may want to do differently.
The manager should observe the employee’s behavior and work habits to see what successful changes he has made, and provide both positive and constructive feedback. These post-training activities can both strengthen the manager-employee relationship and help the employee apply what he’s learned.
Need help with keeping track of what activities managers can do to support their employees attending training, and what sample questions managers can ask before and after the training? Click here for EnVision’s checklist.
¹Broad, Mary L. (2005) Beyond Transfer of Training: Engaging Systems to Improve Performance. San Francisco, CA: Pfeiffer, An Imprint of Wiley, page 31.
Have you been in a situation where you’ve engaged a Subject Matter Expert (SME) from your organization to develop a training program? It can make a lot of sense to do that – she will have the content knowledge and experience to pass along to others, and she will be thrilled to share it—it’s her work, and she finds it meaningful.
There’s a downside, though. Courses designed by experts with little ID experience often won’t make the grade. They can be too long, include too much irrelevant or “nice to know” content (because, after all, doesn’t the SME want to share all the information she loves so much?), or don’t engage the learner. The result? You’ll have courses that aren’t as effective as you might need them to be.
Other times, you may hire a novice learning professional into an ID position. You assign him to develop a course from scratch, hoping his lack of knowledge of learning science and current best practices won’t be an issue.
I have encountered both of these situations many times, and I’ve seen L&D managers struggle with educating their novice IDers, usually due to a time crunch.
So, how can you get your novice IDer up and running? One approach our clients have used is targeted mentoring. With mentoring, you can engage an experienced instructional designer mentor – your very own coach – who will work one-on-one with the individual to address specific learning gaps and help her develop self-reliance.
How Can ID Mentoring Work?
Let’s use an example to illustrate how you might use mentoring to develop a team member.
My client, a new IDer whom we’ll call Alex, had worked as an admin in his company’s learning and organizational development group. Alex was promoted to instructional designer and started out updating existing courses. Then he needed to create a course from the ground up. However, his manager, stretched thin, didn’t have the time needed to support his development and had no other ID professionals to help. So, she engaged me to help Alex get up to speed. Our goal was to help him design the course and to work more independently in the future.
At our initial meeting, Alex and I identified the primary goal of our mentoring engagement: to develop and launch a Good Documentation Practices course. Next, we discussed the business need for this course, the problems it should solve, before considering the foundational skills Alex would need. Since this course was being developed from the ground up, the list included:
- conducting needs assessments
- building successful partnerships with subject matter experts
- writing learning objectives
- determining the evaluation strategy
- identifying learning modalities
- incorporating techniques to engage learners and create effective learning.
We planned a series of meetings, about twice a month for six or more months.
In the follow-up meetings, we focused on one skill at a time. I would share some (but not too much!) of the theory associated with a skill, a method or approach for applying it, and the importance of using the method or approach. For example, when we covered writing learning objectives, we studied Robert Mager’s approach and Bloom’s taxonomy. We discussed the ways those methods could help Alex and where they might be challenging to use. Then, we created a couple of learning objectives for his course and analyzed how each one aligned with the two systems.
Then, the rubber met the road. I assigned Alex to write the rest of the learning objectives for his course.
Before our next meeting, I asked Alex to send me the learning objectives he created, along with any questions he had. I responded to the questions with guidance and encouraged him to continue working (most important!)
The next time we met, we reviewed Alex’s final learning objectives, made tweaks, and revisited Mager and Bloom. Then, we moved onto the next skill and wash, rinse, repeat. After every one or two meetings, I updated Alex’s manager with highlights of the sessions and a list of Alex’s next steps. When appropriate, I also suggested ways the manager could provide support.
Why Choose Targeted Mentoring over ID Classes?
- You can develop a timeline to fit the mentee’s schedule. If a mentee needs to be away from work, she won’t miss a class and you don’t need to worry about what she didn’t get to learn. Instead, you have the flexibility to create a schedule around the learner’s calendar or make last minute changes when necessary.
- Your involvement promotes accountability. As the learner’s manager, you should attend the initial meeting between her and her mentor, and then consider joining periodically, which sends a message that you take the program seriously. Also, if you request regular updates about the learner’s progress, you can monitor and support it. This also enables you share your input and shape the mentorship to fit your team’s goals.
- The individualized action learning approach focuses on the mentee’s specific needs. The process of studying a topic, immediately applying it to a real project, and then receiving targeted feedback and assistance allows a new IDer to fill specific competency gaps.
So, returning to our example, what is Alex up to, a few months after completing his mentorship? He has successfully launched the course he was working on, and it has received a positive reception from the business unit it supports. Meanwhile, he and his manager feel that his skillset and professional potential have grown. So, Alex’s manager has assigned him more courses to develop, which are launching according to plan. It’s a win-win!
Five Ways to Improve Your Networking
Summer is an important season to focus on building your network. People feel relaxed and want to get out and mingle, so it’s easy to make new contacts. This does not, however, mean that networking itself is easy. While I enjoy networking, it’s definitely a skill — like any other — I’ve honed over the years.
Below, I thought I’d share some of the networking approaches I have used most successfully. And, if you have others to share, please write me and I’ll be happy to add your approach to the list!
- Setting three specific goals before attending a networking event or meeting. When I come in with goals, I am better focused and get more out of the event. My goals depend on the event and could be as different as connecting with someone proficient in a particular skill, or meeting three new people with whom I will follow up later. This bring us to our next tip…
- Following up with contacts after the event. Yes, it’s time consuming to do so, but following up with a call or email helps build meaningful relationships. I find it beneficial to plan time in my calendar to follow up after every event, so this action item doesn’t get overlooked.
- Reaching out to ask for help. Asking someone else for assistance is a great way to make a connection! Too often, we don’t do this, because we don’t want to admit that we need help. However, people like to be asked to assist, and it’s a good way to establish common ground and bond with folks in our career fields. For example, I was interested in having something translated from Dutch. I put out a request on my town’s listserv and was well rewarded, with multiple connections from whom to choose!
- Remembering that anyone has the potential to be a worthwhile connection. When networking, people tend to limit themselves—to their field, career level, or both. The truth is that people from a wide variety of fields—or someone they know—could be a future employer, partner, or client. A few years ago I met someone who had an executive coaching practice. He eventually took a job within an organization, but we stayed in touch. When the stars aligned (well, more likely because we kept in contact), he reached out to me when a need arose.
- Staying in touch regularly, and not just when I need something. While people do like to help their colleagues, if I’m only in touch when I need something, others will see through that. I treat my colleagues like I would any other relationship—I offer support, ask for support, and am available when they want to run something by me. I might share an article with a contact, or simply say hi now and then. Networking can be a lot like growing plants—it takes time, patience, and water, not to mention the right weather, to see those green shoots sprout.
Here’s an example showing how these tips worked for me to land a significant client: I once volunteered as a coach in an annual undergrad event at Babson and set a goal of meeting 3 new people during the training session. I introduced myself to the other coaches seated around me, asking about their companies and the work they do. One of these coaches was a new training manager. After the event, I followed up with him a few times, offering to be a sounding board and listening to how his transition was going. About six months later, he had a need for EnVision’s services and engaged us in more projects over several years. We still stay in touch and help each other out with advice and connections.
I will continue to allocate regular time for networking activities. I never know whom I might meet or to whom I might be introduced. Maybe I’ll see you at an event soon? And feel free to reach out anytime you’d like to bounce ideas around – whether you have on your flip-flops or any other types of footwear!
Last week, I went to the doctor for my routine exam – something I do every year. While I don’t find the experience unpleasant, it can be kind of a pain. I need to take time from my busy schedule to attend the appointment and any follow-up care that results. Sometimes I wonder, why do I go? Is it really necessary? I feel “good enough,” right?
After my appointment, it occurred to me that my routine doctor’s exam resembles a course audit in more than a few ways. While I often feel “good enough” when I go to my regular appointment, I go to get a picture of my well-being. This knowledge will hopefully allow me to stay healthy or improve my well-being. Similarly, professionals who seek feedback on a course want to ensure the course’s “health.” They don’t want their course to be just “good enough.”
Of course, we monitor a course’s “health” differently than we do our own. In a doctor’s exam, the physician will assess the patient’s heart, lungs, and general appearance, as well as her blood pressure. Medical benchmarks or “objectives”— for blood pressure, cholesterol, glucose levels and more — give the physician and patient a guide of where she should be. If the patient is not within a healthy range for some of these barometers, the doctor counsels the patient how to improve her health, whether it be through diet, exercise, or medication.
Instructional designers pronounce a course “healthy” when it engages learners and effectively meets the course’s learning objectives and/or the company’s business goals. EnVision’s course audit offers both assessment and advice, and it’s a pretty thorough exam!
EnVision’s consultants will attend the class if it’s live, and review all class materials. We measure the class against our proprietary 80-point checklist. We’ll then determine what in the course worked well and can stay, what needs to be improved or should go, and how well the material and activities support the learning objectives. We prioritize the list of improvements, so the client knows which ones will have the highest impact. We include these points in a written report that we go over with the client in a follow-up conversation.
EnVision’s course audit provides a snapshot of a course’s “health.” Our clients then have the option of updating the course themselves, or engaging us to help.
Just like a medical exam provides a person with information to better her health, a course audit gives the organization tools to improve a course, whether it is a live class or elearning. This “health snapshot” points the instructional designer in the best direction to update the course, and ensures it meets the needs of both the learners and the organization.
I have a 10-year-old car. It’s a great car – a silver Honda CRV with over 125,000 miles that’s whisked our family on many vacations, transported the kids to and from school, and conveyed me to many a work meeting. I take good care of my car, having it serviced (at the dealer!) whenever it prompts me for service. Then one day, a low rumble from the car’s exhaust startled me.
My dealer told me it would be well over $1000 to replace the muffler and pipe. Again, I love my car. But I couldn’t see spending that amount on it. So, I went to Lou, an exhaust specialist. Lou looked around and identified the problem – a hole in the exhaust pipe. He could repair the pipe for $80. Lou advised me NOT to replace the rest of the still-excellent pipe, and told me the muffler was fine. It would take 20 minutes to fix.
Wow. I probably would have been willing to pay $500, and would have been happy to have saved at least that much. Now I’m sharing the name of the exhaust specialist with my friends and family. What did Lou do that was so great (besides the obvious huge dollar savings)?
I could pretty clearly see a process Lou followed, one I’ve been spending some time with recently. It’s called the Trust Process, from The Trusted Advisor by David Maister, Charles Green, and Robert Galford. Here are the key steps in the trust process, and how I saw them played out:
- Engage – Lou was highly recommended and rated, so I felt he would be a good person to have take a look at my car. Once I met Lou, he helped me feel confident by letting me know he would figure out what the problem was.
- Listen – After listening to my car complaint, Lou put my car on the lift and took a look while I checked my email in the waiting area.
- Frame the Issue – Then he brought me under my car (cool!) and showed my exactly what was going on. I saw the hole, understood how he could repair it, and agreed to the $80 fix. He also told me what I didn’t need to do (replace the whole pipe or the muffler).
- Envision an Outcome – Lou said yes, that this fix would eliminate the noise, and that there would be no adverse impact.
- Commit – I agreed to the work. And, 20 minutes later it was done. My car sounds less “exhaust”ed now.
While this scenario is a bit simplistic, we can probably all apply it to many work situations we encounter. Here is one example from EnVision’s work:
- Engage – In a regular conversation with Barry, a client, about an ongoing project, I realized he might have a need for mentoring, a service we offer. Since we have a strong relationship, I was able to probe and ask more about the situation. This conversation got him engaged and interested in discussing this further.
- Listen – I listened to Barry’s situation, concerns, and needs, asking additional questions to gather more information. This helped him feel he was being heard, and allowed him to clarify his situation.
- Frame the Issue – I recapped what I was hearing about the cause of the challenge Barry shared and explained our mentoring process, asking if that seemed like a useful and appropriate solution, and what concerns Barry had about it. I shared a couple of cases where mentoring has helped other clients in similar situations. This helped Barry to know that I really could help him.
- Envision an Outcome – I continued with the cases, explaining how they turned out and followed with what the outcome could be for Barry’s situation. Now, Barry could see a possible outcome that would be beneficial to him. I suggested we try out mentoring on a small scale, as part of the work we were already doing.
- Commit – Barry agreed to have us provide the limited mentoring services to get his team through immediate deliverables.
Building trust, and a reputation as a trusted advisor, is a key to long-term, productive relationships. What are some ways you already use these steps to build trust with your key stakeholders and partners? What is one step in the trust process you plan to work on enhancing this month?
As instructional designers, we often confront ambiguity when we take on a new project, especially when working with a stakeholder who has already created, the process, tool, or content. It can be overwhelming to achieve clarity – to get our arms around the purpose of the proposed change, impact on performers and the organization, which tools are in place to support the change, and sometimes even to understand the processes and content well enough to work our “magic.”
Of course, ambiguity can be experienced in many areas, not just instructional design. For example, there’s often confusion in learning a new skill or job, assuming management of a group or project, or even starting a new hobby, like the genealogy research I just got involved with.
So, you want to know how to get from ambiguity to clarity in three easy steps? First…
…Well, really did you think it was that easy? This is a skill that, like so many others, takes a lot of practice, some solid fortitude, and trust in the process. I can’t deliver three easy steps to you, but I can offer three considerations to help you through. Let’s start with….
1. Trust the Process
It can be hard to “let go” and allow the process to work for us. But work for us it will. If left to its own devices, the brain will devise strategies to clarify its thoughts, and one of these main strategies is creating categories.
A team of researchers led by the Ruhm University Bochum found that people create and place information in different categories to make sense of the world. “Thinking in categories or pigeonholing helps our brain in bringing order into a constantly changing world and it reduces the information load,” wrote the researchers.
So, allow yourself time for clarity to emerge– sleep on it, meditate, do some yoga, or go for a run or walk – and your brain will take over. You might experience a number of “aha” moments – feel the satisfaction as these occur!
2. Ask questions
Particularly in situations where we are not the expert, we can’t expect that we’ll just know information or what’s in the head of our stakeholder. After doing our preliminary research, the only way to get solid information is by asking for it. Common questions for use in many situations (training or otherwise) are:
- What is the goal of this effort? What do you hope will change?
- Why is that important to you? To the organization?
- Who will be impacted by this? How? What happens if they don’t get on board with the changes?
- Is there a glossary or list of abbreviations, acronyms, and definitions (…so I can get up to speed on the subject quickly)?
- What is the current/proposed process or workflow?
- What do you mean by [fill in the blank]?
That last question is intended to prevent you from making assumptions. The stakeholder might speak as though you should know everything she knows, but that’s unrealistic and sometimes you need to just ask. Sometimes I find this question actually causes the stakeholder to rethink her own assumptions.
3. Draw it out
Create your own picture of how you understand the situation. This could be a flowchart, diagram, or other image; high-level project plan; or a draft course outline. In doing this, you’ll identify areas for which you need more information, areas you thought were clear to you but are still ambiguous, and things you believe are accurate.
Then review this picture or plan with your stakeholder, who can fill in the blanks, clarify ambiguities, and confirm or correct what’s left. She might even identify points you totally missed, or facts that she took for granted and didn’t think to mention to you.
So, while those three techniques are not necessarily “easy,” I hope they will help. The important thing is to realize that many new projects will have ambiguities. If everything were so clear and obvious, life would be so much easier. And probably a lot less interesting.
Would an experienced runner set off from a marathon starting line without warming up? No! A world class athlete takes time to prepare for a race, and often goes through the paces of a regime created by her running coach. She may run a short distance, then hold 30-second stretches. Her coach knows these energizing moves will get her muscles, and mind, prepared to run 26.2 miles.
Similarly, a trainer or facilitator (the “running coach”) starts a training session with an energizer or icebreaker, an opening activity designed to limber up the learners’ “mental muscles” and help them prepare to learn.
What are some effective ways to motivate learners? EnVision team members share their favorite icebreakers below.
For Ginny Maglio, who delivers leadership curricula to EnVision’s clients, it’s all about sparking learner interest. One of Maglio’s favorite icebreakers involves none other than – LEGO® blocks. “Participants build something related to the course content,” Maglio says. “For example, in a program on how to build successful teams, participants are asked to build something that depicts their organization’s mission….short, simple engaging activities can be used very successfully to initiate learner interest.” When learners feel energetic, they’re also engaged in the course.
Irene Stern Frielich, EnVision’s president, “breaks the ice” for learners in a different way. During learner introductions, she asks participants to share a fun thing they did over the weekend. This recollection, and the positive social vibes from sharing it, get learners in the frame of mind to think and process.
Scientific research supports this tactic. In the article How Emotions Influence Learning and Memory Processes in the Brain, Dr. Shlomo Wagner found that “Different emotions cause the brain to work differently, in terms of cognitive processes such as learning and memory.” In his study, Dr. Wagner discovered that pleasant social encounters between rats caused the rodents to build memories, whereas negative emotions did not. It makes sense that when people, too, recall happy times, it puts them in a good mindset for learning.
Paula Spizziri, an instructional designer with EnVision, ensures she starts a module with an icebreaker that takes a page from the playbook of the running coach. Spizziri asks the learners to engage in a brief spurt of physical activity, since exercise stimulates the brain. Or, she may prompt the learners in an exercise in which they take turns completing the sentence “I’m late because…” with a movie plot. The other learners then try to guess each film, and both actions spark the learners’ thinking.
It’s clear that icebreakers that target both learners’ emotions and energy work. According to an article by Michael Higley on elearningindustry.com, “Appropriately framing a lesson with an icebreaker activity is a useful technique in establishing context in which new learning will take place,” Higley writes. “The initial experiences students have with any course establish the tone for future tasks.” So, a course’s eventual success can be traced, in part, to an icebreaker’s efficacy — just like stretching sets the stage for a great race.
Bea Smart, an instructional designer, started working recently with Joe King, a compliance director, about potential training needs. She’s excited about the opportunity to partner with the Compliance Department and wants to impress her new internal client.
Bea starts off with a training needs analysis by completing several employee interviews, an activity that had never been conducted in the Compliance Department.
The interviews strongly suggest that changes other than training are needed to meet business goals, including clearly written standard operating procedures. However, Bea’s focus is instructional design, and she’s not sure Joe would agree with the recommendations that arise from the interviews, so she doesn’t share the interview data or her conclusions with him.
After agreeing on learning objectives and an elearning modality for a course Joe would like developed, Bea agrees to deliver the draft 1 storyboard to Joe in five days. Though she needs input from several stakeholders and believes a complete storyboard will probably take closer to ten days to create, she wants to get this project done ahead of schedule, establish clout with the Compliance Department, and hopefully get more work from Joe’s team later.
After two weeks go by, Bea has not received the feedback she needs to finish the draft 1 storyboard. Joe, under pressure from his manager, is ready to send out the PowerPoint slides he created for a recent presentation on the topic, and use that as “training.” Bea asks Joe to hold off and immediately gets in touch with her colleagues, works a few late nights, and manages to finish the storyboard after three more days.
The following day, Bea meets with Joe to discuss his feedback on the storyboard. Joe has made many content changes and Bea believes they no longer address the approved learning objectives. His changes also impact effective learning techniques and result in more of a “page turner” course – a course that is a reading exercise with no learner engagement. Bea remembers the friction over the missed storyboard deadline, though, and decides to keep quiet, figuring Joe knows best. And even if he doesn’t, she doesn’t want to create more friction.
The training module finally pilots, then launches. Three months later, analysis finds that 70% of learners don’t complete the course. In addition, there is no improvement in business performance and the same errors continue to be made. Where did Bea go wrong in working with her client, and what skills did she miss?
- Being Transparent. When Bea performed her needs assessment, she uncovered information via interviews and drew conclusions she didn’t share with Joe. She was concerned he’d disagree, potentially imperiling their relationship.But by sharing this information, and offering her expert judgment—which is, after all, likely what Joe expects—she could have helped Joe realize that SOPs needed updating in order to support successful training outcomes.
- Setting Expectations. Bea wanted to impress Joe by offering a fast, even unrealistic, turnaround time. It would have been wiser, though, to give the standard ten days a storyboard like this usually takes Bea—and then delight the client if it is completed sooner.In addition, Bea was expecting information from the stakeholders, but had she (or Joe) communicated requirements to them? And if so and they were late, she should check in with them and alert Joe immediately. Joe can then decide on the best way to proceed.
- Communicating Fully and Clearly. Bea doesn’t push back or inquire about Joe’s significant changes. While he may have very good reasons for changing his mind (perhaps a recent internal audit turned up some significant new issues), making unstated assumptions is never a good option! Asking questions to understand Joe’s reasoning—and then offering her own professional judgment—would help clarify assumptions and result in an appropriate learning solution.
Partnering with clients—internal or external—can be a delicate balancing act. It is only natural to try to keep the client happy to further your professional relationship. Yet, if you’re withholding information, offering unrealistic expectations, or communicating incompletely, the relationship will flounder. Be transparent, set expectations, and communicate fully and clearly, to help your client relationships—whether internal or external—flourish.
Looking for a fun night out? You could head off to your favorite neighborhood dive – or, er, brew pub – and grab a burger and a couple of beers. You’d see the same regulars and the familiar, weathered interior of the pub. And you’d enjoy the comforting routine…though in the back of your mind, you may be a bit bored.
But if you want to remember the night? Try a pub crawl. You’d meet up with a group of friends. Together, you’d go from bar to bar – the Irish pub, the “townie” spot, the hipster place you’ve been hearing about – and spend a bit of time in each, trying different craft beers and meeting new “regulars.” You’d spend less time in each bar than you would have at your neighborhood haunt, but you’d have a full night. At the end, you’d likely reminisce about the experience and what you found unique about each pub.
People often choose a pub crawl for a special occasion – it’s more fun than camping out in one bar. Why?
You get a new bar visit – or “snippet” experience – each time you change venues. At one bar, you’ll kick back to some acoustic live music; at another, sample a small batch house brew; and at a third, stand crammed shoulder-to-shoulder with your fellow bar patrons, making sure your feet don’t stick to the beer-covered floor.
Each of these pub stops offers something different that will stand out in your memory, much more than two hours at your favorite dive, as comforting as that may be. By choosing and changing up your environment, each snippet experience becomes more special — and memorable.
How does this relate to learning? If you follow the formula of learning followed by a quiz and repeat until you’re done, your learners’ eyes will start to glaze over – and they won’t even be tipsy. Create a well-planned blend of short, tightly-focused elearning lessons, videos, games, and quizzes supported by in-person classroom modules and targeted action learning assignments to practice skills on the job? That’s a “pub crawl” your learners can get on board with.
A learning experience comprised of short snippets is also known as microlearning. Microlearning succeeds, in part, because learners are able to seek out specific, focused content when they need it. “Microlearning…enables learners to access information in a variety of formats that fits their learning style. When employees seek out content, it helps them better retain what they have learned and stokes their appetite to learn more.”(Fox, Amy. (2016). Microlearning for Effective Performance. TD Magazine)
Whether you’re creating learning solutions or pub crawling, don’t schedule too many snippets or “stops” – it could prove tricky for the participants to remember the experience. Choose a pub that offers brews your friends like — maybe a wide variety on tap, for example (or a game for a learning activity). And you may want to plan out which local watering holes to avoid – maybe your best friend doesn’t get along with the regulars, or your learners don’t respond well to games. Plan thoughtfully!
So, the lesson is, whether pub crawling or creating a learning solution, mix it up with appropriate, small bursts of new material and offer choices to the learners. You’ll make effective use of their time — and create a memorable experience.
Are you – or is someone you work closely with – a SME (subject matter expert)? Instructional designers often require a SME’s knowledge and input to develop training. And it can be a challenge to obtain the key content (and only the key content).
First, let’s introduce a typical SME, Simone. Simone is the director of compliance for the organization and comes to this role with 13 years’ experience. She is well-regarded in the organization and is often the go-to person when difficult compliance questions arise. Over the years, the compliance team has put into place a series of standard operating procedures, tools, and best practices that help the organization stay in compliance so they can continue selling their products. Simone was instrumental in creating these documents and approaches, and really loves her work. She loves it so much she tries to share her knowledge with any willing (or not so willing!) listener.
Are you working with a SME, and feeling the pain, wondering how such a fabulous compliance director can also be such a challenge to work with? Let’s explore the symptoms you might experience, and provide some prevention strategies.
Bleeding content: Too much material
Simone’s slides – for management presentations and for training – are filled with text and there are few images or diagrams. Her slides include a lot of details that are important to Simone. Much of this content, however, isn’t applicable to the target audience. When she meets with the instructional designer or presents to a group, she reads off the slides and adds even more details – whether or not they apply. After all, she loves her content! Everyone should want to know all she knows.
Unclear diagnosis: Missing content and next steps
Often there are disconnects. It’s not clear how Simone got from point A to point B. And sometimes it’s not clear to the listeners what they need to be able to do with the information. Are they responsible for completing the tasks? For ensuring the tasks have been completed? In what situations? How often? What is the impact of not doing it or not doing it correctly? Simone means very well, but she takes the content and connections for granted. She already knows how it all ties together and she’s been working with it for years so she assumes everyone else understands the connections, too.
Constant emergencies and “setbacks”: Lacks time to support the instructional designer
Simone is, of course, a very busy person. She didn’t plan to spend hours answering questions the instructional designer is posing to her. She doesn’t have time to review a design document, a storyboard, and an elearning course or instructor notes, slides, and handouts for a classroom course. Sometimes she misses meetings, comes poorly prepared, or even misses deadlines you agreed upon.
There are many prevention options, usually a combination of which are most effective. Here are three of my favorites.
Control the content: Stay focused on the goal and how the learner needs to perform
Typically, I start with a needs analysis looking at the business goals and performance required to achieve those goals. The learning objectives, then, are focused on supporting the required performance and I obtain stakeholder agreement on these objectives.
Having laid the groundwork, I can keep these goals and objectives front and center. When Simone asks me to add more content or explains content she already provided me, I’ll ask questions like:
- Is this critical information needed for the learner to be able to [insert learning objective here]?
- Or is the information “nice to have”? If so, I suggest we eliminate it so the learner isn’t overloaded with non-critical information.
- Or is the information really helpful but not critical? Then, I might suggest we include it as an optional “tip” or in a resource list.
Another set of follow-up questions I like is: If you were the target learner, what would you need to be able to do with this information? Why is that so important? What if I don’t know/do that?
I might end up sounding like a broken record, but I’ll keep asking these same questions. Eventually Simone will catch on and start self-editing.
Create a logical, connected sequence: Simplify content and make it accessible
When I am handed an existing slideshow the SME would like to use, and I see it is filled with lots of words, one approach I use is to diagram what the words mean. For example, if I am developing training about a process, I show it in a simplified flowchart and break it down (using more flowcharting) throughout the course. This both helps anchor learners in where they are in the process and helps them see interrelationships.
I also like to use scenarios, with visual representations of the characters and what they are doing. Telling a story via the scenario helps learners feel immersed in the learning. You can do this as an alternative to blocks of classroom lecture or pages of text in an elearning course.
I always check in with Simone to confirm the diagram or scenarios convey the key points accurately. After doing this for a bit, eventually Simone will hopefully appreciate how I’m bringing her content to life and the value I provide as an instructional designer.
Set expectations early and often
Have you ever brought your car in for service and they tell you it will take 3 hours and cost no more than $300? Then, six hours later – no car yet and they decided it will be $1500? Of course a good repair shop will set expectations with you up front and update you so there are no last-minute surprises.
Imagine Simone’s situation. She needs to know how much of her time will be needed and when so she can plan appropriately. If you underestimate or don’t offer any indication, she can’t easily ensure she’ll have the time needed. So, I like to let the SME know early on how much time is needed and when the “heavy” weeks will be. Then, as the project morphs and the timelines change, I keep her updated and check in on her ability to meet the new deadlines before setting them in stone.
Another helpful tip is to plan meetings in advance. Having a weekly or biweekly meeting on the calendar for check-ins and working time is a way to guarantee a minimum of time together. Simone will appreciate having an agenda in advance and an action item list after the meeting. If you don’t need a meeting, you can always cancel. I never hear complaints about a canceled meeting. And remain flexible so the SME stays engaged.
Finally, acknowledge your SME is very busy and may have true emergencies to tend to. When I know that might be an ongoing concern, I ask Simone to provide an alternate SME and ensure that Simone will have opportunities to review materials. This can protect her time and help keep the project moving along.
So, what’s the prognosis for SME syndrome? If you implement your prevention plan from the start – staying organized and communicating openly with the SME – you greatly increase your ability to develop a training program that meets the needs of the learner and the organization.
In our last blog post we saw how immersive, sensory events can help us remember experiences, using the example of my trip to the Blue Lagoon spa. We don’t need to visit a spa to help learners remember, though…we can create immersive experiences to enhance our instructional design. Here are two examples of physically immersive activities from our library.
The Human Flowchart
Training people in a specific process often involves use of a flowchart, but review of this tool in a classroom can be a dull exercise. We can bring the training to life—and make it more effective—with a “human flowchart.”
We used immersion in this way when we designed a class for a medical device company on FDA complaint handling regulations. The client developed a process to meet the FDA requirements and created a flowchart with a series of 20 steps to be executed by seven distinct roles.
EnVision immersed the learners in the training using movement. We broke up the learners into seven groups, each representing a different role and gave each group signs, listing their part in the complaint handling process.
Getting up from their desks, the learners then needed to move into a flowchart according to their part in the complaint process. Once everyone was in the right spot, the instructor started in the role of a customer calling in a complaint. Each group then shared what action they planned to take and what information would move on to the next group. The movement and interaction with others kept the learners involved in the entire process of complaint handling. Back at work, they should be more likely to recall the details of their role’s steps based on the physical locations of each learner.
For another client, this one a global pharma organization, we designed a course for new managers. One of the learning objectives was for learners “to be able to describe how managers’ actions impact the engagement of their team members.” To achieve this, we created an activity in which “managers” set and communicated performance objectives around building widgets, in this case paper origami cranes, with “employees” who were to make them. During different rounds, managers could give no feedback or some feedback and coaching. Debrief followed with a focus on the value of feedback and coaching to engage employees.
Picture a classroom filled with learners eagerly (or not so eagerly without coaching!) folding paper cranes to achieve their goals. The feel of the crinkled paper, the sound of their managers coaching them (or the dead air when they didn’t!)…these involved the senses and helped learners become fully immersed in the activity. The experience became more than “knowledge,” but an experience to remember and be able to describe concretely.
Creating Your Own Activities to Engage the Senses
Some of the best learning activities allow the learners to immerse in the learning activity, much as tourists immerse in the spa baths at the Blue Lagoon.
So, for your next course, try finding a way to immerse learners even more deeply in the learning by involving multiple senses. It can be a lot of fun to create activities like these, and it requires imagination, creativity, and lots of planning. Want your own activity to engage the senses? Schedule a call for a free 1-hour consultation on how to make that happen.
This summer, my husband and I traveled to Iceland. Among other wonders of nature, we visited the Blue Lagoon, a spa formed 40 years ago during the creation of a geothermal power plant. Today, people come from all over to bathe in the warm, soothing water.
There are a surprising number of things to do at the Blue Lagoon. You can laze underneath a man-made waterfall, order a drink from the bar, or even enjoy a massage in the water. We wandered across the bridges over the pool and explored the area. We even used the silica clay as a mud mask to soften our skin and further the sense of relaxation.
Even with all the amenities, I was spellbound first and foremost by the setting. The water, turned a beautiful blue-green by naturally occurring algae, had a temperature of 100 degrees Fahrenheit – about 45 degrees warmer than the air. This contrast between water and air made the warm pools feel even more comfortable and welcoming.
With nearly 20 hours of sunlight in July, it didn’t feel strange to be soaking outside at nearly any time of day. In the daylight, I was able to appreciate the setting clearly. Lava fields topped with moss surround the pool. When I closed my eyes, I heard the gentle cascade of the waterfall. With everything I felt, saw and heard, the Blue Lagoon was an immersion for the senses.
Research shows that these strong sensory impressions could help me remember my visit: a specific sight or sound can bring back a particular memory, and there’s a very good reason for it. “The same part of our brain that’s in charge of processing our senses is also responsible, at least in part, for storing emotional memories,” writes Rachel Reittner on livescience.com.
This storage receptacle is known as the sensory cortex. Benedetto Sachetti, a researcher in Italy, conducted an experiment with rats in which he trained the rodents to associate a specific sound with an electric shock. The rats would freeze upon hearing the sound, but after the scientists created lesions on the rats’ sensory auditory cortexes, they froze less frequently. This suggested the rats correlated the sound with the electric shock, and could no longer do so after their lesions (Sacco & Sachetti (2010). Role of Secondary Sensory Cortices in Emotional Memory Storage. Science, 329 (5992), pp. 649-656.)
The practice of mindfulness can include focusing one’s attention on the senses, and there is evidentiary support that it, too, helps memory. In another experiment led by Michael Mrazek, a graduate student at UC Santa Barbara, undergraduates were directed to take a class in either mindfulness or nutrition for two weeks. “The students in the mindfulness group were taught how to pay attention to their sensory processes, like tasting food and breathing,” writes Alice G. Walton in an article describing the experiment.
The students’ recall was tested both by working memory tests and a verbal GRE (graduate admissions) exercise. After one week of instruction, the mindfulness group improved more than the nutrition group in both assessments.
The takeaway? The senses have a big role in memory retention. If possible, try to incorporate a strong sensory component into your instructional design. Oh, and the trip to the Blue Lagoon was amazing….just ask me in 10 years. I’ll remember.
For case studies see our post on Using Multiple Senses to Improve Learning.
We’re all enjoying the Rio Olympics right now, and witnessing incredible athletic feats. Some sports achievements are very easy to measure. Take track and field, for example. In a race, the runner who crosses the finish line first wins. In soccer or water polo, the team that scores the most points wins the game.
In other Olympic sports, however, the winner may not be so nearly clear-cut. In gymnastics, an athlete prepares a program to the best of her (or his) ability. The judges score her on many elements, and the highest score among the athletes determines the winner. The gymnast will not know immediately following her program whether or not she won, as her competitors follow her.
Similarly, instructional designers don’t face an immediate, clear-cut path to knowing if their training “stuck” — and if the learners are able to apply what they’ve learned to their jobs. However, with evaluations, instructional designers can learn a great deal about their training programs’ success.
The Kirkpatrick evaluation levels assess learner reaction (level 1), learning (level 2), application of skills to the job (level 3), and business results (level 4). Challenges may arise with executing each evaluation level. Let’s look at a few of the challenges in levels 1-3 and see how instructional designers might mitigate them.
Measuring learner reaction, level 1, can be done by a course evaluation form, show of learners’ hands, verbal check-ins, or via a “dashboard” that measures the pace of the course and learner energy levels. The trainer notes learner reactions during the course and can use these to amend the course both “on the fly” and after. Clients may, however, read too much into level 1 results and want to change the learning solution without further assessment. It is important to explain to clients exactly what level 1 assessments measure — solely learner reaction — and to use other evaluation tools as well.
Level 2 evaluations, or measurements of learning, frequently take the form of final written tests or simulations. However, writing assessment questions can often be a challenge in itself. If the questions are too difficult for learners to understand, this could impact the validity of the test. Similarly, it can be hard to write appropriate “distractor” questions that are not obviously incorrect. After all, some people pass a learning assessment with flying colors — without ever taking the class or having done the job for which they are training.
The other issue for instructional designers is that there is not necessarily a direct correlation between level 2 “test” success and on-the-job achievement. Extensive knowledge may not lead to professional efficacy. To test for skills, ask assessment questions that explore the best way to execute certain job tasks or make on-the-job decisions, rather than questions seeking rote knowledge.
Level 3 evaluations measure application of skills to the job, often via learner or manager surveys, interviews with learners or managers, or on-the-job observation. An important consideration when measuring at level 3 is to ensure appropriate resources and tools — coaching, performance support, etc. — are in place to reinforce learning and support behavioral change.
The “success case method” created by Robert Brinkerhoff also may reveal successful on-the-job behavioral changes. With this method, stakeholders interview a sample of learners — typically high performers — to determine which factors have made them successful — and then corroborate this success with independent evidence. Because the strong learners share which influences contributed to their success, their interview responses can yield clues to help the less successful learners.
The success/case method can be very effective, giving stakeholders qualitative feedback beyond data, says Paula Spizziri, an EnVision consultant. “Sometimes you don’t know what’s behind the curtain…this gives you that,” she said. However, an organization using the success/case method needs to allocate time and resources for the interviews and independent data collection.
While all evaluation methods have some challenges, many of these hurdles can be mitigated. Because learning success, like that of a gymnastics program, may not be immediate, the evaluation process strengthens your training program.
I happen to be a big music lover, and enjoy both listening and performing as a flutist in the Sharon Concert Band. At last week’s band rehearsal our conductor, Steve, was doing his usual great job of keeping us on tempo, signaling when each section needs to come in, and gesturing to show we should play more piano (Steve often needs to remind us to play more softly). As I was playing, it occurred to me that a symphonic performance resembles the process of training.
In both, people assume distinct roles to accomplish the goal — whether it be performing the piece or executing the training program. And both require a specific process, anchored by practice, to reach those goals.
A symphony (or other musical piece) begins in the mind of a composer, who creates the music to be played. After learning the piece of music, the conductor rehearses the musicians in preparation of performing the symphony.
An instructional designer, similar to a composer, authors the learning solution. A trainer (“conductor”) then takes over, using the leader’s guide (music “score”) and teaches the learners (“musicians”) the material they need to know. A trainer may use visual aids or workbooks to guide the learner, just like a conductor uses a baton to lead musicians and keep the beat.
A symphony may feature a few soloists throughout the piece. A soloist shines in an orchestral performance, much like an active participant stands out in the classroom. A section leader in each instrument group organizes the section, much like a facilitator may help “conduct” a breakout group in classroom instruction.
While musicians and learners “rehearse” all together, each “musician” must put in individual practice time to learn the material. During a learning session, the trainer may break the learners into groups, where each group can work on a directed activity and then share the results with the entire classroom. Likewise, a conductor rehearses one group of musicians at a time, such as the flutes, to work on a specific part. Afterward, the entire orchestra can play together.
A successful symphonic performance depends both on the musicians rehearsing together (as in classroom instruction) and each player practicing (or learning) on his or her own. Facilitation by an experienced conductor, or trainer, brings out the best in the musicians or learners. Each person has his or her “part” to play, and each role must be “performed” to the best of the performer’s abilities to achieve a winning final product.
Have you ever gone on a treasure hunt, or maybe seen one in the movies? The seeker searches for a treasure chest—usually attending to challenges along the way. Imagine large rolling rocks and a few poison darts, all difficult to control! Eventually the seeker locates the elusive treasure chest, but not all its contents are gleaming. There is some culling to do, and some polishing, before being ready to present the gems to the funders of the treasure hunt.
Back in the real world, learning & development professionals are given a mission to add value to the business, often through a request for “training.” On the L&D professionals’ “treasure search” they often attend to challenges along the way. Imagine the multiple competing priorities, regulatory controls, and insufficient resources that can impact successful learning.
Once the treasure is “found” (the learners have learned something!), what culling and polishing needs to be done so you can clearly see your treasure, before you present your findings to the funders and other stakeholders in your organization? Sometimes it can be challenging to find those precious gems and polish them so we can see how they glitter!
Earlier this month, I presented a workshop at the regional ATD conference. The topic, Polishing Your Gems, focused on creating an evaluation plan for a learning solution. Joined by client Matt Matosic from the Boston Public Health Commission (BPHC), we worked through a case study summarizing BPHC’s course on Hospital-Based Patient Decontamination and discussed options for evaluating its success. Participants then had an opportunity to begin developing their own evaluation plans, using an Evaluation and Measurement Planning Checklist we provided.
But where to start? Here are the core questions we suggested in our workshop to help folks determine which treasures to seek, as they begin their own evaluation planning.
Business issues questions
- Why is this training (or other intervention) being requested?
- What is driving the request or identification of this issue?
- How will solving this problem support the organization’s “business” goals?
- What do you need to be able to show as an outcome of your efforts?
- Who will see the results? (Consider: L&D management, business line management, senior management, learners themselves, instructional designer, instructor, funders, and other stakeholders)
- What will each stakeholder do with the information?
You might recognize that many of these questions get at Kirkpatrick’s level 4 (business impact).
Job performance questions
- What changes in job performance are needed to support meeting the business goals?
- What changes in job performance can reasonably be addressed via a learning solution?
- What workplace supports (such as managers, mentors, or peers; job aids or other performance support tools; periodic updates or web meetings) are realistically available to support the change in job performance?
These questions relate to Kirkpatrick’s level 3 (behavior change on the job).
Answers to these questions will help you define the learning objectives, set up workplace support systems to ensure learning continues and is reinforced on the job, and develop evaluation tools at all the Kirkpatrick levels.
For more information on our Evaluation and Measurement Planning Checklist, contact email@example.com.