What is the most significant advantage of conducting learning tests with trainees?

• Resources
• Articles / Ensure Your Training is Effective

Tips for effective training…

The training session is over and trainees have returned to their jobs to begin applying what they’ve learned, so the trainer’s job is done, right? Wrong.

A successful training program is always a work in progress, and the training cycle isn’t complete without an evaluation of training’s effectiveness, which leads to decision-making and planning for future training. Therefore, a useful and informative evaluation program needs to be a part of your overall training operation.

Here are several methodologies for evaluation as well as practical ways to retrieve good results.

To begin with, what should you be looking for in your evaluations?

• Was training delivered as planned, on time and to the appointed audience?
• Which training methods worked with which topics and which audience groups?
• Which methods did not with which topics or audiences?
• What specific problems occurred?
• How effective was the trainer at engaging the audience and conveying information?
• How did the training affect employee performance?
• Did the training satisfy regulatory and legal requirements?
• Were all stated goals reached? If not, why not?

This is a lot of information to gather; fortunately there are many methods and tools available to help you.

Kirkpatrick’s Four-Level Model of Evaluation

Donald Kirkpatrick, author of many books, such as Evaluating Training Programs: The Four Levels and Another Look at Evaluating Training Programs, created a four-level evaluation system more than 40 years ago, which has stood the test of time and continues to be utilized in many training programs today. Here are Kirkpatrick’s four levels of evaluation.

Level I: Reaction

Kirkpatrick’s first level evaluates how well participants liked the training session. If you’re interested in running the best training program possible, you want participants to be motivated for and engaged with training. This exercise gives trainees the opportunity to give feedback to the trainer on the pros and cons of the session, which is valuable information that shows trainers specific areas to
improve.

Follow these tips when obtaining feedback on trainees’ reactions:

• Observe trainees during the session for your own perception of their reception.
• Get trainee feedback in writing immediately following the session.
• Use measurable and meaningful terms.
• Use uniform feedback forms so results can be quantified and tabulated for the whole group.

Reaction Surveys

Ideally, trainers will include reaction surveys as the final section of training sessions and will insist on 100 percent participation. Make surveys easy to fill out

by following these guidelines:

• Use a numbered rating system, i.e. 1 to 5, with 1 being the low end of satisfaction and 5 being the high end.
• Use close-ended questions, such as those requiring the numbered assessment mentioned above to assess trainees’ impressions of the overall success of the session.
• Use open-ended yet directed questions that require more than a Yes or No answer along with space for trainees to write their comments in order to get detailed feedback on specific things that worked or didn’t work.
• Include space for undirected questions or comments on topics that trainees want to address.

Level II: Learning

This level measures how much of the desired principles, techniques, and skills trainees learned in the training session. In order to determine what trainees learn during a session, you need to know what they knew before training.

Follow these suggestions for measuring learning:

• Use pre- and post-knowledge and/or skills testing.
• Use objective measurements to assess what trainees now know or can do that they didn’t know or couldn’t do before training.
• Use a control group of employees who did not attend the training session to compare their performance to employees who received training.

Level III: Behavior

The third level measures employee behavior changes based on training. Your goal is to see how well trainees incorporate learned principles, skills, and knowledge into their jobs on a permanent
basis—or at least until they learn a new and better way to perform.

Follow these methods for evaluating behavior:

• Solicit the help of trainers, supervisors, and others who work closely with trainees to observe these employees before and after training, and to give their measurable, objective feedback on performance.
• Continue observations for 3 to 4 months after the training session, so you can get an accurate assessment of whether trainees have made permanent performance improvements based on training.

Level IV: Results

Kirkpatrick’s first three levels focus on trainees and the effect of training on their performance. The last level in Kirkpatrick’s evaluation model focuses on the results of training on the company in
terms of:

• Reduction of costs
• Reduction of turnover and absenteeism
• Reduction of grievances
• Increase in quality
• Increase in quantity or production
• Improved morale

Follow these suggestions for measuring the results of training on the company’s overall performance:

• Measure statistics in each of the categories listed above (or whichever categories you included in your goal-setting) before and after training.
• Use a control group, if possible, for comparison.
• Measure more than once over several months to allow time for changes from training to affect the areas you listed.

Evaluation by Return on Investment (ROI) Analysis (or the Fifth Level of Evaluation)

Some training professionals consider ROI analysis to be one method for determining the results of Kirkpatrick’s fourth level of evaluation. Others consider ROI its own level and make it the 5th level of evaluation. In any case, this method is an effective way to measure the success of your training program.

ROI analysis gives the trainer data about the financial impact training programs have on the organization. It differs from Level IV evaluation in the sense that Level IV takes into consideration nonfinancial data such as employee satisfaction. ROI analysis deals strictly with the financial impact of training. It answers the question “For every dollar invested in training, how many dollars does the employer get back?”

Here are three great reasons to use ROI analysis:

1. It’s a concrete way to validate your training program as a business tool.
2. It can be used to justify the cost of your training program to upper management.
3. It can be a useful tool for choosing future training methods.

Many business executives view training as a business expense and, therefore, measure its worth in terms of profits made or savings earned from this expense. You need to make sure training is seen as beneficial to your company. Use the following formula when measuring the ROI for your company to get the percentage of profit earned for every training dollar spent.

ROI Formula:

ROI (%) =         Monetary Benefits   –   Training Costs x 100
                                                      Costs

To get the figures for this formula, keep track of training costs, including:

• Design and development
• Promotion
• Administration
• Delivery (staff or technology)
• Materials
• Facilities
• Employee wages
• Evaluation

After training, keep track of monetary benefits, including:

• Labor savings
• Productivity increases
• Income generation
• New leads
• New products
• Lower turnover costs

Here’s an example for Ace Widget Company. Assume that a widget-making skills training session increased production by 20% and that a 1% increase in production yielded $40,000 in annual
revenue.

$800,000 total benefits $200,000 in total cost

[(800,000-200,000)/200,000] x 100=300%

So in this example for every $1 spent on training, the organization gained a net benefit of $3.

ROI Analysis is fast becoming an essential level of evaluation for companies who invest in
training.

Standardized Tests

Tests can be used for collecting information at all five levels of evaluation. They can be given prior
to training, during training, and after training. There are several types of tests from which to choose.

True-False

When designing true-false tests, keep these factors in mind:

• Write statements that are clearly true or clearly false.
• Take statements used in training and rephrase them slightly.
• Avoid words that tip trainees to the answer, such as “may” or “generally” for true statements, or “always” or “never” for false statements.
• Avoid double negatives, which make statements unnecessarily confusing.
• Remember that you’re not trying to stump trainees, you’re trying to make sure they learned the material.

Multiple Choice

These tests may be the most popular with trainees, but they can be the hardest to write for trainers. Follow these suggestions when designing multiple choice tests:

• Cover one topic per question.
• Include only one right answer. Of course, you want all the choices to seem right, but make sure you don’t get so close to right that someone might have a valid argument as to their choice being correct.
• If space allows, use more than three choices. Remember, the more choices you have, the less guessing is involved, e.g. four choices means a guess rate of 25 percent, five choices means 20 percent.
• Avoid always making the right answer the longest choice.
• Avoid making the right answer choice “C” very often. This practice thwarts seasoned multiple choice test takers, who know the ditty, “when in doubt, guess ‘C.’” One way to achieve randomness is to list answers in alphabetical order.
• Avoid giving a grammatical clue in your question, such as using “a” with only one answer beginning with a consonant or “an” with only one answer beginning with a vowel.
• Limit your use of “all of the above” and “none of the above.” But if you use it for one question, you must use it for at least one more, or trainees will take a clue that it’s probably the right choice for the only question where you included it.

Matching

This type of test is used less often but can be fun for the trainee and much easier to design for trainers. Here’s how to write a matching quiz:

• Cover one topic per matching exercise. For example, one test could match a list of chemicals with a list of personal protective equipment the chemical requires. A separate test could match a list of chemical regulations with a list of quotations from those regs. Or one test could match a list of employment laws with a list of the rights they protect and another test could list employment law acronyms with a list of their full names.
• Limit the number of items to around 10. Fewer than 8 can be too easy and more than 12 can get too confusing.
• Lay out the test on one page so trainees don’t have to flip back and forth.
• Make the items in each list brief. Use names, objects, tools, agencies, etc. Avoid making an item longer than one sentence.

Fill in the Blank

These tests can be tricky to write, but they prove that trainees learned the information because they have to produce the right answer without seeing it on the page as in any of the previous tests mentioned. Follow these tips for composing fill in the blank questions:

• Use only one blank per question. Too many blanks don’t give trainees enough information to even grasp the topic.
• Keep the blanks to specific information, such as regulation titles or government agencies, etc. Make the nouns or verbs in a sentence the blank lines, not the adjectives or adverbs.
• Word statements so that there can be only one answer that correctly fills in the blank.
• Place blanks later in the sentence, which helps to give trainees the context of the topic.
• Avoid grammatical clues immediately preceding the blank, such as “a” or “an.”

Other Evaluation Methods

Some other evaluation methods include first-hand observation and production data analysis. You can use observation both in the classroom to monitor trainees as they practice new skills and later on the job to see how they’re incorporating training into their performance. Your goal is to see how well the trainee learned the skill. You can develop a rating scale or simply make comments.

Use production data analysis by keeping track of production quotas and other data before and after training. Compare the results to see measurable ways training improved job performance. Make sure you take into account variables other than training that could have affected performance when you look at these numbers.

Transferring Learning to Work

In order to ensure your training has been effective, you need to do more than evaluate. You also need to take post-training time to help trainees transfer new skills and knowledge to the workplace— and to make these changes stick. You may need to help employees overcome certain obstacles

to applying training to the job.

Obstacles to the Transfer

Here are common obstacles employees may encounter:

• Learners can be set in their ways and reluctant to embrace change.
• Upper management may not offer enough support either in terms of morale or materials.
• Learners may lack enough motivation or incentive.
• Learners may not have enough time to practice new skills and achieve a comfort level.
• Co-workers may apply peer pressure against changing methods.

How to Avoid Obstacles

Here’s how you can help trainees avoid obstacles to transferring what they’ve learned:

• Give your personal support. A few weeks after the training program, send trainees a note or e-mail, or give them a call. Ask questions like these:
  • How is what you learned affecting your work?
  • Are you having any problems or concerns?
  • Do you have any final comments or suggestions about the learning experience?
• Make your support ongoing. Have an open-door policy with employees so they know they can talk with you at any point about any issues that come up as they transfer learning to the job.
• Establish group support. At the end of training sessions, assign trainees to small groups that will meet regularly for a while after training. Participants can use these groups to discuss common problems and concerns, how they are doing in applying the new skills, and to perform practice sessions. Group support gives members a sense of unity and security, which they can fall back on when they run into problems.
• Compose checklists or cheat sheets. Give these out at training sessions for employees to take with them back to work and refer to as needed. These are especially helpful for new procedures or new steps in existing procedures.
• Set up a coaching program. Coaches can be trainers, qualified experienced employees, or members of upper management. The point to this method is to have someone on call to answer questions, give feedback, give support, and to be a role model for proper
  behavior.

Putting It All Together

Investing in human capital is a wise decision for every organization. Training makes better employees, and better employees make better companies. Keep in mind training is much more than a one-time event as long as methods and technologies keep changing the way we work. For companies to stay competitive, they must invest in their employees by turning them into lifelong learners.

Follow the steps outlined in this manual to design and run an effective training program that will motivate and train your workforce to be the best in the industry—and that will have positive effects on every department in your company.

• Make training a top priority at all levels of the company.
• Develop a training program that meets training needs and is customized to your company and its employees.
• Choose the right training techniques for your training topics.
• Decide when and how to outsource some of your training needs.
• Prepare people, places, and things for your training sessions.
• Conduct effective training sessions with a sound presentation that incorporates engaging activities.
• Evaluate training at every level.
• Assist trainees as they transfer learned skills and behavior into their work.

With these steps in hand, you’re now ready to put together a top-notch training program.