elearning

Evaluating Training Effectiveness and ROI

Dec 10, 2015

Every year, companies all over the world create hundreds of thousands of e-learning courses and conduct hundreds of thousands of trainings. Courses are created for training both employees within the company and unaffiliated personnel, such as clients or other third party individuals. The creation of courses and their consequent employment in the education process is, in most cases, quite costly. When training the company employees, it is important to consider that the acquisition of new knowledge and skills will distract the employees from their duties and cost the company time. To evaluate the effectiveness of education and its practicability from the financial standpoint, it is necessary to calculate the expenses connected with creating and conducting the trainings, gauge the results achieved by the employees who have taken part in them, and decide whether the increase in the employees’ efficiency and company profits was sufficient to recoup the associated costs.

 

The price of training and all accompanying activities concerns all managers, and, in fact, should concern the instructional designers as well. What reason is there to conduct further trainings if the benefits of the ones already conducted have not been assessed? How to convince one’s manager that it is worthwhile (and profitable) to continue the training program and expand it to include other departments if you have no data to profit cost ratio to back you up?

How, then, does one calculate the usefulness of training? Luckily, there exists an all-purpose tool widely used by managers responsible for internal training processes - Donald Kirkpatrick’s Learning Evaluation Model. The model is relatively time-consuming to implement, but the accuracy with which it helps you understand whether your training program should be continued and how it can be improved is well worth the effort.

The Donald Kirkpatrick’s Learning Evaluation Model consists of four levels:

  • Level 1. Reaction.
  • Level 2. Learning.
  • Level 3. Behavior.
  • Level 4. Results.

You can read about these levels in-depth in my previous article, Getting to Know ADDIE: Evaluation. In this article, I would like to focus on the fifth level, which was suggested for addition by Jack Phillips. It is this fifth level that helps to assess the financial viability of training, its costs and benefits.

Level 5. Return on Investment (ROI)

When evaluating the effectiveness of training, it is customary to consider an additional level of the Kirkpatrick’s model, namely, the ROI methodology, developed by Jack Phillips in 1991. This methodology enables one to express the evaluation data obtained on the fourth level in terms of money, and then compare the estimated profit figure with the expenses the training program incurred.

The head of the company would require information about the projected costs of a training program before giving it the green light, especially if the budget is tight. In most cases, it is the management that insists on using the ROI methodology for assessing the results of training and personnel development. This makes the use of the methodology more or less a given when trainings are conducted.

The ROI methodology is often used to estimate the potential profit from conducting a training program, and to make sure that the projected costs would fit the budget. The ROI coefficient takes the form of a percentage, expressing the relationship between the projected profit and the projected costs of a training program, calculated according to the following formula:

ROI = ((projected profit - projected costs) / projected costs) x 100%

The fifth level of evaluation, described by the Phillips methodology, makes it possible to:

  • Estimate the cost of a training program and make a prediction regarding whether conducting the program will be cost-effective.
  • Demonstrate a direct relationship between the company’s productiveness and the training of employees.
  • Evaluate a training program as a business tool.

Is using the fifth level of evaluation always necessary?

Considering that implementing the fourth and fifth levels of evaluation according to the Kirkpatrick’s model are costly in terms of both time and money, it is important to understand whether conducting such in-depth evaluation is pertinent in your specific situation. The ROI evaluation is usually conducted sparingly, for no more than 5-10% of the total number of training programs.

The fourth and fifth levels of evaluation are usually employed only to validate the training programs concerning the company’s strategic interests, as such programs demand significant investment and are closely monitored by the company’s management. This does not mean that training programs of lesser importance should not be evaluated at all - just that the use of the first three levels of the evaluation model is usually sufficient. Here are the general guidelines for evaluating your training programs using the Kirkpatrick’s model:

  • Ideally, every program should be evaluated at least on the first level (Reaction).
  • Most programs should be evaluated on the second level (Learning) regularly, and only periodically on the third (Behavior).
  • A few programs, those largest in scope and with the greatest impact, should be evaluated on the third (Behavior), fourth (Results), and fifth (ROI) levels.

Once you’ve decided to evaluate a training program on the fifth level, it is vital to calculate the data carefully, and not resort to guesstimations. Should the resulting ROI prove to be negative, diligently calculated data will help to pinpoint the weak links in the training program.

Using the ROI prognosis in planning

Company managers usually need to know the estimated ROI of a training course long before it is developed or implemented. For that reason, in many successful companies it is customary to use the ROI prognosis to assist the managers in decision making. If any deficiencies are discovered at this stage, the corresponding changes are made to ensure that the training program is financially viable.

In conclusion (plus one more bonus of the ROI model)

Another not insignificant advantage to using the ROI model on a regular basis is the fact that it changes the attitude both managers of other branches and the top company management have towards training. Regularly evaluating training programs and demonstrating their impact in terms of hard numbers helps promote the role of training in the development of the company’s employees, as well as the company itself. Calculating ROI when planning and evaluating training programs helps to keep those responsible for their creation focused on the company’s business goals, and improves the design, development, and delivery of trainings. Thus, besides improving the effectiveness of training programs, using the ROI model changes how the company management and those in charge of approving training programs view training as a whole.