#learning roi

A colleague of mine, who was a group HR director of a global 500, was asked by his CEO 15 years ago, a simple series of questions. How much are we spending on training? Where is it working and where is it not? Where should we be spending less and where should we be spending more? Curious as to what the answer was, I asked and he responded, “Honestly I have no idea”. After lots of "Big 4 Consulting" he managed to finally arrive at somewhat of an answer. They were spending around £100m on this training, but he still had no way to measure what was having an impact and what wasn’t, and had only been given instinctive recommendations on where to better apply that £100 million. Today, I think if the current CEO of this global 500 asked the same question to his HR director or Chief Learning Officer, he would still get the same answer. Crazy that this is likely the last area of big business that spends this much, yet can’t provide a measurable value answer back to the CEO.

Fifteen years ago Marketing would have been equally as fluffy, but not today. New marketing automation software, new mindsets, skillsets and a laser focus on KPI and business value has made marketing today completely different. When I recently interviewed for Fuse Universal’s Marketing Leader, like most CEOs, I filtered out anyone that wasn’t able to give concrete examples of campaigns that they had led, which weren’t able to describe measurable business value. This is a minimum criteria to prove that a Marketing Leader has moved their mindset into the modern age of marketing, where having a data DNA, is as important as their creativity.

Gone are the days where a marketing leader would say “We assume it’s working or people feel good about it afterwards”, which crazily enough I still hear from Learning Leaders. Kirsty (our new Marketing Leader), like any good Marketing Leader, defines her campaigns and chooses the marketing weapons to deploy, based on which ones are most likely to achieve the KPI that campaign is focused on. Be it a number of sales qualified leads or top of the funnel new contacts. The mindset, skillset and toolset of a modern marketing professional today compared to 15 years ago is radically different but if you look at the world of L&D – not a huge amount has changed, but just as digital disruption came in the form of Fintech & Martech, next-generation learning tech is enabling modern thinkers and practioners, like TJ at Merck and Peter Stone at Catalyst to spearhead the same level of digital disruption that we have seen across almost every other industry. L&D may have been late to the digital disruption party, but we are now well and truly on the guest list.

One of the problems that digital disruption is going to solve, has been something many of us have kept as a quiet secret up until recently, and that is there is a whole lot of corporate training out there that offers almost zero business value. Worse is that we have hidden behind an untruth for a long time that “You can’t measure the impact of learning and training”. This may have been a historical truth as it was historically for marketing, but it is no longer true today. Now that access to rich data and rich data tools allows us to measure people performance, it is revealing the bigger more scary truth that corporate training is rarely designed for business value and often when it happens, it happens by chance and not by design.

...A small idea that changes everything; design backward from business outcomes not forward from learning outcomes

There is one way to make sure that the impact of a learning design will be far higher. By doing just one small but significant thing, we can change corporate learning forever. Our friend Rachel Hutchinson and her team at Hilti are doing just this, by “Deciding on the problem they are solving, being clear on the outcome and business impact they are going to measure, and most importantly being crystal clear on exactly how the measurement of business value will be achieved before any learning design is even considered." If the business asks for 2 courses on x or y and you accept, then it’s too late. Measurement of business value must come before learning design, as it impacts every decision on learning design thereafter.

In the case of Hilti, Rachel and her team hypothesis the impact that they believe they can make by looking at descriptive and predictive analytics, in order to forecast the delta of difference, a learning experience can make on every learning experience. If they conclude they can’t make a difference, then come to the conclusion that they don’t do it.

This one simple idea of designing backward from the business outcome rather than forwards from the learning outcome, will change everything in the learning design, because every choice becomes about shifting that performance dial, and anything else is secondary. In that secondary bucket includes traditional learning thinking and instructional design techniques, because they are all learning output focused, not business outcome focused. It’s time to blow them up and start again. If you do design learning that is actually designed with a business outcome in mind, it will allow learning professionals to talk the language of the CEO/business, and transition their transformation from someone you go to ask for a course, towards being strategic business partners that can directly help improve the performance of your team and company.

So how can learning be measured for business impact? The answer is in a multitude of ways. Here is one example: almost every role in every company has performance metrics – Sales may be the most obvious, but Marketing have theirs as do service agents, and within every company, there are high performers and average performers. If you map the behaviors, skills, and knowledge of the best people and design a learning experience that shifts the median upwards toward the best people, then the delta of difference is measurable, and the impact is measurable. In Sales this measurement may be conversion rates; for developers, it may quality of code, documentation and velocity. The key is to correlate learning data and performance data to prove the value that the learning is having.

A common excuse of learning professionals to avoid this new reality that our profession can be measured is to say by there are so many other variables that its impossible to say it was the learning that made the difference. A relatively quick way to prove it was your design and your hypothesis was right is A/B testing e.g run the learning experience the old way for a segment of the business and the new way for the other. It’s a popular marketing technique and provides a quick way to get data and move the myth that learning can’t be measured for impact. Its also a great way to get feedback quickly and iterate continuously. 

A more advanced technique is working with the internal performance team to understand the variables and account for them. As an example, we did some work with University College London and Carpetright to test the hypothesis that bottling the greatness of the best people on selling one key product and giving that understanding would have around 10% positive impact on revenue. UCL looked at 3 years of Carpetright's performance data to understand variables such as seasonality and it showed as around 13%, not accounting for 5% seasonality, which was a useful start!  

The data also hinted that simply doing a one-off event and not using follow up techniques such as social learning and performance coaching (which they are now turning on), meant the performance curve almost mimicked the forgetting curve. By asking the question of "How can this improve the business outcome?," the team at Carpetright are evolving the learning experience with performance coaching and social learning and they are measuring the impact.

Rachel and her team initial challenge, at Hilti, was how to bring down “time to competence” for new sales starters from 12 months to 6 months and the measurement they decided to use was "time to payback" eg the cost of salary, development, travel etc. They made the initial hypothesis that through the use of modern learning technology, micro-content, role-playing face to face and the team they had they could achieve this 6 months saving. This they did but after their initial success (because they were focused on business outcomes not learning outputs), they challenged themselves again with the question of “What would we change to shift that metric from 6 months to 3 months? It’s a great question and exactly why the business outcome vs learning output creates the right mindset to ask that question. 

After some discussion they decided that helping their face to face trainers become more digitally savvy, may help reduce the time again as their trainers would transform from only using traditional classroom techniques to becoming more socially and digitally aware. This would enable them to be able to mentor new starters before their first face to face classroom session, giving them feedback on demo pitches they had recorded and uploaded to the platform. And it worked bringing the time down to almost 3 months!

Thinking backwards from the business problem, defining the outcome and using data & analytics, will radically change the learning experience design and allows a mature conversation with the business, that is no longer about how many courses they want for how many people but what problem can we solve, what outcomes are possible and what is the business willing to commit to together for. It may be a little uncomfortable at first, as is all change - but for those on the other side, they will never go back to the old ways.

Like Rachel, Siri Wikander and her team at Scandic, partnered with Sonja Prest and her team at Atom, came to the realisation that social learning technology could be used to enhance a daily learning culture, which would assist in people engagement and staff attraction. They would benefit hugely if the 20,000 colleagues in Scandic hotels were recommending friends to work in their hotels. They are now 18 months in and the data says the hypothesis holds true and with every hypothesis, the next one gets easier, as you have more historic data to predict the future with.

As Charles Jennings said in his recent video “We need to move from course takers to value makers.” Designing backwards from the goal, rather than asking how do we prove value after we designed a course, is the beginning of new world of measurable business value and L&D moving from the corner room to strategic partner.

More and more learning professionals are now switching their thinking from reactively building the courses they are asked to by the business and instead searching for business problems they can solve. They are learning continuously on what do more of, what to do less of and what value they are receiving from the budgets they get. Flipping the thinking of measurement from after to before seems small but it’s an idea that fundamentally changes everything in the learning design and makes traditional instructional design redundant as a whole new mindset for design takes over, which I'll cover in the next article (if you got this far :)

For those who are about to move from learning outputs to business outcomes for the first time, you will never think about designing learning in the same way again. Blue pill or red pill ?

Would you like to hear more? 

Get in touch [email protected]