Key points summary

Conversation transcript

Robin Hoyle: First of all, Robert Brinkerhoff, thank you very much for joining us this month. It's great to have you here and I guess you're most well-known for your work on the Success Case Method and it'd be really interesting just to understand what it is and where that all came from.

Robert Brinkerhoff: I did a doctoral programme in Programme Evaluation at the University of Virginia and I then got engaged in all sorts of evaluation studies, mostly with government contracts, evaluating Department of Defence training programmes and the like, and that took me into trying to earn a living as an evaluator, which is very difficult because not many folks are that interested in finding out if what they're doing is working or not.

So, I started contracting with different training providers to evaluate their training programmes and one was quite fascinating, that was with Blanchard Training and Development Company, and this was about 25 years ago when their premier programme was Situational Leadership. We had a contract to evaluate dozens of Situational Leadership programmes in all sorts of different organisations. And what finally dawned on me, and should have a lot sooner than it did, was that here we were evaluating a training programme that was delivered precisely the same way every time. It was delivered the same facilitators, with all the same verbiage that went along with it, the same graphics, the same notebooks - but we kept seeing over time, vast differences in the impact of the training.

We defined impact as; were people using their training in their work in a way that would make a difference to their organisational goals. But we found all sorts of variability in impact. You'd go into one company and one business unit was shooting the lights out with their Situational Leadership training and using it well and growing sales or reducing costs. Another unit in the same company had exactly the same training but nobody was using it. Even in the same cohort some people were using it and getting great results, some not.

So, what should have dawned on me a lot earlier as a researcher is that if you're holding the treatment constant but you're getting different results, something else is going on.

Robin Hoyle: So, what were the things that you saw in those companies that were really as you say, 'shooting the lights out' that was different because it's called the Success Case Method.

Robert Brinkerhoff: So, we would always look at the successes and say how did they differ from each other? Were they just smarter than the average bear or what was going on? And in fact, what was going on were some of the same sorts of things we'd find time and again.

One was clearly manager support. Some people got sent to the training probably as a punishment as well as a reward. Some people were unwilling to be there. Some people were very happy to be there. Some people had their manager support to be there, some did not.

Some people had a job where they could really use it and had a need to use it, others did not. Some people got reinforcement after the training to use it and recognition, others did not.

So, all those something else’s fell pretty neatly into when we looked at the successes and the non-successes. They were either things that happened before they got to the training or things that happened after they completed the training that made all the difference because it wasn't the training itself. That was always exactly the same for everybody.

Robin Hoyle: One of the things you've talked about is that idea of accountability in the past. The idea that it's not just that people have had a conversation, but there's some expectations that have been set. Just define those kinds of expectations.

Robert Brinkerhoff: Yeah, expectations. Let me just tell you a quick story that comes down on the other side of that and this was prior to my getting into becoming an evaluator. I was an officer in the US navy back in the Vietnam era and a freshly minted officer on my first naval station, which was down in the Caribbean on the little island of Grand Turk. I got sent for two weeks off-duty to go up to Norfolk, Virginia and take a top-secret course. We were tracking Russian submarines at the time, and we had to communicate very quickly with other naval units.

So, I spent two weeks up there learning how top-secret messages quickly and route them around the base quickly so everybody was aligned with them. I spent two weeks at that training, came back to the base. We were just a very small unit, only 65 people there. The captain, the skipper was out planting some posies by the front gate. He loved to plant flowers. As I pulled in, he said, 'Hey, Brinkerhoff.' He said, 'You've been gone for a while?' I said, ‘Oh, yes, sir, I was’. He said, 'Oh, that's right. You were up in Norfolk at training, weren't you?'. I said, 'Yeah, that's right.' And he said, 'How'd it go’? 'Wonderful’, I told him and he said 'Well look', and he waxed philosophical for a minute. He said, 'You know, you can look at the world in two ways. We can look at the world the way they did things up there in Norfolk and we can look at the world the way we do things down here.' And then, I said, ‘Yeah, that's true.' And then he said, 'So where are you now, Brinkerhoff?' I said, 'Well, I'm here, sir.’ He said, 'Then how are we going to do things?‘. I said, 'The way we do them here?'. 'That's right’ he said.

So, he undid, in that 1 minute conversation, he undid two weeks of training.

I took all my materials, I shredded them and put them in the burn bin, which is how you deal with top secret materials if you're not a president of a country. And I never used that training one lick.

If he had, on the other hand, said, ‘So glad you're back, we got to clean up this mess here with communication, and I hope you'll be using this training and talk to me next week about how it's going', that would have been a world of difference. But that expectation was never there to use it. I was accountable to meet expectations and using that training was not one of them.

Robin Hoyle: Since you first started life as an evaluator, you've taken some of the lessons that, I guess you've learned from the very different evaluations that you've done and started to work more around high-performance learning journeys. Just tell me a little bit about that work that you're doing now and how that learning from those evaluations that you carried out in many different organisations has come to affect what you do now.

Robert Brinkerhoff: Yeah, and again, Robin, it took, embarrassingly, a long time to get there! When I started doing all the evaluation work, the principal approach to training in most organisations was; take time away from work, go to a dedicated learning centre, take a course, then come back to work. The training was always separate from the work and you came back from the training and back to work. As we learned more and more that the training had to be good, don't get me wrong, you couldn't do lousy, no good training and expect results. But by and large, most of the training that I saw delivered was good enough.

Knowing that it wasn't the training itself, it was the things that happened before the training, like how you were prepared to go, why you went, what expectations you had going in and the things that happened afterwards, we realised, well, obviously, impact from training is a process, not an event.

Once we knew it was a process, then it became a struggle of, well, how can you expand the definition of that process? It includes engineering the front end and engineering the back end. That was a real uphill struggle in most organisations, the way that got interpreted in many places.

They thought, let's send some people some pre-work before they get to the training. Well, pre-work was the kind of stuff that you took out of your briefcase on the train or the aeroplane on the way to the training to see what they wanted you to do, knowing full well that no one was going to do it anyway and even if you didn't do it, it wouldn't make a lick of difference. So, you put it back in your briefcase and watched the movie instead.

Pre-work just didn't do it. In fact, the very definition of pre-work reinforced the notion that it's all about the event. The struggle was to re-envision training and reconceptualize it not as an event, but a journey, like you say, and also to look at it through a performance improvement lens, not necessarily a learning lens. It isn't about giving people some stuff to study, have them read it and study it again and then have them revisit it later. That's sort of a knowledge perspective, but performance perspective would talk about let's get the conditions for performance right before we go in. Let's get expectations clear, let's make sure there's alignment with the training and the person's job and then the stuff that came afterwards had to be reinforcement of the expectation to use it, coaching if it was needed, opportunity to practise in low risk scenarios and so on.

That meant you had to stretch it, but stretching it was hard when the whole premise was training was something that happened away from work, not in work. I think the biggest accelerator to our whole understanding of learning journeys has been the COVID-19 epidemic that forced us to break the training up into bits and stretch it out over time. And it wasn't just a matter of chunking up the knowledge stuff into bite sized pieces and delivering over time but envisioning it as more of a performance improvement process. The fact that most people were undertaking those learning interventions from home and were working from home, actually that was the first time, certainly in my 30 odd years in the industry, that we had training actually happening in the workplace in a meaningful way. Now, I know there are things that have happened in the past. We've done apprenticeships and things like that for years and years. But this was different in the environment in which people, with the tools that they were using, whether it was Zoom or Teams or whatever, had access to resources via their corporate intranet while they were in their back bedroom or their kitchen table. Whatever it was, that created or rather broke that idea of training being something you went away to do.

It was much easier for people straight away. Clearly all of that really helped immensely, that we had no choice but to stretch it out and you couldn't expect people to sit for a two-day workshop at home. We had to do something differently.

Robin Hoyle: Now, you've talked about evaluation and performance improvement and the idea of the point of training is for people to do things differently. Where do you see people are getting that right? What sorts of things are they doing to ensure that there is, first of all, that manager involvement, but also that there is an environment which in some cases just allows people to do things differently when they return to work, wherever that may be?

Robert Brinkerhoff: I think the first thing that I've seen successful organisations doing and successful Learning & Development leaders doing is being really clear up front on the value proposition of the training. Frankly, a good percentage of training that's done in organisations is not intended to change performance. Think about emergency training pilots, how to land a fully loaded 747 aircraft when the landing gear won't come down - training pilots to be able to do that. We hope they never use that. The expectation is you're not using that on a daily basis, you got other problems. Training is not sort of a uniform concept, it's got variance. Some training is done as a staff benefit, you can't expect to hire people and retain them if you don't offer them some kind of career growth.

Some training is done as a regulatory requirement. You simply have to have people go through it and record that they did it. Some of it is for just career growth and helping people advance in their careers, So, be very clear on that. But then there's some training that is sort of make or break for the organisation. Where we're talking about executing a new strategic initiative that requires new behaviours. So, getting very clear first on the value proposition is key. Is this the training that we expect and need people to use in their work? If the answer to that is yes, then let's design it in a way that will make it work, rather than designing it in ways that match all those other expectations. Now, if people arrive in a training, get engaged in the training, and they're not clear on the value proposition either, then there's trouble. So that value proposition has to be communicated up to senior leadership and all the way down to participants.

Make it clear that this stuff, we actually care about. We want you to use it and we're going to make sure you're using it and give you feedback and we're going to give you time to practice it, and we're going to give you some coaching and feedback so that we’re constructing it as a journey and constructing it as a performance improvement journey - that is what needs to be done if using it is what you're after.

Robin Hoyle: For people like me, I work with large corporate organisations, but I'm not an employee of those corporate organisations, I have to get that communication around the senior team and around the line managers. What are the barriers that people are facing there and how are they overcoming them?

Robert Brinkerhoff: Yeah, I think a good part of the barrier, and I know you as a vendor trying to sell training run into, is that often the buyers, our customers, don't really understand. They may not be clear on the value proposition. They are used to, if we look at the history of Training & Development over the last 50 years, training being sort of bought by the pound. You went to the annual training conference and you visited the vendor booths and they showed you their notebooks and they said, now look at this one, Robin, want you to open the front cover here. Look at it. It's got an insert right here for the audio tapes. Everything's all in one package here. And the heft of that notebook, nice thick cover, beautiful.

So, people were buying training, not results. Once they purchased the training, they figured, done and dusted, we've got it. Now all we got to do is just run people through it.

So, the barrier, I think, is helping our customers whether you're internal or external, you're still in exactly the same role. You're in a sales role. The highest art of selling is consultation. Helping your client succeed. Helping them understand that, no, you don't just need training, you need the results the training can give you. The training is the means, not the end. And now, if they're saying, no, you don't understand, Robin, we just got to put on some training, we need it next week and we don't have three days, we only got one day - wo what can you take out? So often they're commodity buyers and in that case it's very difficult. But in the case where the client needs the results – then I think we help them understand what it's going to take to get to those results.

One of the most useful things anybody can ever do. And this is where evaluation comes full circle - is if you're running up against a client that's resistant to making it a journey, which is pretty much all of them. Say, look, you got 3000 people you need to train, let's do this - let's take 60 of them right now and put something on and then come back and let's visit it and see how it went.

So, sell a pilot. Now, I know that's anathema to a salesperson who's compensated for the numbers that they put through, but for the internal people, that's an easier struggle. But that's what I think we've got to do, is sell the pilot, demonstrate the proof of concept, show them that it works, and when it works, tell them why it works and say, you may have noticed 30 people got great results, 30 people didn't. We've looked into why that was the case. Those 30 people that got the good results, their manager actually spent some time with them.

So, before we do the next round, let's do some manager education and make sure that you're not just pouring money down a rat hole here.

Robin Hoyle: Do you find that the managers themselves, once they're clear about what their role is, are reasonably happy to undertake?

When it seems like a mystery or it seems extra, then we see resistance. They don't know what to do. They're scared that they might get it wrong and what's more, I've got enough on my plate already, thank you. From your evaluations and from the stuff that you've been doing on high performance learning journeys, how have you seen people overcome some of those challenges?

Robert Brinkerhoff: Well first off, I think you got to recognise the reality that probably most managers define support for training as permission to go ‘well, I gave Robin some time to go. What else do you expect? I'm busy so the rest is up to him’. I think we need to give them a definition of what it means to support training and not ask them to do it all the time but again, make sure that the value proposition is very clear. There's probably only 10% of the training that your direct reports go to where you really need to give them a lot of support.

So, be very clear when it's needed. And then where I think evaluation helps immensely is give them some evidence that it works. Don't ask them to do something that you can't back up with evidence that it will work and don't say, ‘well, research shows if you do it… Robin’. Bring some data that shows them that three of their colleagues have increased their sales 30% over the last six months because they're doing some things differently from the average bear and some of those things are these things; Having a 15 minute conversation with people before you put them in this training and make clear what you expect from them, meeting with them afterwards. Sometimes 30 minutes of manager intervention can make a huge difference in whether people end up using it or not.

So, I think it's incumbent on us not to ask managers to do anything that we don't have evidence that would make a difference. And when it works, give them evidence, and when it's not working, let them know why it didn't work.

Many Managers are paid to get results. They're compensated on results. If you can show them that it's going to help them, get a result that matters to them, they're on your side.

Robin Hoyle: Yes, and at the end of the day, it comes down to the what's in it for me factor, doesn't it?

Robert Brinkerhoff: Sure, Well, I think a mistake that we've made in Learning & Development over the years is approaching managers as a supplicant. 'I'm from Learning & Development and Robin, I know you got 30 people over here. Look, you could really help me out if you if you just do these things.' Rather than approaching them as a consultant and saying, 'Robin, I know you've got your got your back against the wall to increase sales in the next quarter. I'd like to talk to you about some ways that might help you do that.'

Very different.

Robin Hoyle: Yeah. Very different. Robert Brinkhoff, it's been an absolute joy talking to you. Thank you ever so much for your time and I hope and pray that other people will pick up on some of these things and will really learn from your experience of trying things out, testing, what works and then going back with the evidence, which I think we all say that's got to be the important stuff.

Robert Brinkerhoff: Absolutely. Thank you very much indeed. Thank you for the opportunity to do this. It's been a joy to talk with you and I'm sorry our time is up as fast as it was.