When people do have an evaluative mindset, they begin to appreciate having a specialist who helps them look at things through the eyes of an outsider. It is something I do often. As an evaluator, I help managers, employees and even shareholders understand how evaluators think, how they define criteria and interpret data, how they draw conclusions – but they would still need an evaluator to make their job easier, to have another pair of eyes to look at things from a different perspective. Helping to think in an evaluative way is about improving communication and understanding of what’s going on.
So you don’t think that evaluation will become just a part of management, because the evaluator has other functions besides allowing you to understand what’s really going on.
Yes, that’s right. Evaluators also give an additional point of view. Even where there are internal evaluators in the management, their job is to do the evaluation and make sure there’s good data and make sure people take the time to interpret it. I believe the evaluator function will remain, but the better management understands evaluation, the better communication and use of evaluation results will be.
In your 2009 article, “The Future of Program Evaluation,” you wrote that the “gold standard” of evaluation methodologies would be hotly debated over the next few decades. Can we say that the answer to this question has changed significantly since then? Or do we still have the same leader – randomized controlled trials?
The debate continues, but it’s becoming increasingly clear that in a rapidly changing world, we need fast-turnaround techniques that produce real data in real time. Randomized controlled trials are really not very useful in a complex, dynamic, rapidly changing world. They work best in a stable situation where there are clear fixed interventions.
In a rapidly changing world, we need fast-turnaround techniques that produce real data in real time and provide them just as quickly to people.
If you have to work on topics like pandemics, climate change, or civil unrest when things are changing rapidly, randomized controlled trials are not appropriate – they are too rigid and too time-consuming.
The speed of change means that the evaluation must produce results quickly, in real time, and provide them just as quickly to people who need to make decisions in a time-constrained environment.