August 25, 2021 / Edited by Leigh Stewart
A conversation with Dr Jess Dart, Jamie Gamble and Kate McKegg
What happens when suddenly everything changes and the old ways of working just…don’t?
Creators of our brand new course on developmental evaluation met virtually to discuss how developmental evaluation can help evaluators and social innovators respond and adapt to ever-more complex and changeable environments. But first – what exactly is developmental evaluation? And what is it not?
The thing that defines developmental evaluation is that it’s what you’re developing doesn’t exist yet. It’s developmental in purpose, aimed at supporting the process of innovation to make new things and create new solutions and ways of working.
Developmental Evaluation is a practice for complex and unpredictable environments.
It’s particularly useful for when we don’t have high levels of agreement about what the issues are, or how to address them. In these cases, we need an evaluative practice that is able to work alongside initiatives as they develop, as things emerge, and be able to adapt and respond to changing needs, while at the same time, supporting critical thinking, inquiry and rapid learning.
I agree with Jess that it’s useful for when you’re creating something that hasn’t existed before.
And it can always be employed when you are seeing significant changes and need to adapt to these as all of the sudden what you’re doing fundamentally shifts. Or, if you want to take a success and begin applying it to different part of a program or initiative – developmental evaluation can be a great support to for that shift.
Jamie Gamble
It’s not about abandoning evaluation – it’s about bringing the best of evaluation into a social initiative, including critical evaluative thinking, use of evidence, use of data.
Kate McKegg
Developmental evaluation is not just about tracking and monitoring and describing what’s going on, nor is it about developing long term evaluation plans and designs, although it’s likely to use monitoring data to inform inquiry, questioning, and evaluative thinking as an initiative is unfolding. Developmental Evaluation is likely to unfold in smaller cycles of design and implementation, and always has to be open to adapting when things change.
Dr Jess Dart
It’s not ‘evaluation light’. We still have to apply all the tools in our evaluation toolkit, but we have to do it on our feet. It’s agile evaluation – I’ve even sometimes seen it referred to as ‘realtime’ evaluation. We work alongside program design – we bring evaluation tools to the table, and we have rigour around testing things.
Kate McKegg
I love how much more attuned DE can be to the immediate learning and inquiry needs of people innovating and engaging in really tricky and tough issues. I also love how it can work with the rhythm of those on the ground, with our communities, and not be squashed into a timeframe or design frame that is imposed by someone else. It really can be by us, for us.
Jamie Gamble
I think we’ve reached a saturation point for traditional problem solving. We’ve actually made good progress on technical problems, but we’re facing diminishing returns on deploying those traditional approaches to the significant problem areas that remain, for example poverty, climate, inequality. We need tools approaches and resources that support the ability to scale innovation and to adapt as needed to find our way through these massive challenges we face. Developmental evaluation is optimally suited to do that.
Dr Jess Dart
The time for social innovation is now. That means learning your way through change, trying things you haven’t tried before and working with communities in new ways. As evaluators and social change makers, we can see that these issues are getting harder to address, so how does evaluation sit with that?
Developmental evaluation can power new ways of working – it can help make initiatives better, scale faster, achieve greater impact, and even “de-risk” what you’re doing. Some governmental will only support social innovation if it is accompanied by evaluation, because it de-risks the design process. For example, if you’re looking at how well the engagement is going, if you’re looking at how people are responding, how safe it is – if you’re evaluating your initiative or program from all angles at the same time, then it’s a safer ride.
It’s already proving a really appropriate methodology for all the changes people have had to make throughout the pandemic.
Jamie Gamble
Check that it’s really a developmental situation – is something table that involves serious innovation and complexity? Developmental evaluation tends to be a little more resource intensive, so you want to ensure you’re directing resources to where they can achieve the most.
Secondly, are you – your organisation, your initiative – really ready to learn? Do you have the authority, the will, the capacity to adapt? Because while developmental evaluation may help you may learn a ton about how your project is creating impact (or not), if you’re in a rigid system and you can’t actually apply what’s being learned then you can short-change the development. Check readiness before you proceed.
Kate McKegg
It’s called evaluation for a reason – it’s got “value” in the middle. While developmental evaluation is not evidence light, it is values rich. We need a critical practice in developmental evaluation to discover, explore and unpack our assumptions and the values that we hold and that others hold, because they make a difference to the choices that we make, they impact decisions we make about how we proceed, and those will affect someone.
It’s important to get to the heart of the things that are really important to you, your community and everyone with a stake in your project – what really matters and what you are trying to achieve? What counts as valuable? What would you see, hear, and feel and what would the facts be if you saw value in this initiative, and asking these questions often.
Dr Jess Dart
I’d say don’t rush into evaluation planning, or spend a large part of your time developing a big evaluation plan upfront. Try to be useful and stand behind the social innovation process, and let the evaluation plan unfold, perhaps later than you usually would, and try to keep it light, as it’s likely to change. In fact, it has to change as the thing you’re developing starts to develop, because you don’t yet know exactly what you’re going to be doing. Don’t use up your resources developing up detailed plans at the outset – it’s better to be useful.
132B Gwynne St, Cremorne 3121 VIC
P (+61) 3 9425 7777 | info@clearhorizon.com.au
Clear Horizon acknowledges Aboriginal and Torres Strait Islander people as the Traditional Custodians of the land and acknowledges and pays respect to their Elders, past and present.
Our Community