This is our second post from Chris Connolly, one of our current CIDA IYIP interns based in Durban, South Africa. (Read his first post here.) Over his 6-month posting, Chris’ work will focus on schools and communications.
Anyone who has spent even a bit of time in the development world knows all about the dreaded “logic models”. For the uninitiated, you can think of these like the dominant language or lingua franca of program evaluation in international development. Using these involves breaking down your program into a clean little chart that defines specific project goals, and then linearly spells these out in terms of project inputs, activities, immediate outputs and long-term consequences (outcomes). The push from this comes from a desire for well-planned, accountable projects that either deliver on what they promise or explain actual outcomes on those terms. Not a bad idea, right?
Well, it depends. The thing is, like all languages, logic models have a way of reshaping how we make sense of reality. The need for clear, specific and measurable goals seeps into our thinking throughout the lifecycle of a project. After all, there is a seductive—well, logic to their sanitized lines and fenced-in activity areas. More still, even as an approach to evaluation, it reaches back into the world of project planning, and beckons us to confront intractable problems like poverty, inequality and marginalization in predictable and rational ways. It allows us to scratch that itch we all have as Westerners to recast reality as something orderly and rational, something that we can put into boxes.
The problem is this: social change is neither predictable, rational nor linear. Deep down, I think all development workers know in their heart of hearts what funders simply don’t want to hear: that the traditional, bottom line–oriented, goals-based approach to evaluation—and, by extension, our projects—suffers from this fundamentally flawed starting point. Maybe, just maybe, our strongest asset as drivers of social change is not our logic, but our capacity to harness the energy and innovativeness of the real people who understand their plight better than we do.
Luckily, my colleagues at Imagine Durban get this. They have a solid culture of experimentation, ongoing reflection and partnership with community groups. Throughout their history, they have not been afraid to do things differently, to ask hard questions about their process, and to chase the energy of emerging opportunities from community groups. The development of their Schools Program, which I have been charged with evaluating, is illustrative of these values. The program was not created as a singular program so much as it developed organically from the overall activities of Imagine Durban. Around 2007, the consultation efforts with schools took on a life of their own as the need for partnering with schoolchildren became more evident. Previous activities are diverse and have included a number of schools-focused projects such as a climate change competition, a partnership with the Wildlife and Ecological Society of South Africa’s (WESSA) Eco-Schools programme, the involvement of schools in the annual Sustainable Living Exhibition and ongoing interactions with schools through presentations and discussions. The overall vision is to have schools implementing projects that contribute to the fulfillment of the overall Imagine Durban plan, in whatever way is appropriate to their local needs.
Keeping with Imagine Durban’s belief in doing things differently, I have been given the leeway as evaluator of this program to undertake a more responsive, practical approach called “Developmental Evaluation”. This involves asking evaluation questions in a way that energizes rather than stifles the social innovation already taking place. It involves integrating both creative and critical thinking, asking probing questions, and tracking the results to provide feedback and support adaptations along the way. Such an approach is especially important in this early explorative phase of the schools programme, which likely will be characterized by false starts, dead ends and trial-and-error experimentation. Above all, I wanted my evaluation to help nurture this important exploration.
The implication, as pointed out in the outstanding book “Getting to Maybe”, is that there is less of a focus on clear, specific and measureable goals, because clarity, specificity and measureability are limiting at the early stages of social innovation. Moreover, the goals outlined in the Imagine Durban long-term plan are broad enough so that goal setting comes from participants in the program themselves. My personal goal for the evaluation will be to offer, in the words of the authors, “a process for periodic reflection—systematically looking back and seriously looking ahead—to gauge progress, harvest important lessons and rigorously examine what is working and what is not.” Ultimately, this could result in a longer-term, partnering relationship between Imagine Durban staff and those engaged in innovative initiatives on the ground.
So you may be asking: Supporting local innovation is all well and good, but how does developmental evaluation actually do that? Basically, there are two fundamental ways of looking at the evaluation of Imagine Durban’s schools projects:
a) What are the activity areas of Imagine Durban, and how have these translated into impacts at the level of individual schools, if any?, OR
b) What are some of the overall impacts that schools are experiencing, and how were these achieved? Conversely, if there hasn’t been an impact, what is missing?
The first (i.e. traditional) approach (a), supports a sort of tunnel vision in which the way we are doing things now is the only thing worth asking questions about, rather than what we could be doing differently. Alternatively, the second line of questioning (b), which is better suited to our goal of tracking progress, extracting important lessons and finding out what is working and what isn’t.
This is developmental evaluation in action: asking questions that support exploration and innovation towards the achievement of a shared vision of change. Sounds logical enough, right?