It has been a year (already!) since the conference on theory of change (ToC) development organised by the Methods circle of DeGEval, the German Evaluation Society. We met at the impressive (1950s) premises of DeStatis, the Federal Statistical Office in Wiesbaden. It was a massively enjoyable and super well-organised event with insightful contributions about the history and practice of ToC development.
A theory of change helps us explore why and how well-defined things have (or haven’t) happened in an intervention. But beware! It is just a theory – not any ultimate truth.
Apparently, the term ToC was coined in the early 1970s by late Carol Weiss, a key founder of evaluation of social programmes and policies. A ToC can clarify how an intervention (such as a project or programme) is supposed to work – what it is doing, what it is intended to accomplish, and what the likely connections are between actions, their products and wider results. Carol Weiss showed, among other things, that a theory of change helped finding out whether a programme or policy did not work because it was not carried out as planned (implementation failure), or because it was based on faulty thinking on the ways in which the desired change would happen (theory failure).
One can develop a ToC, or a tentative ToC, deductively, using generalisable evidence from research, evaluations and evidence reviews – ideally when planning an intervention. Evaluators often find themselves working inductively only, reconstructing ToCs later in the life of an intervention, when there has been no explicit theory of change in the beginning: We pull together data from the project documentation and facilitate participatory processes to reflect on the intervention, its intended results, and the logical connections between activities and their broader outcomes. In the process, we also develop hypotheses on how things happen in the ToC – that is an abductive activity. Then test that tentative ToC against existing evidence (deduction, again) and data from the intervention (induction).
Apologies for this addictive use of fancy terminology! It makes ToC development sound very complicated. If you do it thoroughly, it is complicated. Anna von Werthern showed, in her conference presentation on unpacking the black box of ToC, how she performed several loops of careful data collection and joint analysis to develop, step by step, a well-founded ToC grounded in evidence. Such iteration is important.
What does a ToC need to be useful? That question guided the discussion concluding the workshop. The consensus was: To be accepted and trusted, a ToC does not need to deliver utter certainty. But it needs to be clear (defining its elements), well presented (with a crisp diagram that focuses on necessary elements), practical (elements that matter most to the stakeholders, and terminology that is familiar to them – using fewer big words than this blog post 😉), and transparent about its limits.
Depending on what an evaluation needs to find out, ToC development or review can be quick and rough, or more thorough. There are only few cases when I would skip it totally in evaluation or planning. Even a lighter-touch ToC development process with a very tentative theory of change helps structure one’s thinking. If forces people to define what they do and what they want to achieve, surfacing assumptions and making it easier to test them.