The Organisation for Economic Co-operation and Development (OECD) has issued a guidance document on Applying a Human Rights and Gender Equality Lens to the OECD Evaluation Criteria. It is wonderful to have this new resource – but a few shortcomings may make it hard to apply the guidance in real life.
First, the good things! About half of the 60-page volume is dedicated to explaining how human rights and gender equality (HRGE) considerations can be integrated (mainstreamed) into each of the six OECD criteria for evaluation in international cooperation (relevance, coherence, effectiveness, efficiency, impact, sustainability). There are inspiring ideas: For instance, the volume invites evaluators to assess internal coherence of the intervention under evaluation (the evaluand) by checking whether it is aligned with human rights treaties and related policies. Also, the publication includes helpful definitions and great examples from real evaluations, as well as sample evaluation questions. It is written in a style that is accessible to evaluation specialists (not quite plain English, but not too jargony). And it points to plenty of useful references.
The document encourages evaluators to apply a human rights and gender equality lens to all evaluation criteria. However, there is no discussion of the RESOURCES needed for meaningful implementation of the guidance. In that way, OECD risks encouraging tokenist tick-boxing/flag-waving motions instead of serious consideration of HRGE. For example, OECD invites evaluators to reconstruct the evaluand’s theory of change with special attention to HRGE, to detect intended and unintended HRGE effects on various groups of people. That can work if the project focuses on HRGE, but if it doesn’t, it takes extra time and expertise to add the HRGE dimensions. Also, the document commendably advocates for systematic participation of a wide spectrum of rights holders in the evaluation process – not just as data sources. To make this possible, people need to be reached, invited and reimbursed for any costs, translation needs to be organised and so forth. The guidance would be more useful if it included estimates of the extra time and resources it takes to translate it into practice.
Another issue is about LEARNING. I love the fact that there are nifty tables with sample HRGE-sensitive evaluation questions for each OECD criterion. But most questions start with the phrase „To what extent…“, which invites accountability-focused answers of the type yes/no/somehow. But isn’t evaluation also about discovering what has worked, under what conditions, and what not, and why? Obviously, these are aspects that an enterprising evaluator can discuss even under a question that starts with „To what extent…“, but we don’t always have time to add extra depth. The guidance would be more useful if it was geared to support both accountability- and learning-oriented evaluations.
There is another point I find difficult. Human rights are, by definition, indivisible and interdependent (a good reference is the definition by the Office of the High Commissioner for Human Rights). But the OECD resource invites evaluators to define which human rights principles are most relevant to the evaluand. Again, that could be OK for an intervention that focuses on a specific set of rights (e.g., political participation of indigenous people). But how can an evaluation team decide which set of rights should be considered when evaluating a solar energy project, a police training initiative or a multi-sector regional development programme? Should they privilege political over social rights, for example? Should they pick the right that it takes the least effort to consider (if no extra resources are available for the HRGE lens)? Is it legitimate to pick just one set of rights and leave aside all others? That question deserves careful consideration in future editions of the guidance.
Rather than attemting to plough HRGE concerns into all OECD criteria, one could put togehter a few minimum standards for HRGE sensitivity that should be applied (and resourced for) across all evaluations. This could include:
- Impeccable ethics (including trauma sensitivity, as helpfully pointed out in the OECD volume)
- A degree of equity orientation, e.g. by considering unequal distribution of desired effects, and unexpected/unwanted effects by population group
- Communication of the evaluation purpose and findings to rights holders (as suggested by OECD)
- Mainstreaming HRGE concerns into evaluation questions around relevance and effectiveness