Alternative and Bottom-Up Peace Indicators
Edited by: Roger Mac Ginty
Routledge, 2015
Many recent Peace and Conflict Studies publications touch on the issues of impact and evaluation. They discuss how we can best assess what peace processes do or do not accomplish. This literature is related to parallel discussions regarding the standardization of peacebuilding practice and the professionalization of the field, the institutional or top-down nature of international interventionism more broadly, and the apparent failure of many peace interventions specifically. These trends – often identified in the practice of international peacebuilding – have inspired both more critique of existing evaluation processes and increasing calls for alternatives that are less institutionally focused, more locally salient, and less marginalizing of the supposed beneficiaries of international interventions for the purpose of peace. Roger Mac Ginty’s Alternative and Bottom-up Peace Indicators (2015) is a collection of essays previously published as a special issue of The Journal of Intervention and Statebuilding (December 2013) that attempts to build on this momentum by developing alternatives to top-down evaluation and by framing evaluation itself as a process that can empower local actors.
In the introduction Mac Ginty provides an introduction to some of the key problems described above. He first notes that the standardization of “template-style interventions” (p1) continues to dominate post-conflict contexts and that state-centric or top-down evaluation remains dominant. He then describes five specific problems with this approach: three technical issues; 1) what to measure, 2) at what level to measure, and 3) how to measure (top-down vs. bottom-up), and two more systematic challenges; 4) the exclusionary language or “professionalized argot” of such evaluations (p5) and 5) the economic incentives of the monitoring and evaluation industry. The purpose of the book, however, is not to dwell on these challenges or to rehash the critiques of current peacebuilding evaluation. Instead the purpose of the later chapters is to conceive of and outline alternative modes of evaluation that can address the 5 challenges noted by Mac Ginty in the introduction; to extend and build on existing evaluation practices to more accurately assess the impact of peacebuilding interventions on local actors.
However, while three of the five contributions to the book do propose some new ideas regarding evaluation, which will be noted below, there are also a couple of very weak chapters and collectively the book serves more to initiate a conversation regarding alternatives to the current orthodoxy which requires a much more sustained and significant inquiry. Below I will highlight the more useful recommendations described by the contributing authors and some of the weaknesses (or areas for future improvement) that are left to be tackled in what is an area in need of great attention.
To focus first on the strengths: While Mac Ginty noted the over-reliance on quantitative methodologies of evaluation, a number of the contributing authors build on this critique by describing the specific limitations of such methodologies in weak or conflict-affected states. Sarah Holt describes, for example, the challenges for quantitative methods of evaluation in states lacking the necessary resources to collect and report accurate and reliable data (p13-14) – an argument echoed by recent work (Jervin 2013; Millar 2014) – and supports the move towards more locally contextualized and qualitative research. While not really a new insight this approach nonetheless leads three of the chapters to provide initial typologies of alternative peace indicators (Holt: p25; Bowd and Özerdem: p41-47; Brown: p79-83), and while these typologies contain varying levels of detail and are obviously limited in their applicability to different cases, contexts and disciplines, they nonetheless provide a useful starting point for future thinking about alternative measures of everyday peace.
A second strength of the volume is, indeed, the willingness among the authors to recognize the contextual sensititivity necessary for peacebuilding evaluation. In countering the top-down approach and focusing on local everyday experiences of peace the contributors open the door for local definitions of peace indicators. Brown, for example, builds on the work of Kalyvas (2006) and others to argue quite forcefully – and I feel convincingly – that local or everyday measures of peace must be used as a measure of peace at the national level (p77), thus flipping the usual approach of measuring national level variables as proxies for local experiences on its head and opening the door for valuing local experiences as proxies for the success of national level peacebuilding projects.
This is associated also with the volume’s third strength, which is a shared recognition of the need to include more diverse voices in the process of evaluation. A number of the chapters call for evaluators to listen to a wider selection of people and thus to measure local experiences of peacebuilding interventions among populations often excluded by top-down approaches. However, in addition, the contributors also recognize the need to include a wider variety of individuals in the process of evaluation as evaluators. Bowd and Özerdem most directly describe a need for participatory research methodologies and for this form of evaluation to feed back into what they call “dynamic programming” (p40) that is responsive to local people’s needs and approaches. It is argued – again I think quite convincingly – that such evaluation can produce better practice and, as a result, better outcomes.
However, while these chapters do forward some new thinking about evaluation and alternative peace-indicators, the book as a whole both leaves much work to be accomplished (as can be expected) and fails in many respects to address the real complexity of alternative conceptions of peace (which is a more significant problem). The various contributors come from very different perspectives, focus on a diversity of cases, and discuss evaluation of a variety of different peacebuilding mechanisms; from broad foreign policy (Pia), to social reintegration (Bowd and Özerdem), to political memory (Brown). This is representative of the depth and breadth of the peacebuilding field, but it is also a real weakness in trying to conceive of what that field might mean by “alternative peace indicators”. Indeed, it left this reader somewhat concerned about the sheer variety of experiences and outcomes that need to be considered across a multitude of cases, an array of peace interventions, and a universe of localities.
This is directly related to the failure of the various chapters to provide satisfactory responses to the broader challenges inherent to the form of evaluation they each propose. A number of the chapters acknowledge that there are huge challenges facing participatory, ethnographic, or generally more qualitative processes of evaluation. These challenges are clear in that such evaluative methodologies require massive amounts of financial resources, time, and human capital, each of which is already at a premium in weak and conflict-affected states (as is also recognized by these authors). But while Holt, Bowd and Ozerdem, and Brown all quite rightly note these challenges they then fail to articulate or suggest any solutions, instead outlining very broad typologies of alternative evaluative methodologies with little reference to the real costs of implementing them. Again, this problem left me further questioning the practical value of alternative peace indicators for the field as it stands today.
And finally, and partly as a result of the two failings noted above, the various chapters sometimes betray the same kind of normative assumptions that have characterized the quantitative push towards evaluation in peacebuilding over the past decade in the sometimes unquestioned assumption that a particular method or a specific kind of approach will be “better” than others. This is most evident in the chapters authored by Pia and Lugo-Ocando, Kent, and Narváez. Pia, in her article on Narrative Therapy and Peacebuilding, consistently seems to take it as given that a particular process will produce particular social effects (which seems to counter the entire idea of bottom-up evaluation), while the article by Lugo-Ocando, Kent, and Narváez seems to misunderstand or simply ignore the distinction between evaluation and research more broadly (as well as very clear problems with the use of media as a source of accurate and reliable data regarding impact). The very substantial limitations of these chapters easily outweigh their contribution to the key question and diminish the impact of the book more generally.
In conclusion, reading this book was very much a mixed experience. The book as a whole makes clear the limitations of quantitative and top-down evaluation, while also acknowledging the various and substantial challenges facing alternative and bottom-up approaches and failing to put forth real solutions to these challenges. The different disciplinary perspectives presented, peacebuilding processes addressed, and cases described highlight the need for context sensitivity but also draw our attention to the substantial challenges facing peacebuilding scholars and the field more generally. Further, while I found three of the five substantive chapters to be extremely helpful in starting to conceive of alternative or bottom-up peace indicators and to therefore be of value to the field as it tackles these issues, the other two seemed unsuited to this task and to raise more questions than they answer. This is, perhaps, the nature of an edited volume originally published as a special issue, but it leaves this reader wanting a much more comprehensive and focused attempt to tackle this important challenge.
