By Anzél Schönfeldt
A large contingent of SAMEA members attended the bi-annual African Evaluation Association (AfrEA) conference co-organized by the International Initiative for Impact Evaluation (3ie), UNICEF and the Network of Networks on Impact Evaluation (NONIE) in Cairo during 31 March to 2 April 2009. Some tangible outputs resulting from the conference included:
- The formalization of AfrEA on its tenth anniversary culminated in its 2009 board election with Florence Etta as president, while Benita Williams, our current SAMEA treasurer, was elected to the secretariat
- The launch of the Journal of Development Effectiveness (http://www.tandf.co.uk/journals/titles/19439342.asp)
- The release of the NONIE Guidance on Impact Evaluation Draft (http://www.worldbank.org/ieg/nonie/guidance.html); an updated version with incorporated comments from stakeholders during the Cairo conference is set to be released in June
- Significant progress in the debate around and towards the launch of an African Evaluation Journal under the guidance of Bob Williams, Donna Mertins, and Bagele Chilisa and SAMEA’s Kevin Kelly
- Indications of initiatives to develop Centers of excellence to improve M&E capacity in Africa by the World Bank’s Independent Evaluation Group
- The identification of Madagascar as host of the next AfrEA conference in 2011
All about Partnerships
“This conference was the epitome of partnership”
- Dr Florence Etta, AfrEA president
More than 700 participants from a wide range of industries and countries came together to join the debate around the translation of (often highly contested) impact related definitions, methodologies and design into action within the challenging context of the current global economic crisis. Ms Odette Ramsingh, Director-General from the Office of the South African Public Service Commission, reiterated in the conference’s summarizing address that “there was a clear recognition that development interventions are characterized by partnership and networks, and the development discourse recognizes that many partners, drawn from different sectors, contribute to the development of place and people”.
As such, the recently launched Association of African Public Service Commissions in Africa (AAPSCOMS), NONIE and 3ie were recognized as important platforms for the sharing of ideas around impact evaluation. With hundreds of African evaluation association representatives in attendance, including the newly formed Egyptian evaluation group, the Cairo conference has generated an increased interest in peer and organizational collaboration. As a result, several Cairo conference participants and influential evaluation expert expressed interest in SAMEA’s upcoming conference in August, including Jennifer Greene, Jim Rugh and Howard White.
Capacitating African M&E
Capacity building was a recurrent conference theme. Through the provision of capacity building events and building the demand for impact evaluation by broadening the understanding of impact evaluation, Ms Ramsingh urged that evaluators should question how “our own expertise, forged in incubators like AfrEA, to do rigorous evaluations” is used to contribute to quality of governance as M&E is “a critical element of the management of any country”.
More specifically, one of the conference outcomes included common country actions and the stimulation/coordination of in-country impact evaluation dissemination efforts. As a result, a group of SAMEA members who received bursaries to attend the conference has decided to make a collective effort in disseminating and promoting impact evaluation through SAMEA’s existing structures. SAMEA will also be launching impact evaluation topical interest groups through its website in June 2009.
Improving lives – The bottom line
In the context of diverse stakeholder interests, methodological debates, multiple goals and increasing pressure to produce results, it was clear that there are commonalities which drive impact evaluation; coupled with words of caution as to not to lose sight of impact evaluation’s true aim:
“We have come a long way from Cape to Cairo, and this conference is about asking the hard question of what the relationship is between impact evaluation and improving development…Ultimately development must be about improving the lives of ordinary people; that the process must be empowering and owned. If communities are to play a meaningful role in the interventions that seek to make a positive impact on their lives, their voices must be amplified in the evaluation process. Through such interactions we will then get a clearer understanding of the situation. If we silence their voices, how will we truly know?”
No Golden Standard
Impact evaluation is clearly contested terrain, however, given the extent of debate around evaluation and impact definitions, context, design, methodologies and perspectives, discussions during the Cairo conference reiterated that there is no universally set golden standard. Some lively discussions included the usage of randomized controlled trials (whose proponents were referred to as “randomistas” at some point) versus evaluation as a capacity building/improved governance process. It was suggested that evaluators should try and combine methods in ways that answer the most important questions. As Jennifer Greene argued, issues of substance should be the departure point of such debates; “methods should be the servants” of substance.
As noted in the Cairo summary session, “the more important conversation is about values and voices, not methods”. With this in mind, we look forward to exploring these issues further at the August SAMEA conference under the very appropriate theme “eVALUation”
We would like to make use of this opportunity to thank the sponsors, the Cairo organizing committee and all the other key role players, including the SAMEA board, for making the Cairo conference an unforgettable learning experience.
An impression from Cairo: