We're seeing higher-quality evaluations

I am writing to thank Dr. Jafar Javan for stepping up to champion evaluation in the United Nations. As my Division, the Inspection and Evaluation Division of the UN Office of Internal Oversight Services (OIOS-IED) has been engaged in promoting evaluation in the UN Secretariat for some years now, it is encouraging to be joined by colleagues in the respected UNSSC.

Dr. Javan’s post is an opportunity for the UN evaluation community to take stock of both recent progress and continued areas for improvement. Based on the regular evaluation capacity assessments across the UN Secretariat conducted by OIOS-IED, we would like to suggest a more balanced picture of evaluation is warranted than what Dr. Javan suggests.

To be sure, his observations are well founded in some areas based on the most recent available evidence. There is certainly a need for a culture change across the UN, as the Secretary-General has advocated with his commitment to strengthen the culture of accountability and evaluation through ongoing reforms. The recent OIOS-IED Biennial Report on Strengthening Evaluation (Mar 2019) found that organizational culture continued to be considered by stakeholders as a major hurdle to strengthening evaluation. Surveyed programme managers across the UN Secretariat provided mixed ratings on dimensions of evaluation culture with innovation, lessons learning, and other types of self-assessment considered the weakest areas.

Dr. Javan is also right to underline specific areas for continuous improvement in evaluation practice, including the need for both sufficient funding and time for producing quality work. Too often evaluation units are under-resourced below minimum organizational benchmarks. There were equally mixed results in the Biennial Report in how well evaluations led to policy and programme discussions.

Yet Dr. Javan skims over progress in both the quality of evaluations and the institutional framework that supports the conduct of credible evaluations. Many UN evaluation units now strive to adhere to the norms and standards for evaluation in the UN system. As found in the Biennial Report, the overall quality of evaluation reports improved modestly according to an independent quality assessment based on United Nations Evaluation Group (UNEG) criteria. A survey of programme staff also indicated that the majority positively rated the credibility, timeliness, usefulness and relevance of these evaluations. Nonetheless, the numbers should be even higher to ensure that evaluation is a “best friend” for all.

To Dr. Javan’s aptly titled post, friendships are built on a mutual respect and trust. In recognizing real and urgent areas for improvement, evaluation in the UN must build on its recent progress to consistently deliver value across the system, and evaluators must continue working to forge friendships with management and programme colleagues, so as to be less misunderstood, and not be seen to be ‘missing in action’.