On Wednesday, November 29th, the Independent Advisory and Evaluation Service (IAES) held its first webinar in Spanish, titled: Broadening CGIAR’s Inclusive Voice: The Independent Advisory and Evaluation Service speaks Spanish.
Dr Oscar Ortiz, Senior Director of crop-based systems of the CGIAR scientific research group on Resilient Agrifood Systems (RAFS), inaugurated the event where, in his opening remarks, mentioned that he had long been responsible for both the ex-ante and ex-post measurement of science quality at the International Potato Center (CIP), a duty he now undertakes in the CGIAR RAFS Action Area. In addition to being one of the main users of the new Framework himself, he also pointed out that the community of Deputy General Directors (DGA) has contributed significantly toward the Framework’s development, which lays the groundwork for how the quality of science is currently referred to at CGIAR. Thus, the Independent Science for Development Council (ISDC)–formerly the CGIAR Independent Science and Partnership Council (ISPC)–has over the years engaged the DGAs of research and other stakeholders in the issue of quality of science, in particular:
The DGAs, CGIAR Research Program (CRP) Leaders and other stakeholders contributed to the formulation of case studies on applying this Framework in CGIAR Research Programs and Centers.
The ISDC, through its commitment to co-design, sought the support of various stakeholders regarding this Framework’s implementation, including the ex-ante review of CGIAR research proposals in the most recent portfolio.
Dr Ortiz explained how CGIAR’s IAES has continued to do important work with the DGAs and research group leaders to align the ex-ante and ex-post evaluation of the quality of science. The IAES, he sustained, is perfectly positioned to help unify the two debates on ex-ante and ex-post evaluation of quality, given its provision of operational support as Secretariat of the ISDC and the Standing Panel on Impact Assessment as well as its implementation of CGIAR’s multi-year independent evaluation plan approved by the System Council. Finally, he highlighted the importance of the Evaluation Function. In his own words, he explained that evaluation exercises help answer the question of whether or not projects and programs are doing science of excellent quality and why, and whether their results are being applied to achieve the Sustainable Development Goals (SDGs).
Next, Dr Graham Thiele, former leader of the CGIAR CRP on Roots, Tubers and Bananas (RTB), explained the need for independent evaluation from the viewpoint of a CRP leader who has been independently evaluated in 2015 and 2020 to improve the contribution of the RTB Program to CGIAR’s goals of food security for nutrition and health.
He emphasized the importance of the independent evaluation function in CGIAR is key for assessing products and scopes, and guiding the programs. In particular, he remarked, “the Quality of Research for Development framework created by the ISDC is valuable for organizing evaluations and preparing reports”. He also recommended that evaluations be scheduled at times when they can significantly influence decision-making regarding the direction of the programs.
Ibtissem Jouini, Senior Evaluation Manager of the IAES, then presented the various products and materials produced by the IAES available in Spanish and that may be used both within and beyond the CGIAR framework, including the guidelines for evaluation of the quality of science, mentioned by Dr. Ortiz. These guidelines, pioneers in the field of evaluation (along with the Framework and the new Evaluation Policy, available in Spanish soon), offer a framework, criteria, dimensions and methods for the evaluation of the research quality for development in CGIAR, which in the words of Dr Jouini, are adaptable to similar organizations that carry out research for development (R4D). The other products available in Spanish are included in the seminar website.
Dr Pierre Boulanger, Advisor to the ISDC Secretariat, presented the preparation process of key documents on Quality of Research for Development (QoR4D) frameworks and on Comparative Advantage. He introduced the concept of comparative advantage and its benefits for identifying the areas in which the efforts of CGIAR and its partners are most critical to achieving its mission. Highlighting possible sources of comparative advantage, he described the four key steps of ananalyzing comparative advantage at different levels of aggregation in agricultural research for development. Furthermore, Dr Magalí García, Professor Emeritus of Universidad Mayor de San Andrés (Bolivia) and member of the ISDC, explained the bases, philosophy and implementation of the QoR4D reference framework that was adapted as part of proposal reviews for CGIAR research initiatives and that is under revision for its application in the evaluation of the upcoming project portfolio, a central step in implementing the CGIAR 2030 Research and Innovation Strategy.
Questions asked to the four panelists during the Q&A session addressed important issues concerning the way in which program relevance is evaluated, to which panelists responded by highlighting the importance that has been given to relevance as a basic assessment and evaluation criterion in the guide, Framework and policy relating to the quality of science and research for development. In addition, the concern related to the importance of guaranteeing that the results and recommendations of evaluations are produced and reach users for their timely implementation was responded to by pointing out that both direct and indirect feedback mechanisms have been considered. Also, to guarantee their implementation, the recommendations are expected to be formulated at an adequate and realistic level. Another important question discussed at the event has to do with the type of consultation and degree of participation of the target population during the process, acknowledged as critical, and that is made explicit by the use of criteria of legitimacy, relevance, credibility and effectiveness.
The webinar, when concluding, included the words of Roberto La Rovere, independent evaluator and member of the Evaluation Reference Group, who, with his knowledge and experience (12 years in the CGIAR in addition to another 12 in the United Nations system), advocated that innovative initiatives such as those presented in this seminar be guaranteed and sustained over the long term by the CGIAR to avoid retreats towards conventional approaches, processes and practices, something that is not uncommon in the CGIAR.
La Rovere also invited us to reflect on the importance of independent evaluation in a center like CGIAR and on the work that the IAES (Independent Advisory and Evaluation Service) has done, producing evaluations, frameworks, normative guidelines, materials, tools for standardizing evaluation practices in the CGIAR and ensuring objective evaluations of the quality of science and research.
He did not hide the challenges faced by the IAES within the CGIAR, especially the small size of the Evaluation Function in relation to its enormous workload, similar to what he observed in the United Nations system, given the CGIAR budget.
At the same time, he congratulated these types of initiatives, like the seminar, that demonstrate how the Evaluation Function has not only grown but aims to be even more inclusive: “This reaching out to those colleagues who need or prefer to work in Spanish is especially pertinent and allows us to capture the opinions of a greater number of partners and beneficiaries.” At the same time, he expresses his wish that the IAES evaluation in the CGIAR can be more and better understood, appreciated, and considered to be even more useful and, therefore, more used, especially on an internal level, which is the ultimate purpose–also by Spanish speakers.
To close with his words, “We are members of a shared effort and encounter the same challenges in attaining the Sustainable Development Goals (SDGs). Therefore, the universal evaluation principles can help us achieve the same objectives.”
