View the original post on CGIAR.org here.
More than ever before, remote evaluations and evaluation approaches that require a rapid and cost-effective assessment of research for development (R4D) interventions are being sought. Such approaches, while not covering the full gamut of evaluation criteria or in-depth exploration of cross-cutting themes, have the potential to bring top-level evidence to decision-makers in a short turnaround time.
The CGIAR System Council through its Strategic Information, Monitoring & Evaluation Committee (SIMEC) requested reviews of the CGIAR Research Programs (CRP) with an accountability-focused mandate. In response, in 2020, the CAS Secretariat Evaluation Function coordinated the independent reviews of each of the 12 CRPs. The reviews were desk-based and utilized mixed-methods in assessing the extent to which from 2017 through 2019 CRPs delivered quality of science (QoS), effectiveness in relation to their Theories of Change, and evidence of future relevance and orientation. Each of the CRP reviews was conducted by a duo comprising a Senior Evaluator and a Subject Matter Expert and their work was independently peer-reviewed. The reports were then released from July 2020 to January 2021 in four batches.
What are the high-level findings from the reviews?
The CRP 2020 reviews generated insights about the research contexts and programs of work with key gains reported across both agri-food systems and integrating CRPs. As to what works, lessons to be learned for One CGIAR were plentiful. The lessons learned indicated that tailoring research to the needs of next-stage users helps achieve outcomes. Moreover, South-South collaboration has strong potential as a vehicle for delivering valuable outcomes. Strategic concepts such as the Roots Tubers and Banana (RTB) Program’s "golden eggs" which has broader applicability, as well as Agriculture for Nutrition and Health (A4NH) Program’s country focus, to name but a few, have positioned CRPs for future effectiveness. A snapshot of some of the selected findings and recommendations is presented in Table 1.
Table 1: Summary on Strengths and Lessons Learned from the CRP 2020 Reviews
| CRP | Key Strength | Lesson Learned |
| Integrating CRPs |
| Agriculture for Nutrition and Health (A4NH) | Substantial progress in farm households’ adoption of biofortified varieties. | The country focus of A4NH’ is useful for better tailoring of research to needs. |
| Climate Change, Agriculture and Food Security (CCAFS) | Substantial evidence of testing of technical and institutional innovations that have catalyzed climate-smart agriculture on the ground. | Facilitation of South-South engagement can lead to unpredictable but highly valuable outcomes. |
| Policies, Institutions and Markets (PIM) | PIM informed policymaking at the global, national, and sub-national level, and to a lesser extent at the regional levels. | The capacity of the communication teams that provide communication and outreach support is vital in this type of programming. |
| Water, Land and Ecosystems (WLE) | Approach for co-creating integrated, field-tested solutions at local, landscape and national scales with local implementation partners. | To achieve outcomes requires a timespan beyond the timeframes of individual bilateral projects or even a CRP phase. |
| Agri-Food Systems CRPs |
| Grains, Legumes and Dryland Cereals (GLDC) | Seed systems activities have already achieved impact at scale. | Funding is required for the system- transformative approaches, i.e. scaling out innovations; to understand scaling models, market preferences, aspirations, drivers behind farmers’ strategies. The core science should accompany sufficient funding. |
| Roots, Tubers and Bananas (RTB) | The unique structure of RTB enabled cross-crop and cross-center collaborative research and learning: cross-cutting clusters feature in all Flagship Programs (FPs) while in the Flagship 5 (Improved Livelihoods at Scale) all clusters were cross-cutting. | Golden Eggs (collective knowledge assets) position RTB to play an important role in further contributing to the SDGs |
| FISH | Effective in impacting fish and aquaculture policies. | The poor are not always able to adopt the most effective new technologies. |
| LIVESTOCK | The CRP demonstrates a high level of research collaboration with an appropriate range of countries. | Delivering science alone is not sufficient. It is essential to ensure sufficient resources, both financial and human, to allow science to deliver on development objectives. |
| Forests, Trees and Agroforestry (FTA) | FTA’s governance and management arrangements have strengthened it as a collaborative global program, and the current setup represents a good practice example for CRPs. | New programs require a warm-up phase to account for slow-moving variables such as the building of trust and effective institutional linkages. |
| RICE | Estimated impact of RICE is on more than 2 million farmers during 2017–19 (considered under the legacy impact of around 10.4 million farmers over the Global Rice Science Partnership [GRiSP] period). | Potential scope for improved collaboration: the review encouraged cross-CGIAR resource sharing on path-breaking technologies in breeding and agronomy in the lab and in the field. |
| MAIZE | Scientific and technology inputs and approaches are generally of high quality, sometimes the world’s best, i.e. the case of doubled haploid (DH) production. | Knowledge dissemination may be too heavily focused on scientific publications and too little on tailored communication targeting next-stage(s) users. |
| WHEAT | WHEAT and its flagships tailor outputs to well-defined end users. | Exchange of material and information is paramount for a competitive wheat industry, which in WHEAT’s target countries is largely in the public sector. |
Another high-level finding was that QoS is unique as a key evaluation criterion and of particular interest to CGIAR stakeholders. Related evidence and lessons across CRPs are well captured in the CAS blog post, “Bibliometrics in Assessing QoS”. Other process-related learnings are captured in the blog posts “Evaluative Reviews - A Streamlined Approach to Accountability” and “Reflection on the CRP Reviews” as well.
What are the cross-cutting implications?
For the cross-cutting themes, at the CRP level, the recurring refrain called for effective partnership delivery mechanisms and deeper engagement with local partners. Furthermore, calls for improvements in the strategy for capacity development and youth themes were made. While gains were recorded in the implementation of the gender strategy and mainstreaming gender, opportunities for further enhancing gender integration in research and gender balance in the workplace were noted. The reviews also highlighted the need for a better reporting of the cross-cutting themes. A cross-section of some findings and recommendations from the CRP 2020 reviews for cross-cutting themes are consolidated in Table 2.
Table 2: Summary on a Cross-Section of Strengths and Recommendations for Cross-Cutting Issues from the CRP 2020 Reviews
| Cross-Cutting Theme | Key Strength | Recommendation |
| Youth | Youth targeting in research produced some notable outputs, and some marginal achievements were recorded in youth programming. | Consider framing future youth, gender and social inclusion within an understanding of intersectionality. Strategic and contextually appropriate prioritization of youth agenda should also be considered. |
| Gender | Gender integration in research has improved over time and added value to research outputs for end-users. Gender work has significantly increased the participation of women. Gender diversity in the workplace is gaining pace and can be improved. | Set up programs to support and mentor underrepresented groups and develop a cadre to maximize their value. If an appropriate balance of gender, age, and ethnicity among researchers and staff is a CGIAR aim, then data on these parameters should be collected and publicly available. CGIAR should consider reinstating the requirement to reserve 10 percent of the budget for gender-related work, following the existence of a viable and effective gender strategy before receipt of funding. The inclusion of gender focal points within each FP (or whatever form new CRPs take) should be maintained. |
| Climate Change | Climate change is mainstreamed across the CRPs. The research agenda and science are informed by climate change, which has led to the production of high-quality research outputs and innovations related to climate change. | Integrate the climate change and nutrition agendas more closely. Strengthen and integrate theory-based work in planning, monitoring and evaluation for systematic learning to inform strategic decision making. |
| Capacity Development | Capacity development is targeted to diverse audiences, experiences and nationalities (NARS, international partners, senior and junior scientists et al.,) with a wide range of activities, such as short trainings, M.Sc, PhD (including providing scholarships), Post Doc, mentoring scientists, and NARS, conferences and institutional support. | Conduct a strategic review of capacity development to provide clear guidance on its aims and relationship with other key actors, such as national universities and national agricultural research systems (NARSs), in achieving these aims. Targets for capacity development should be reviewed, for clarity on measuring progress, and respond to national-level needs. Formulate a capacity development strategy for junior and emerging scientists in CGIAR. This may include launching a program to develop the scientific writing capacity of emerging researchers who publish fewer than two papers in a three-year cycle; and diversification grants to recruit young, female and developing country interns with complementary skills and qualifications. |
| Partnerships | Partner agreements and annual audits in CRP Phase II helped ensure that partner institutions met CGIAR standards. Partnering with next-stage users is the modus operandi for some CRPs. | Develop a System-wide suite of practical and ethical guidelines to promote collaboration with local partners (NGOs, NARS, etc.) and local communities in project design and implementation. Partner with and involve medium and large enterprises in technology transfer to the poor to increase the likelihood of successful dissemination with benefits across the entire value chain. |
What is next?
The 2021 meta-review, synthesis of evidence and lessons learned from evaluation across the CGIAR System and CRP evaluations is underway. Commissioned and coordinated by the CAS Evaluation Function, the CRP 2021 Synthesis pools evaluative evidence spanning CRP phases I and II, layered with evaluations of cross-cutting and key themes. The objective of the CRP 2021 Synthesis is to crystallize findings, unpack trends, identify the patterns around priority topics, bring to the fore evidence gaps and provide evidence-based recommendations for One CGIAR. The CRP 2021 Synthesis will be delivered by a team of well-respected experts with deep knowledge related to CGIAR’s portfolio.
Learnings from the CRP 2021 Synthesis could be a strong entry point to evidence-fit the new CGIAR Research Portfolio and emerging investment plan, and to deliver and drive the One CGIAR Research Strategy while navigating complexity and uncertainty to achieve impact. The newly released CAS blog “Evaluation Syntheses for One CGIAR: Taking Stock of a Decade of Investment in Agricultural Research” provides some insight into the forthcoming Synthesis, which will be released in June 2021.
The CRP 2021 Synthesis Team
- Mark Holderness, Team Leader, former Exec. Secretary, Global Forum on Agricultural Research and Innovation (GFAR)
- Julie Howard, Co-author, Agricultural Economist, former USAID Bureau for Food Security (BFS) Chief Scientist, subject-matter expert in research, policy and capacity development
- Debbie Templeton, SME Agricultural Economist, CGIAR & Australian Centre for International Agricultural Research (ACIAR), in-depth experience with research evaluation
- Carlos Iglesias, Professor of Horticulture, NCSU Director, Plant Breeding Consortium
- Nigel Maxted, Professor of Plant Genetic Conservation, University of Birmingham
- David Molden, Former Director General, ICIMOD & International Water Management Institute (IWMI) Research Director, expert in water and natural resources management
- Ibtissem Jouini, Senior Evaluator, EvalChange Network
