CGIAR Advisory Services facilitates expert independent advice to CGIAR as it navigates complex waters by supporting knowledge systems that bridge subject matter expert assessment, crystallized through evaluation exercises, and investor decisions.
The Purpose of Evaluation Syntheses
Science helps us understand processes through analytical inquiry (analysis). However, to make sense of the world and to make scientific knowledge relevant, we need to recombine this knowledge via deductive reasoning (synthesis). This is particularly important if the goal is to learn from the past.
Evaluation syntheses offer a means to understand larger patterns essential to navigate research for development outcomes. Syntheses allow us to look back at what was done and consider our journey so far, the boat’s wake. They also allow us to look ahead, registering the waters in front of us, drawing on learned experiences for forthcoming project design by aggregating evidence (USAID, 2019). Agencies use evaluation syntheses to facilitate collective reflection on past experiences to guide future actions, among other purposes (IFAD).
The foundation of any science organization, including the CGIAR, is its disciplinary expertise. However, simply providing a synthesis of past experiences rarely improves decision-making per se as science has a tendency to create ‘islands of knowledge in a sea of ignorance’. Bridging these ‘islands of knowledge' is essential to address the existential problems of our planet that go beyond the realm of any discipline, such as poverty, hunger, and social justice. Thoughtful synthesis can build such bridges, particularly at a time when the organization is undergoing transformational change.
Building Upon the 2016 Synthesis and Lessons Learned From 15 CRP Evaluations: Patterns Emerging 5 Years Later
The CGIAR Advisory Services’ (CAS) Evaluation Function is in the midst of conducting a meta-review of findings from 47 reviews and evaluations, to synthesize a decade of CGIAR investments—while looking forward and actively informing One CGIAR design. CGIAR used evaluation synthesis in 2016 for reflection and learning, and CAS is continuing in this tradition, standing on the shoulders of this earlier 2016 work (Birner & Byerlee, 2016).
The 2016 Synthesis touched on key themes of quality of science; decision-making (management, governance, and priority setting); progress along impact pathways described in Theories of Change; partners’ role in the research for development agenda; and thematic observation related to gendered research and capacity development. The 2021 Synthesis is building on these themes.
We can already glean a preliminary understanding of how lessons were addressed after the 2016 work. For instance, the CRP Phase I Synthesis called for greater emphasis to address gender in the workplace and use CGIAR’s gender-disaggregated research data. Since then, CGIAR launched the Gender Platform and invested in specific staff Gender, Diversity and Inclusion measures, evidencing a strong commitment to improve; these investments notwithstanding, the CRP 2020 Reviews reveal continued and pronounced under-representation of female scientists among top published researchers in CGIAR. Likewise, the 2016 Synthesis recommended more strategies for mentoring junior scientists. This need remains as strong today as it was in 2016.
points to a wealth of past evaluation
evidence. Various types of evaluations and
reviews pertaining to a decade of CGIAR
investment will set the course of travel.
The Synthesis also highlighted the imperative for better integration of social science, beyond economics, across the CRPs, underscored in the CRP 2020 Reviews. Further, the 2016 Synthesis pointed towards a need for more CGIAR efforts to adapt the Theory of Change (TOC) for R4D settings. The CRP 2020 Reviews acknowledged good progress on TOC design but highlighted an urgent need to better use the TOC during project implementation. The Birner and Byerlee synthesis stressed that research partners in the South must be engaged more fully from the initial research design to co-authored publications in order to overcome the unequal power dynamics inherent in these relationships.
The ongoing 2021 Synthesis brings together professional evaluators with subject matter experts to review systematically CGIAR progress over a decade. This will be done along core themes agreed with stakeholder groups. Here, we have only given a cursory summary. Meanwhile, as One CGIAR portfolio development continues full steam ahead, CAS is facilitating interactions and transfer of the learning from this work to the Independent Science Development Council (ISDC).
Bridging Islands of Knowledge
ISDC provides independent expert advice to CGIAR investors and key partners of CGIAR System Council with reviews of the Investment Plan and each Research Initiative when these are available.
In conducting its work, ISDC formulates advice with an overt aim to bridge communities of experts and communities of investors, who may sometimes feel they are on separate islands (Meinke et al, 2006). Just as CGIAR research at its best bridges the needs, interests, and perspectives of societal actors with scientists through a process of co-production, ISDC bridges the needs of investors in governance bodies with expert opinions produced by independent authorities. Similar to CGIAR researchers, ISDC assesses its own advice through the Quality of Research for Development frame of reference, which has been inspired by the literature on bridging knowledge systems: Is it relevant? Is it legitimate? Does it possess scientific credibility? Is it effective? (ISPC 2017; ISDC 2020). Many authors have elaborated on co-production and knowledge systems for sustainability and the importance of effective science advising at the interface between communities of experts and communities of decision-makers (Clark 2020, Matson 2016, Jasanoff 2004, Cash 2003, among others). Sustainability challenges benefit from knowledge co-production and, as Norstrom (2020) suggests, co-production succeeds best when a principled approach is adopted.
CAS is developing the 2021 Synthesis to support learning across CGIAR and notably, in its new portfolio. ISDC will review the portfolio and new projects on behalf of the system governance. Therefore, CAS Evaluation Function is packaging knowledge products for ISDC that draw learning from evaluation and synthesis work, as ISDC prepares to review the new projects. ISDC is interrogating evaluation findings and recommendations, ready to cross-reference these with forthcoming project review. The advice ISDC provides to CGIAR investors is intended to bridge subject matter expert opinion captured in evaluations with the needs of the investor community.
A New Evaluative Synthesis for One CGIAR
Looking at the evidence emerging from the CRP 2020 Reviews against some of the major threads of the 2016 Synthesis reveals progress on many fronts; it also highlights several systemic concerns. CGIAR has the opportunity to learn from insights and patterns revealed through evaluation syntheses. The upcoming 2021 Synthesis formulated by external experts will use scores of reviews and evaluations assessing 10 years of investment, to draw out key learnings; the product will be framed around the architecture of the CGIAR 2030 Research and Innovation Strategy. ISDC will help to bridge the views of experts with the views of investors, connecting islands of knowledge to support sound co-production between investors and the research community. CAS will continue to support CGIAR to look back at its wake, and tap into insights emerging from a decade of progress while bringing on board lessons as we cast our eyes to the open waters in front of us.
The individual CRP reviews are available here; the discussion draft from the 2021 Synthesis of CRP Evaluations will be available in June 2021. The knowledge products work with ISDC is an ongoing activity. The CGIAR 2030 Research & Innovation Strategy is available here.
Birner, R., & Byerlee D. (2016). Synthesis and Lessons Learned from 15 CRP Evaluations. Summary Report. Rome: Independent Evaluation Arrangement (IEA) of CGIAR.
Cash, D.W., Clark, W.C., Alcock, F., Dickson, N.M., Eckley, N., Guston, D., Jager, J. and Mitchell, R.B. (2003). Knowledge systems for sustainable development. PNAS, 100(14), 8086–8091.
Clark, W.C., van Kerkhoff, L., Lebel, L., and Gallopin, G. (2016). Crafting usable knowledge for sustainable development. PNAS, 113(17), 4570–4578.
Independent Science for Development Council (ISDC). (2020). Technical Note: Quality of Research for Development in the CGIAR Context. Rome: Independent Science Development Council.
Independent Office of Evaluation, IFAD. (2016, May 14). Evaluation synthesis reports. https://www.ifad.org/fr/web/ioe/evaluation/asset/39988740
Jasanoff, S. ed. (2004). States of Knowledge: The coproduction of science and social order. Routledge Press.
Matson, P., Clark, W.C., Andersson, K. (2016). Pursuing Sustainability: A Guide to the Science and Practice. Princeton University Press.
Meinke, H., Nelson, R., Kokic, P., Stone, R., Selvaraju, R., Baethgen, W. (2006). Actionable climate knowledge: From analysis to synthesis. Climate Research, 33, 101–110.
Norstrom, A.V., et al. Principles for Knowledge Co-Production in Sustainability Research. (2020). Nature Sustainability, 3, 182-190.
USAID. (2019, May 7). Discussion Note: Making Evidence Accessible through Evaluation Synthesis. https://usaidlearninglab.org/sites/default/files/resource/files/evaluat…
