Evaluations of Science Groups: Synthesis on Quality of Science

Evaluating the quality of science, research, and innovation is an integral part of achieving and measuring the process and progress of the SDGs (EvalSDGs Insight #18). The CGIAR’s Guidelines on Evaluating QoS (also Spanish) elevated QoS as a cross-cutting criterion within the six OECD/DAC criteria for framing evaluative judgments and as a stand-alone evaluation criterion with four dimensions (design, inputs, processes, and outputs). The QOS elements of credibility and legitimacy are embedded in the QoS criterion. The value and utility of the criterion were demonstrated through a summary of the Evaluative Evidence from the CGIAR Portfolio on Quality of Science (2020-2023 reviews and evaluations of CGIAR Research Programs (CRPs) and Platforms. The learning informed the design of the Science Group Evaluations (Terms of Reference) and data collection methods to answer two overarching questions:

• To what extent do the SGs' management processes ensure the QoS, including credibility, legitimacy, relevance to next-stage users, and potential effectiveness, of the research and operations?
• In what ways are the research outputs by the SGs of high quality and influential? 

In 2024, evidence on the QoS inquiry from the evaluations of the Science Groups (SGs): Genetic Innovation (GI), Resilient Agri-food Systems (RAFS) and System Transformation (ST) was synthesized across the four QoS dimensions and complemented by 11 case studies and in-depth analysesan online survey, key informant interviews, and Altmetric Attention Scores (AAS) to assess influence. Findings were analyzed against the OECD/DAC criteria of relevance, coherence, efficiency, and effectiveness despite initial challenges in applying QoS guidelines. Intended for evaluators, policymakers, and researchers involved in CGIAR and related initiatives, the CGIAR Science Group Evaluations: Brief on Quality of Science asses QoS through the four key dimensions earlier mentioned. 

The report includes valuable lessons from each Science Group and outlines recommended actions for improving QoS within the 2025–2030 CGIAR Portfolio, emphasizing stronger alignment, stakeholder engagement, and interdisciplinary collaboration.

To dive deeper into the findings and recommendations:

• Download the full report: Access detailed insights, lessons learned, and actionable steps to drive high-quality research and innovation aligned with the SDGs.
• Visit the Science Group evaluations portal: Explore comprehensive evaluations and reports on Quality of Science.
• Explore the QoS Resource Hub: Access tools, guidelines, and resources to support research excellence and impactful innovation.

CGIAR Independent Advisory and Evaluation Service (IAES). (2024). Evaluations of Science Groups: Synthesis on Quality of Science. Rome: IAES Evaluation Function.