Abstract
To improve future ISDC external review of CGIAR proposals, ISDC conducted an ex-post analysis of the Initiative review process through a reviewer survey, CGIAR stakeholder consultations, examination of QoR4D reviewer scoring divergence, and Initiative Design Team (IDT) responses to ISDC reviews. The Lessons Learned report details the analysis’ recommendations grouped by: (1) QoR4D and its application to Initiatives, (2) insights for the design of the Initiative proposal template, and (3) areas for deeper consideration by IDTs and senior leadership in developing future proposals.
- For QoR4D and its future application, the 17 review criteria will be revised to improve clarity and remove any redundancy. ISDC designed the criteria to apply to Global Thematic and Regional Integrated Initiatives and learned through this analysis they were less well aligned to Initiatives that were service oriented or focused on underpinning technologies (e.g., Genebanks). In future, the 17 criteria will be modified so they also will be suitable for the assessment of proposals involving different goals. The QoR4D criteria have been revised to explicitly ensure research hypotheses, comparative advantage, and inclusive innovation prompt well considered responses by IDTs.
- For the Initiative proposal templates designed by CGIAR System Office, recommendations included adding a two-page summary that permits proponents to articulate the value proposition, approach, and projected impact in a more persuasive, narrative form. Reorganization of templates for a better “flow” with sufficient space to provide important process detail that currently is only located in appendix sections. Although the goal of keeping proposals short continues to be recommended, the right balance still needs to be continuously reviewed.
- To improve the development of future Initiatives, the analysis recommends that IDTs and senior leadership emphasize consultation and partnerships, comparative advantage, research justification and research questions, as well as the overall cohesion across the portfolio.
The consultations confirmed that the QoR4D criteria and the process used for the ISDC Initiative reviews were beneficial to System Council and assisted in its decision making. The lessons provided in this ex-post analysis will further improve Initiative review processes for independent assessments and will have implications if ISDC participates in future stage-gating of current Initiatives.
Citation
ISDC. 2022. Application of the Quality of Research for Development framework to Initiative reviews: Lessons learned. Rome: Independent Advisory and Evaluation Service.
Publisher
Independent Science for Development Council (ISDC)
