Demonstrating the value of agricultural research for CGIAR has never been more important. As funding environments grow more competitive and accountability expectations rise, return on investment (ROI) has become a common, and reasonable, request. Before asking "What is the ROI?", two prior questions need answering: is ROI the appropriate metric for this type of research/innovation? And if it is, do we have the data to calculate it credibly?
Stage 1: Is ROI an appropriate metric?
ROI works when benefits are tangible, monetizable, and attributable to specific research investments within a defined timeframe. CGIAR’s portfolio has greatly diversified from primarily technological innovations like improved crop varieties or mechanical technologies to now include institutional and policy innovations, partnerships with private sector, and other global public goods like genebanks.
This diversified portfolio produces many intangible benefits that are difficult to quantify. For instance, the development and dissemination of certain innovations — including axial flow pumps in Bangladesh and sustainable financng mechanisms to derisk agricultural investments — have only been possible through the engagement of the private sector, which has boosted both the reach and impacts of these innovations. However, it is challenging to have accurate information about the costs and benefits accruing to the private sector.
In a similar vein, CGIAR genebanks continue to play a major role in generating agricultural solutions for the developing world. However, estimating benefits of developing improved seed varieties across the world and for posterity continues to be a challenging task. Finally, when it comes to institutional and policy innovations, isolating CGIAR's contribution from other influences including political priorities, domestic institutions, parallel research, and donor projects is genuinely complex. This all implies that the CGIAR portfolio, in its entire breadth, is simply not "ROI-Appropriate".
In other words: some parts of the portfolio lend themselves to the ROI framework and others do not.
Stage 2: Is the innovation "ROI-Feasible"?
Even if many CGIAR innovations are ROI-Appropriate, do we have access to all information required to estimate ROI for them? As discussed in the recent SPIA ROI study, three sources of data are necessary to feasibly execute credible ROI estimation:
1) Cost data: Costs of developing and disseminating a particular innovation should be assembled based on a clear R&D pathway to the distinct innovation and role of CGIAR in this process. Although cost data could exist, precisely mapping CGIAR’s role through the project/program funding pathways (especially when multiple donors and partners are involved) poses challenges in apportioning the share for each of them. Establishing CGIAR’s investment cost for a particular innovation can become difficult.
2) Reach data: Rigorous evidence of the reach or adoption of each innovation is required. Depending on the nature of innovation, this adoption may be measured at different levels (household, community, subnational) and may not be converted easily to figures at scale (i.e. national). Furthermore, adoption studies that rely on nationally representative samples are scarce, ruling out several innovations and geographies where the CGIAR operates.
3) Impact data: Rigorous estimates of the impacts of the innovation (both short- and long-term), and its potential spillover effects are essential. These impact estimates should be generated in settings that are as close to the real world as possible, and when possible, in areas where reliable adoption data has been collected. Issues of external validity of the impact estimates should be carefully examined.
How SPIA applied this framework
SPIA applied this two-stage framework to 14 of CGIAR's innovation-level successes, which emerged during prior country-level research. Out of these ROI-Appropriate innovations, only four were deemed ROI-Feasible. This highlights major data infrastructure gaps and the need for: i) better cost documentation from program inception, ii) better adoption measurement integrated into dissemination planning, and iii) impact evaluation designed ex-ante rather than retrofitted after scaling.
Even for these four innovations, estimation revealed a further layer of complexity. Every ROI calculation requires assumptions about cost allocation, reach, and impact extrapolation. These are unavoidable, but they can be handled transparently.
We'll cover SPIA's approach to these assumptions — and what that means for interpreting the results — in Part 2 of this blog series. To learn about these concepts in more depth, read SPIA's report "Estimating the Returns on Investment for Select CGIAR Innovations."
