Dr A. Rama Rao is former head of the Indian Council of Agricultural Research (ICAR)’s National Academy of Agricultural Research Management (NAARM), where he served for almost four decades.
On February 27 and 28, 2023, he attended a workshop in Rome, Italy about the CGIAR’s Independent Advisory and Evaluation Service (IAES)’s new set of evaluation guidelines. These build on the CGIAR Independent Science for Development Council (ISDC)’s Quality of Research for Development (QoR4D) Frame of Reference, and provide the framing, criteria, dimensions, and methods for assessing QoR4D – both within CGIAR, and in other like-minded organizations. The hybrid online and in-person event was designed to help practitioners across and beyond the CGIAR system to understand and apply the new guidelines in their own evaluative contexts.
We spoke to Rama Rao to find out more about his experience of the workshop and his aspirations in this arena going forward.
Q: How familiar were you with the ISDC and the Evaluation Function prior to the workshop?
Somewhat familiar, but I work outside of the CGIAR system.
Q: How much experience did you have in evaluating quality of science and QoR4D?
A reasonable amount. During my tenure at ICAR, a fair amount of my time was spent at NAARM, which is ICAR’s think tank as well as a capacity-building institution. Everyone who comes in as a scientist at ICAR gets trained at the academy, and we take care of things like grant criteria and prioritization, and monitoring and evaluation systems – with a lot of focus on innovation and results management.
Q: Were your expectations from the workshop met? Did you have any unanticipated learnings?
I didn't go with any particular expectations. But I certainly found that my Indian experience, which I shared at the workshop, was quite dissimilar to the CGIAR experience. There are a lot of differences in the way we look at things, which I tried to share with them there.
For instance, I don't really know who is the client of a CGIAR institution, but at ICAR we have a clear definition: the farmer is the client. So everything that we do, our impact evaluation teams – which ICAR consults every five years – will assess it from that perspective. If you are a scientist who has done wonders in biotechnology, they are not going to be mesmerized by that. They will be looking at what you have done for the farmer, and what the farmers got from the institution in the last five years. Did they get enough from you? Where are the blockages? Is it the funding, or the manpower, or a governance issue?
At CGIAR, the whole discussion was on the contribution to science, whereas in our case, the impact evaluation teams focus more on what you're giving to the farmers, and contribution to science only makes up about 20-30 percent of the evaluation effort.
Q: What will you take forward into your work? Where/how?
If my director-general asked me ‘how can we go ahead with [the CGIAR guidelines]?’, I would say ‘first, let’s get some first-hand experience from the CGIAR institution’ – from an average mid-level researcher with about 10-15 years of service. I’d like to first see how they find [using the tool], before we jump into applying it here. Including agricultural universities, ours is a big system with 30,000 scientists, so we have to be careful about introducing something new and facing criticism and backlash.
Q: Are there any other ways in which you would like to continue engaging with IAES/Evaluation and CGIAR on this topic, and around the Guidelines? If so, what?
As I said, I’m very curious to hear how the average researcher from CGIAR finds this tool. It’s great to see how much has changed in this space: for the first two decades of my time at ICAR, we didn’t see much change within research management – we used the same formats for proposals, monitoring, and so on. In the last ten years this has become a very dynamic process, and hearing more about people’s experiences [with the tool] over the next couple of years will be very useful.
