What is an evaluation inception report? And why is it important?
Evaluations are process-driven and typically divided into phases (see figure 1, consistent with IEA’s Guidance[1]). During the preparatory phase, a Terms of Reference (ToR) document is drafted and finalized by the commissioning evaluation unit (commissioner), and an evaluation team is selected to conduct the evaluation.
The evaluation team prepares an inception report - a standard key deliverable of an evaluation’s inception phase. The inception report explains the analytical and methodological dimensions, related to its primary objectives and the evaluation questions to be examined with reference to the evaluation criteria. It outlines the evaluation approach and timeline and serves as a conceptual framework for the evaluation design and process, including detail on the methods to be used, specific tools for data and information acquisition, and analysis to be conducted.
Equally important, the purpose of an inception report is to help ensure a shared understanding of the evaluation conduct between the commissioner and the evaluation team, as well as among key stakeholders. It serves as a roadmap and as a guiding document in the evaluation process for all the stakeholders involved including the evaluation team members themselves and the evaluand, steering proper implementation, monitoring, and mutual quality assurance in all its stages.
The inception phase
The inception phase entails an initial review of existent program documentation, related evaluative evidence, and relevant external literature. In terms of process, the inception report and its related communication activities culminate the inception phase.
To situate the inception phase within evaluations commissioned by CAS Secretariat evaluation function, CAS puts together multidisciplinary evaluation teams that are anchored by subject matter experts to specific areas relevant to the evaluation portfolio, and to CGIAR at large. Subject matter experts bring knowledge regarding technical metrics to complement the assessment, literature review, actual and emerging challenges and solutions, and how these can apply to the evaluand’s context. The role of an evaluation team lead, often the lead evaluator, is to co-develop an inception report in line with the ToR and evaluation standards, and to then ensure that a rigorous methodology is followed, and potential biases are timely addressed.
In standard evaluation practice, the development of an inception report is a co-creative journey among the evaluation commissioner, the evaluand (the subject of an evaluation) and the evaluation team, coordinated by the evaluation team leader, and it sets the basis for the final report. Key stakeholders such as program management, reference groups, funders, etc., are consulted during the preparation of the inception report to provide feedback. The lines of inquiry under the relevant evaluation questions are teased out with key stakeholders, who provide validation of the data sources available, and overall learning needs. It is a thorough and highly participative process that ensures reliability, validity, and buy-in.
Elaboration of issues emerging from consultations and background data reviews allow the evaluation team not only to refine the evaluation questions, but also to focus its efforts on the areas where data and information are more needed and pertinent, and/ or to address main limitations. This framing facilitates a comprehensive understanding of the evaluand and its context for all parties, as it is not possible to evaluate a program whose intricacies and complexities are not well delineated.
Quality assurance in the inception phase
In the Quality Assurance (QA) process, inception reports are reviewed and validated by the evaluation commissioner, as well as the reference or advisory group when present, such as the newly established CAS evaluation reference group, and other relevant groups specified by the commissioner. Factual checks are also performed by the management of the evaluand.
As part of QA, an extra focus is devoted by CAS Secretariat to interrogate the evaluation approach and methodology and enhance the evaluation matrix in the inception report. This is done through a consultation on the methodology conducted during the inception phase with the subject matter experts, peer-reviewers, and professional evaluators as well as expert reviewers on cross cutting issues, such as gender, climate change and others.
What does an inception report include?
The inception report structure varies depending on the commissioning organization. The Guidance Note on Evaluation Inception reports developed by the former CGIAR Independent Evaluation Arrangement (IEA), CAS Evaluation function’s predecessor, advises that inception reports begin with i) executive summary, ii) introduction of the purpose and users of the evaluation, and iii) the purpose and structure of the report. This is followed by a section on the background highlighting an analysis of the evaluation context as a basis for the evaluation design, and a section dedicated to the evaluation scope, criteria, and questions detailing the approach and methodology. If there is a deviation from the objectives and questions outlined in the ToR, the report will provide an explicit elaboration and justification thereof. Inception reports also present evaluation team members expertise, describe the evaluation team structure, roles and responsibilities, as well as the evaluation management processes (quality and risk management). A discussion of the ethical considerations, limitations, and constraints, as well as communication plans and dissemination strategies are also detailed in inception reports.
The inception report requirements and structure may be adapted to be consistent with the ToR, as was the case for the twelve 2020 CGIAR Research Program (CRP) reviews. Consistent with the lean approach for 11-week long evaluative reviews, inception notes replaced full-fledged inception reports, dropping such elements as executive summaries and external reviews. In the interest of time, they were also not published on CAS site and instead presented in the methodology sections of the Annexes to the CRP reviews.
An inception report is not meant to be a conceptual straitjacket, the evaluation methodology and work plan should be adaptive and flexible enough to allow for new and emerging issues throughout the evaluation. The inception phase is followed by the data collection or inquiry phase in line with the evaluation framework detailed in the inception report.
Some examples to share
The inception reports of the evaluation of the CGIAR Big Data in Agriculture and the Excellence in Breeding Platforms are recent cases-in-point of what evaluation inception reports in CAS Secretariat entail. Other inception reports from the Advisory Services Secretariat and the former Independent Evaluation Arrangement (IEA) are available here.
The inception process provides framing, stakeholders mapping, and guides the preparation for further stakeholder engagements. This work ensures inclusiveness of the process while informing the refinement of the evaluation design—evaluation questions, methodological and analytical approach—as well as developing appropriate sampling, tools for data collection, and stakeholder engagement to ensure a robust evaluation via the evaluation framework or matrix. The evaluation team in close concert with the commissioner also assesses the evaluation risks and develops appropriate mitigating actions.
