Remote sensing data and technology are rapidly evolving. This brings about new opportunities for using remote sensing products for analyses that have previously relied on other sources of data.
There are many reasons to be enthusiastic about remote sensing methods’ potential to improve impact evaluation in agriculture and natural resource management research. Satellite data can enable researchers to cost-effectively study the impacts of technology or policy innovations at large spatial scales and over long periods of time. In particular, remotely-sensed data can be used to measure both technology adoption and outcomes of interest so long as they produce a visual signal that is observable on the satellite record. However, while remote sensing data can help overcome some challenges commonly associated with ground-based data, researchers should be aware that using remote sensing data involves its own challenges.
In a recent SPIA guideline, we take a closer look at the challenges faced by eleven teams using remote sensing methods for impact evaluation projects.
Below we highlight three thematic issues encountered by these teams:
Reference data - Satellite data can be used to predict measures of technology adoption or outcomes in areas that are costly or difficult to access with ground-based data collection methods. They can also be used to back-cast such measures to other periods of time covered in the satellite record. However, in order for satellite-based measures to be accurate, it is critical to carefully train and validate remote sensing models. Doing so requires reference data, which is high-quality, georeferenced data on adoption or outcomes – often collected using ground-based methods like surveys – which can be linked to the imagery features observed by satellites. The collection of high-quality reference data is often costly, and may have significant implications for project budgets.
Measurement error – Errors in remote sensing products, or model outputs generated from them, can introduce bias into the econometric analyses that are part of impact evaluation. . For example, in one of the examined case studies, researchers sought to understand how the adoption of zero-tillage practices affected the production of smoke from agricultural fires in India. To quantify impacts, the research team planned to use remote sensing data to measure both zero-till adoption and smoke outcomes. However, they quickly realized that smoke from fires obscured satellite images of the land surface. As a result, satellite-based estimates of zero-till adoption were less accurate in areas affected by smoke. Because errors in their adoption measure were correlated with their primary outcome of interest (smoke) – a situation known as non-classical measurement error – estimates produced from the application of standard econometric techniques in this setting would likely have been biased, and could have led the research team to make false conclusions about the nature or magnitude of impacts.
Understanding the data generating process – In assessing the potential for remote sensing data to be used in an impact evaluation, it is important for researchers to think carefully about the specific pathways through which technology adoption is hypothesized to affect outcomes. In econometric terms, these pathways are referred to as data generating processes. The nature of the data generating process has many implications for research design. For example, if a given technology produces impacts of very small spatial scales (e.g, within a field), then it may only be appropriate to use high-resolution satellite data. Alternatively, if there is a delay between when a technology is adopted and when impacts occur, then assessing impacts could require remote-sensing products with sufficient temporal coverage. Perhaps most importantly, the nature of the data generating process also influences the design of reference data sampling strategies. For example, in a setting where close substitutes for a particular technology exist in the marketplace, it would be important to collect high-frequency reference data on adoption to understand if and when agents are switching between similar technologies. In general, thinking carefully about the data generating process early on and as part of the process of research design, can help researchers avoid many challenges associated with using remote sensing data for impact evaluation.
If you are interested in reading more about remote sensing and its use in impact evaluation, make sure to download the SPIA Guideline. The document provides both best practices and lessons learned from teams using remote sensing methods for their impact evaluation projects.
