In the last decade, the validity of data collected through agricultural surveys administered in developing countries has been shaken up by new methodological survey research. Improved, “objective” measurement techniques have become available that offer the prospect of eliminating systematic measurement errors in survey data. Recent studies (here and here) have shown that measurement error matters for agriculture and left unchecked can bias empirical analyses.
Nevertheless, finding and applying such “gold standard” measures should not be the only priority, as pointed out by SPIA chair Prof. Karen Macours in the opening speech at the workshop held at Paris School of Economics back in December entitled “Measurement error and the adoption of agricultural innovations: Building the research agenda”. Effort should also be made to understand both the extent and nature of the measurement error, and its implications both for empirical study but also for our understanding of farmers’ decision-making. Without clear research objectives and a better understanding of the nature of measurement error, we may question whether the validation data (i.e. from our new gold standard approaches) are themselves free from measurement error. As Ermias Betemariam (ICRAF) put it: “Even gold comes in different carats”.
From a micro-perspective, three interlinked questions emerged from various presentations. Tesfamicheal Wossen (IITA) pointed to the importance of testing for presence of measurement error and estimating the extent to which error distorts variables of interest, as this has consequences that go beyond inference to policy. The second question to be asked is “When does the error really matter?” and John Gibson (University of Waikato) provided an overview of cases, such as mean-reversion, binary variables and correlation with treatment or drivers of treatment. Several models have been developed to predict and parametrize the sources of error, but given considerable heterogeneity (farmers preferences, risk aversion, time consistency…) still there is the need to test across different models in new empirical work, says Andrew Dillon (Northwestern University). Hence, the distinction between misreporting by respondents and misperceptions held by them, is an important one – they have very different implications. In order to distinguish these two sources of error there was consensus that farmers self-reported measures should be collected alongside the objective measures. Indeed, in the case of misperception, recovering the information set available to the farmer may help us understand broader trends such as structural transformation and resource misallocation.
The World Bank Living Standard Measurement Study in collaboration with SPIA facilitates the adoption of improved methods in national surveys. Examples include: measures of soil quality, through wet chemistry and spectral reflectance methods applied to soil samples; and crop varietal identification, through DNA fingerprinting of samples of plant tissue. Implementing these objective measurement approaches allows us to capture more accurately not only agricultural productivity but the interrelations between its determinants. Indeed, as discussed at length by Ermias Betemariam (ICRAF), Leah Bevis (Ohio State University) and others, the agricultural outcomes we are interested in – particularly the first-order effects of crop yield and crop nutritional content – are determined by the interaction of Genotype x Environment x Management. Given that seed, soil and management decisions are intrinsically inter-related, this calls for a holistic approach in survey design.
During the workshop, we broke into groups to discuss specific areas of work that we could develop further. One of the interesting discussions focused on how objective measures could, and indeed should, benefit farmers. For example, more accurate measurement could be fed back to farmers in a structured way to inform their own learning processes, and thus help them make better decisions about how to manage their plots.
Overall, the workshop brought together a great mix of talented young economists from CGIAR – many of whom are making substantive contributions to this agenda – alongside some well-established and influential faculty who are doing the same from top-tier universities. Our hope is that this workshop is a just a staging post in a process of new collaborations being formed to deepen our understanding of the structure and implications of measurement error while broadening the range of economic insights generated from the application of new biophysical measurement approaches in agricultural survey data.
