Qualitative research is often regarded as being less objective, and hence less rigorous, than quantitative research. The latter ensures objectivity by standardizing both testing and analytical procedures (Riessman and Given 2008). But the tools of the qualitative researcher - e.g., semi-structured interviews where questions are deliberately open-ended, or observations of natural social interactions, appear at first glance to be the opposite of standardization. Similarly, the analytical procedures in the qualitative tradition, which emphasize interpretation by the researcher, are questioned for being subjective, biased, and unscientific.
Judged by these criteria, we would not be able to study aspects of social life that do not lend themselves to numerical measurement or reproducible experimental research design. Thankfully, however, the practice of using qualitative methods for social inquiry is robust, ongoing, and yielding valuable knowledge. As more researchers commit to mixed-methods studies that combine qualitative with quantitative approaches, it is important to understand the fundamental differences between the two traditions to move forward in concert. One such fault line is on the different ways of approaching “objectivity”. This blog addresses the question - what does objectivity mean to qualitative researchers whose methods do not rely on quantitative measurement and a controlled environment of experimentation?
Let us start by noting that “qualitative research” is not a singular research tradition, but it encompasses a diverse range of assumptions, methods, and practices. The focus in this blog will be on interpretive social inquiry which is central to scholarship across multiple disciplines including anthropology, history, and sociology. This will be a two-part blog. This first part will briefly review three major assumptions shared by the practitioners of the interpretive mode of inquiry that has implications for how “objectivity” is understood. A second companion blog in the coming weeks will feature strategies and techniques used by interpretive qualitative researchers that follow from the assumptions outlined below.
Assumptions on ontology: What is the nature of social reality? Qualitative researchers start with the assumption that reality is socially constructed by actors who co-produce shared social understandings.
A fundamental assumption when qualitative researchers go to the field is that we do not know what we do not know. Of course, this does not mean that the researchers are bereft of the knowledge of the literature, theories, or hypotheses. What it does mean is that we do not begin the research process with a set of predesigned questions with fixed response categories with the goal of achieving standardization of research procedures. Rather than create a separation between the researcher and the researched through the use of standardized questions in the aid of objectivity, qualitative researchers aim to increase interaction and engagement and remain flexible in their questions, with the aim of achieving cognitive empathy and arriving at a shared understanding of the perspectives, beliefs, and motivations of the subjects of the research (Burawoy 1998; Small and Colarco 2022). The methods of the qualitative researcher - interviews, group discussions and observations - are acts of co-production that reveal the fullness of the social world inhabited by the subjects of the research.
Within the CGIAR, there is a long history of qualitative research that has aimed to bridge the gap between expert knowledge created in lab conditions and the social realities on the field that are far more complex than biophysical parameters. Writing about the use of qualitative assessments for a project that was a long-term partnership between IRRI and NARES, Zolvinski (2008) gives us insights into why research that is co-produced is important and the skill set required to initiate a transfer of knowledge between two sets of experts - scientists and farmers.
The process of farmer participatory research requires scientists to welcome farmers as partners in developing new technologies. In that way, these methods bring together two sorts of “experts” into the research process. One group of experts are the scientists, whose knowledge grounded in empirical experimental research can contribute new technical procedures for growing and managing crops. The other group of experts are the farmers who have an intimate knowledge of the day-to-day conditions under which the technologies will be used. Channeling both experts toward a common cause is a relationship-building exercise that requires the development of rapport, trust, and mutual respect between scientists and farmers, and a realization that both parties bring valuable knowledge into the research process. Relationship-building may require scientists to develop a new skill set to complement their technical qualifications. Scientists must develop listening skills that are sensitive to farmers' concerns, and they must be willing to exercise patience as technologies are modified in the iterative process of on-farm experimentation. Of course, building good relationships requires research scientists to step out of their comfort zones to experience the sort of conditions that farmers face in rural areas. (Page 4-5)
Oftentimes, the skills required to conduct research in a lab or using quantitative approaches is considered “technical” while skills such as “listening”, “rapport building”, “sensitivity”, or “relationship building” are dismissed as “soft skills” that are not essential for a researcher. This distinction of skill sets, in fact, has implications for achieving objectivity.
Let’s consider an example from Zolvinski’s qualitative assessments to understand the adoption of new technologies. While farmers care very much about the same issues that experts do, e.g., rice varieties that are of shorter duration and are resistant to droughts or salt salinity, they care equally about taste, appearance, and suitability to local diets. Farmers worry about rice varieties that “digest too easily”, do not give a “full” feeling in the stomach or are not suitable for making traditional dishes. Factors such as particularities of taste or appearance emerge naturally and organically in interviews because they are important to farmers and because qualitative researchers are trained to build rapport, be sensitive and listen. Too often, it is tempting to blame farmers who care about such preferences as being “backward”, “less progressive,” or “noncooperative” which unsurprisingly results in a world “littered with technologies that started out as good ideas on the research station, but simply did not catch on with farmers.” (Zolvinski, Page 5)
What does research that is co-produced mean for objectivity? Qualitative researchers do not consider it a loss of objectivity when co-production is used to achieve shared understanding. Indeed, building relationships with farmers, considering them as experts, and taking their views into account is a means of maximizing objectivity. The aim of “maximizing objectivity” is discussed further in the second assumption below.
Assumptions on epistemology: How do we know what we know? Qualitative researchers in the interpretive tradition believe that facts are always filtered through our perspectives, interests, experiences, values, and beliefs.
Interpretive qualitative researchers start with the assumption that value neutrality in the social world is a misguided and unattainable goal and commit to a notion of reality and ‘facts’ as being shaped by the researchers’ beliefs and interests. Writing in the inaugural issue of Feminist Economics in 1995, the philosopher of science, Sandra Harding, argued that an uncritical reliance on value neutral methods have too often resulted in the production of scientific knowledge that masks its own particularity and subjectivity as universal and objective. She pointed to the long history of social science that have generalized from Euro American white male middle class historical cultural experience for theoretical claims about all of humanity. Rather than providing us with objective and accurate descriptions,“[T]he neutrality ideal has weakened standards for maximizing objectivity, for it precludes actively seeking socially marginalized locations or vantage points” (Harding, p. 11). Harding distinguished between value neutral (weak objectivity) and value aware (strong objectivity). The former is a methodologically naïve stance that precludes an effective examination of how one’s beliefs and values impact research, whereas the latter recognizes the partiality of the observer and actively incorporates multiple vantage points as a methodological principle.
Qualitative researchers believe that the researcher’s knowledge is partial, and the researcher’s values are an important aspect of knowledge production. If the researcher is oriented towards believing that knowledge is partial and ‘situated’, then being “objective” is recalibrated away from a notion of a singular truth about an object. Instead, the researcher aims to understand a complex reality in which an object presents itself to the world differently depending on the position of the observer. The researcher’s task is not to neutralize these different vantage points but to maximize them to get a fuller picture.
Qualitative approaches in agricultural research aim to bolster this type of objectivity by recognizing the importance of the vantage points of the farmers, by understanding that within the group of “farmers” there are several different vantage points, and last but not the least, the researchers also have their own, separate, and distinct vantage points.
Does the assumption of always seeing reality in light of the researcher’s values condemn the research output to relativism, i.e., that researchers can draw no generalizable conclusions? No, in fact the kind of objectivity that interpretive qualitative researchers aspire to - which Harding calls strong objectivity - is to be aware of our values, experiences, and backgrounds, and accounting for their effects in our research process and findings. The scholarly and methodological training, and the social background, experience and identity of the researcher is deemed relevant and examined as both a potential bias and a resource. In a subsequent blog, we will examine the different strategies employed by qualitative researchers to acknowledge, confront, and scrutinize - where and how are a researcher’s values, experiences and beliefs going to be a bias? Where is it going to be a resource?
Assumptions on data and data analysis. Interpretive qualitative researchers believe that data is not inert, something to be “found”; it is co-produced through the interpretation process. Interpretation is measurement.
The researcher is an instrument of the research because the researcher does the interpretation. We often hear that qualitative research is “biased” because the researcher does not control the data collection process, that it is possible that every interview or discussion can yield a different or new viewpoint, and therefore, how can researchers make sense of such a reality that seems messy, arbitrary, and subjective? For the qualitative researcher, the series of open-ended discussion reveals a shared reality, and this shared reality has identifiable patterns. The patterns emerge and are identified by understanding and interpreting the data.
For example, writing about the development of the project-level Women’s Empowerment in Agriculture Index (pro-WEAI), Meinzen-Dick and colleagues highlight how qualitative research elicited diverse, context-specific understandings of women’s empowerment.
In the common definition of empowerment in the literature and development programming, the ability to decide is commonly understood as individual or independent choice. Our findings show that interpretations that privilege individual decision-making fail to capture the relational aspects and mutual interdependence as well as the multiple modalities of agency that hold significance in different contexts and places…. There is a seeming contradiction when empowered women were described as “submissive.” Following such social norms and ideals of femininity may seem contrary to Western notions of empowerment, but using and even manipulating social norms can be understood as a form of agency… In the pro-WEAI, setting the thresholds for decision-making and ownership of resources to count joint decision-making as empowerment, and the addition of indicators for IPV and intrahousehold harmony, move beyond the notion of women’s empowerment as an individual process to one that considers her in concert with her context.
The researchers showed that “empowerment” is not value neutral. If the researchers had operationalized empowerment only as an ability to make individual or independent choice, it would have been an example of weak objectivity because: (a) individual choice would have been valued and prioritized while mutual interdependence or the importance of intrahousehold harmony would have been ignored, resulting in a measure of empowerment that is western-centric, (b) the western-centric definitions would have been the “objective” yardstick against which all human experience of agency would be measured, (c) the results of the research using such measures would incorrectly claim universality because the particular context within which the definitions arose would be masked and finally (d) the experiences of the respondents who narrated alternate understandings of agency would not only be ignored but worse, be mislabeled as “submissive”.
By being aware of their own positionality as researchers who may or may not share the women’s definitions of empowerment, the researchers heard and recognized other lived experiences of agency. They accounted for their own biases and turned value-awareness into a strength. And the researchers interpreted the data, found identifiable patterns, and concluded that they must include the vantage point of the researched when designing measures. The emergence of a shared understanding through a process of co-production thereby led to “strong objectivity”. It is important to note here that the researchers did not merely retrieve and passively receive the respondents’ understanding of empowerment and agency. Strong objectivity and co-production of knowledge occurs when the researcher’s own horizon of understanding is enlarged to see the different angles through which to understand the lived experience of empowerment.
In conclusion, when researchers’ focus shifts to understanding nuance and complexity, the conversation rightfully shifts from “is this data objective?” to “what constitutes good quality data?” A previous blog touched on the criteria that can be used to judge qualitative research. A forthcoming blog will build on the three elements of objectivity introduced here to highlight strategies and techniques employed by qualitative researchers to ensure accountability, heighten value awareness, and seek rigor in interpretation.
References
Burawoy, Michael. "The extended case method." Sociological theory 16, no. 1 (1998): 4-33.
Harding, S. (1995). Can feminist thought make economics more objective? Feminist economics, 1(1), 7-32.
Meinzen-Dick, R. S., Rubin, D., Elias, M., Mulema, A. A., & Myers, E. (2019). Women’s empowerment in agriculture: Lessons from qualitative research (Vol. 1797). Intl Food Policy Res Inst.
Riessman, C., & Given, L. (2008). Sage Encyclopedia of Qualitative Research Methods.
Small, M. L., & Calarco, J. M. (2022). Qualitative literacy: A guide to evaluating ethnographic and interview research. Univ of California Press.
Zolvinski, S. (2008). Listening to farmers: qualitative impact assessments in unfavorable rice environments.
