SAMPLE: Qual v Quant

I will not prioritise quantitative over qualitative research, instead treating each piece of research on its merits.

I will accept the limitations of each piece of research that I do, and how further qual/quant is necessary to strengthen it.

 

How quant & qual complement one another

When I get a chance to stop and think, it is obvious to me that both qualitative and quantitative research have their respective strengths and weaknesses.

Done well, quantitative research can make use of the miracle of sampling, enabling us to generalise from a relatively small survey sample to a whole population.  And the best quantitative research – particularly randomised trials and natural experiments – can give us greater confidence than any other method in showing that one thing causes another. However, as Cartwright & Hardie (2012) convincingly argue, just because we know that something had a causal effect there (where the RCT happened) does not mean that we can be sure it will have a causal effect here (in the situation we are looking at). To know this, we need to know more than whether there was a causal effect – we need to know the mechanisms of the effect, and the context that allows this mechanism to take place.

Done well, qualitative research can uncover causality in a different way (Mahoney & Goertz 2006), allowing us insights into the mechanisms and context that Cartwright & Hardie (2012) show is so important. It can also us give the richer, deeper, ‘thicker’ (Geertz 1973) understanding of the world that is necessary for us to construct good quantitative measures in the first place. Qualitative research does, however, have exactly the opposite problems as quantitative research. Qualitative studies show depth at the price of generalisability, and show causal complexity at the price of being quite as confident that we really are uncovering a causal effect.

The intellectual case vs. the real world…

Yet when I am doing & using research in practice, I find that these obvious principles can become compromised, in one of two ways.

Firstly, there are many incentives to over-claim what we can show in our research (and these incentives exist in academia & policy debates alike). It is easy to do qualitative research and pretend that it is somehow generalisable, or that its causal claims are stronger than they really are. It is easy to do quantitative research and pretend that a single variable is sufficient to capture the complexity of a situation that we don’t understand, or that an RCT is the last word in a policy debate. I therefore commit to accepting the limitations of all the research I do, and accept that each quant study could be enhanced by a complementary qual study, and vice versa.

Secondly, however much I try to sidestep the debate on qual vs. quant research, there are many (such as Sale et al 2002) who insist that all quantitative research is ‘positivism’, and qual & quant are incompatible research paradigms. Faced with such provocations, it is easy to go back into ‘taking sides’, and becoming a quantitative researcher keen to defend myself against qualitative researchers. This can lead me to prioritise quantitative over qualitative research, even when I know that both are equally necessary to adequately understand the social world. I therefore commit to judging each piece of research on its merits, however much I am provoked to take sides in a debate that I think should simply not exist.

 

Bibliography

Nancy Cartwright and Jeremy Hardie (2012), Evidence-Based Policy: A Practical Guide to Doing It Better.

Geertz, Clifford (1973). “Thick Description: Towards an Interpretative Theory of Culture,” in The Interpretation of Cultures: Selected Essays.

Mahoney, James, and Gary Goertz. “A tale of two cultures: Contrasting quantitative and qualitative research.” Political Analysis (2006): 227-249.

Sale, J et al (2002), ‘Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research’. Quality & Quantity 36: 43–53, 2002.

Advertisements