Qualitative Comparative Analysis – a method for the evaluator’s tool-kit

I recently attended a five-day course on Qualitative Comparative Analysis run by the Australian Consortium for Social and Political Research at the Australian National University. Apart from wanting to be a university student again, if only for a week, I wanted to better understand QCA and its use as an evaluation method.

QCA is a case-based method that attempts to bridge qualitative and quantitative analysis through capturing the richness and complexity of individual cases, while at the same time attempting to identity cross-case patterns. QCA does this through comparing factors across a number of cases in order to identify which combination/s of factors are most important for a particular outcome.

The strength of QCA is that enables evaluators to not only identity how factors combine together to generate a particular outcome, as outcomes are rarely due to one factor, but if there is only one combination of factors or several different combinations that can lead to the outcome of interest and in what contexts these combinations occur. QCA is also ideal for evaluations with medium-sized Ns (e.g. 5 to 50 cases), as in such a range there are often too many cases for evaluators to identify patterns across cases without a systematic approach, but too few cases for most statistical techniques.

I left the course with an understanding of QCA as a useful addition to our evaluation tool-kit. Apart from enabling evaluators to identify patterns across cases, it allows us to test theories of change and in particular, whether the relationship between intermediate outcomes and end of program outcomes holds true or if there are other factors required to achieve higher order outcomes. It can also be used to triangulate the findings of other methods, such as key success factors identified through a contribution analysis.

There are of course a number of limitations such as QCA requiring both expertise in applying the method and in-depth case knowledge, as well as the time needed to collect comparable data across cases and then returning to the data to further define factors and outcomes as contradictions arise when trying to identify cross-case patterns.

If you want a good overview of QCA, including the key steps for undertaking a QCA, check out:

• Schneider, Carsten and Wagemann, Claudius, Set-Theoretic Methods for the Social Sciences (Strategies for Social Inquiry)
• Ragin, Charles, What is QCA: http://eprints.ncrm.ac.uk/250/1/What_is_QCA.pdf
• Legewie, Nicolas, An Introduction to Applied Data Analysis with Qualitative Comparative Analysis: http://www.qualitative-research.net/index.php/fqs/article/view/1961

And useful references for applying QCA in evaluation include:

• Better Evaluation, Qualitative Comparative Analysis: http://www.betterevaluation.org/en/evaluation-options/qualitative_comparative_analysis
• Baptist, Carrie and Befani, Barbara, Qualitative Comparative Analysis – A Rigorous Qualitative Method for Assessing Impact: http://www.coffey.com/assets/Ingenuity/Qualitative-Comparative-Analysis-June-2015.pdf
• Pattyn, Valérie, Molenveld, Astrid, and Befani, Barbara, Qualitative Comparative Analysis as an Evaluation Tool Lessons From an Application in Development Cooperation: http://journals.sagepub.com/doi/abs/10.1177/1098214017710502