Qualitative Synthesis Methods: Critical Interpretive Reviews, Narrative Reviews, Expert Opinions

Image

To view in your browser and/or download the article, click on the PDF below. 

Synthesis as a research method is frequently defined as the integration of multiple sources of data to generate new findings, to increase the statistical power of an analysis, or to broaden the spatial or temporal inference of results. (Also see Carpenter et al. 2009, Hackett et al. 2016.) With individual datasets that are in the same format or those that can be harmonized,¹ researchers can combine them into a single database and subject them to a traditional statistical analysis. Socio-environmental research, however, brings together information from many disciplines and in many forms—including both quantitative and qualitative data. Thus, it is not always possible to use data-focused statistical methods. Further, it is not always desirable to do so because synthesis goals may be related to elaborating perspectives or generating theory rather than evaluating evidence to support a specific question. 

Qualitative and semi-qualitative syntheses help accomplish those goals. They may rely on published or unpublished work and usually are done by scholars with deep knowledge of the topic. However, they may also involve input from non-specialists, especially individuals who are in a position to use the synthesis results to inform policies, management, or practices. In the following sections of this explainer, brief overviews of three types of qualitative review approaches are provided. For more information on these see: Dixon-Woods et al. 2006; Popay et al. 2006; Edwards et al. 2016.

Critical Interpretive Reviews  

A critical interpretive review integrates the methods used in systematic reviews with a qualitative tradition of inquiry. It may clearly delineate how sources of information were selected, but it does not seek to draw conclusions based on all relevant sources or to say what “all the evidence” suggests. Rather, it seeks to generate a framework or theory by interpreting or critiquing a group of studies that are often qualitative in nature. And it uses an interactive and iterative process for collecting information to address a question that may actually evolve over time during the review process. Offering qualitative insights on a topic, critical interpretive reviews often appear in work that requires some subjective insights. McDougall (2015) discusses it well in her article on bioethics research. A socio-environmental example comes from work by Hirons (2021) who used critical interpretive review methods to synthesize selected research. He used synthesis to reflect on key issues and future prospects related to “natural climate solutions” and the extent to which those solutions deliver on their "promises” depends on governance.

Narrative Reviews

In the context of synthesis, a narrative review is interpretative in nature and aimed at developing or advancing concepts, theories, or “plausible truths” (as understood or defined by Greenhalgh et al. 2018). It can read very much like a standard scientific paper, or it can use a story-like narrative summary to explain findings, views, and perspectives based on the author’s knowledge. It usually starts as a broad question or topic and evolves over the course of the synthesis process. Policy makers or other stakeholders may be involved in the process when the purpose of the review is to inform decisions. Often, however, the review is by a scholar with broad knowledge of the field and some perspective on work that has been done on the topic. The review process may involve using tools to analyze textual documents and extract themes, but it can also involve tabulating information, categorizing it as a group, and counting votes. A socio-environmental example comes from work by Froeling et al. (2021) who provided a narrative review to “promote” (their words) the understanding and application of citizen science in environmental epidemiology. They cite literature extensively to bolster their explanations and arguments but do not describe any type of systematic evaluation of the literature. 

Expert Opinions

Expert opinions play a significant role in evaluating or developing policies or procedures in a variety of fields but particularly in the health sciences and natural resource management. There is now wide recognition that policies are more equitable when a broad range of people contribute to their development, and they are also more likely to have traction. How opinions are collected range from highly structured, scientific approaches (e.g., see expert elicitation methods in the SESYNC Explainer: Quantitative Synthesis Methods: Literature Reviews, Expert Elicitation) to very informal methods based on conversations. Many socio-environmental studies rely only on the research team and its discussions or critical evaluations of material as the expert sources. For example, Weiskopf et al. (2022) developed recommendations for improving decision makers' uptake of socio-ecological models based on discussions among the authors who had extensive experience in modeling. Increasingly, researchers are extending their definition of “experts” much more broadly to bring in more diverse types of knowledge, which may lead to more sustainable decisions e.g., Albuquerque et al. 2021.

¹ Data harmonization is the process of building a composite dataset after ensuring data are in a consistent, standardized format. Often, this process involves converting data to common units, but sometimes, data on the same topic has been collected using different methods or at different scales, and thus researchers must use modeling or other tools to make them comparable.

References:
Albuquerque, U.P., Ludwig, D., Feitosa, I.S. et al. (2021). Integrating traditional ecological knowledge into academic research at local and global scales. Regional Environmental Change, 21(2): 1-11. https://doi.org/10.1007/s10113-021-01774-2

Carpenter, S.R. et al. (2009). Accelerate synthesis in ecology and environmental sciences. BioScience, 59: 699–701. https://doi.org/10.1525/bio.2009.59.8.11

Dixon-Woods, M., Cavers, D., Agarwal, S. et al. (2006) Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology, 6(35). https://doi.org/10.1186/1471-2288-6-35

Edwards, J., & Kaimal, G. (2016). Using meta-synthesis to support application of qualitative methods findings in practice: A discussion of meta-ethnography, narrative synthesis, and critical interpretive synthesis. The Arts in Psychotherapy, 51: 30-35. https://doi.org/10.1016/j.aip.2016.07.003

Froeling, F., Gignac, F., Hoek, G., Vermeulen, R. et al. (2021). Narrative review of citizen science in environmental epidemiology: Setting the stage for co-created research projects in environmental epidemiology. Environment International, 152: 106470. https://doi.org/10.1016/j.envint.2021.106470

Greenhalgh, T., Thorne, S., & Malterud, K. (2018). Time to challenge the spurious hierarchy of systematic over narrative reviews? European Journal of Clinical Investigation 48(6): e12931. https://doi.org/10.1111/eci.12931

Hackett, E.J., & Parker, J.N. (2016). From Salomon's house to synthesis centers. In: Heinze, T., Muench, R. (Eds.), Innovation in Science and Organizational Renewal: Historical and Sociological Perspectives. (pp. 53–88). Palgrave Macmillan New York.

Hirons, M. (2021). Governing natural climate solutions: prospects and pitfalls. COSUST 52: 36-44. https://doi.org/10.1016/j.cosust.2021.06.012

McDougall, R. (2015). Reviewing literature in bioethics research: increasing rigor in non-systematic reviews. Bioethics 29: 523-528. https://dx.doi.org/10.1111/bioe.12149

Popay, J., Roberts, H., Sowden, A. et al. (2006). Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: A Product from the ESRC Methods Programme, Version, 1(1), b92.

Weiskopf, S.R., Harmáčková, Z.V., Johnson, C.G., Londoño-Murcia, M.C., Miller, B.W., Myers, B.J., Pereira, L, Arce-Plata, M.I., Blanchard, J.L., Ferrier, S., & Fulton, E.A. (2022). Increasing the uptake of ecological model results in policy decisions to improve biodiversity outcomes. Environmental Modelling & Software, 149: 105318. https://doi.org/10.1016/j.envsoft.2022.105318