Date
Description
Image
To view in your browser and/or download the article, click on the PDF below.
Document
Synthesis as a research method is frequently defined as the integration of multiple sources of data to generate new findings, to increase the statistical power of an analysis, or to broaden the spatial or temporal inference of results. (Also see Carpenter et al. 2009, Hackett et al. 2016.) With individual datasets that are in the same format or those that can be harmonized, researchers can combine them into a single database and subject them to a traditional statistical analysis. Socio-environmental (S-E) research, however, brings together data from many disciplines and in many forms—including both quantitative and qualitative data.
One way to integrate such diverse information is by conducting syntheses. Syntheses based on analyses of published studies belong to a category of synthesis methods called critical reviews that includes Systematic Reviews and Meta-Analyses, each of which are described briefly below. Syntheses that rely on input from people considered to be highly knowledgeable about the topic go by a variety of names under the general rubric of “expert opinion.”
Systematic Reviews and Meta-Analyses
Many syntheses are based on an analysis of published studies’ findings and are carefully designed to evaluate all available and relevant information to draw evidence-based conclusions. When researchers gather the scholarly evidence in comprehensive and reproducible ways, they call them systematic reviews. An example includes work by Lavadinović et al. (2021) who synthesized research on the extent of wildlife poaching to identify the reasons behind it. They used explicit criteria to determine whether to include or exclude research studies from their analysis and evaluated the quality of those studies using standards. Systematic reviews are designed to be as objective as possible. More than one person independently evaluates the studies used in the review; if there are disagreements about interpretations of the evidence, another person weighs in. Often, a team of “experts”—those deemed to have content and methodological expertise on the topic—conduct these reviews. Another socio-environmental (S-E) example of a systematic review comes from Bukvic et al. (2020) who evaluated the effectiveness of coastal vulnerability mapping efforts in addressing physical and social vulnerability to hazards.
Image
In some cases, systematic reviews involve the use of a meta-analysis—a statistically based method for distilling the results from many studies or cases to generate a more robust finding. The method involves pooling the magnitude of outcomes (i.e., the effect of “treatments”) from studies that asked the same question but may not have used the exact same methods. Meta-analyses can be tricky to perform because researchers must take great care to ensure that differences between studies do not result in strong biases or even incorrect conclusions. If studies aggregate data in different ways, then including them in a single analysis can be problematic. For example, combining the effect sizes from all studies evaluating rivers’ ecological restoration may suggest restoration does not ‘work,’ yet we know that the outcome is highly dependent on the restoration method used and the level of prior ecological disturbance (Jones et al. 2018). When carefully done, however, meta-analyses are very useful because they can provide a pooled estimate of effectiveness with confidence intervals that researchers can test statistically. A socio-environmental example is a study by Beckman et al. (2019) who conducted a meta-analysis of 115 studies to determine if land-use intensification results in a trade-off between species diversity and agricultural yields.
Expert Elicitations
Obtaining a synthesis of opinions from experts can vary from informal conversations to highly structured consultations that result in reducing uncertainty around some issue. The latter best describes expert elicitation – a process for obtaining probabilistic judgements from a number of individuals believed to be authorities on the topic. They are asked to respond to prompts or questions associated with scenarios. For example in work by researchers Legge et al. (2022), experts were asked to estimate the proportional loss of a wildlife population following fires of different severity and assuming no or some changes in environmental management. Experts had to include upper and lower bounds on their estimates along with their confidence in those bounds. Such estimates can help inform policies.
For some elicitation exercises, experts discuss their probabilistic estimates (which are kept anonymized) and reach some agreement on “best” estimates. In other cases, experts are judged for their “expertise” based on their responses to scenarios for which answers are known to the researchers; researchers then rely on the “best” expert judgements for their potential policy actions. A socio-environmental example includes a study by St. Laurent et al (2022) who used a Delphi expert elicitation as part of an effort to develop a method for evaluating climate adaptation projects for positive social and biodiversity outcomes. Elicitations using a Delphi approach allow experts to view the responses of other individuals, often with some group discussion followed by arriving at a group consensus or ranking.
References:
Bukvic, A., G. Rohat, A. Apotsos, and A. de Sherbinin. (2020). A systematic review of coastal vulnerability mapping. Sustainability 12(7): 2822. https://doi.org/10.3390/su12072822
Beckmann, M., K. Gerstner, M. Akin-Fajiye et al. (2019). Conventional land-use intensification reduces species richness and increases production: a global meta-analysis. Glob. Change Biol. 25(6): 1941-1956. https://doi.org/10.1111/gcb.14606
Carpenter, S.R, et al. (2009). Accelerate synthesis in ecology and environmental sciences. BioScience 59: 699–701. https://doi.org/10.1525/bio.2009.59.8.11
Hackett, EJ, Parker, JN., 2016. From Salomon's house to synthesis centers. In: Heinze, T., Muench, R. (Eds.), Innovation in Science and Organizational Renewal: Historical and Sociological Perspectives. Palgrave Macmillan, pp. 53–88.
Jones, H.P., P.C. Jones, E. Barbier et al. 2018. Restoration and repair of Earth’s damaged ecosystems. Proc. R. Soc. B 285:20172577. http://dx.doi.org/10.1098/rspb.2017.2577
Lavadinović, V.M., C.A. Islas, M.K. Chatakonda et al. 2021. Mapping the Research Landscape on Poaching: A Decadal Systematic Review. Front. Ecol. Evol. 9: p.630990. https://doi.org/10.3389/fevo.2021.630990
Legge, S., L. Rumpff, C. Woinarski et al. 2022. The conservation impacts of ecological disturbance: Time‐bound estimates of population loss and recovery for fauna affected by the 2019–2020 Australian megafires. Glob. Ecol. Biogeogr. https://doi.org/10.1111/geb.13473
St-Laurent, G.P., L. E. Oakes, M Cross and S. Hagerman. 2022. Flexible and comprehensive criteria for evaluating climate change adaptation success for biodiversity and natural resource conservation. Environ. Sci. Policy 127:87-97. https://doi.org/10.1016/j.envsci.2021.10.019
Share