Skip to main content

Research Repository

Advanced Search

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Christie, AP; Abecasis, D; ..., ...; Meyer, CFJ; ..., ...; Sutherland, WJ

Authors

AP Christie

D Abecasis

... ...

... ...

WJ Sutherland



Abstract

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

Citation

Christie, A., Abecasis, D., ..., .., Meyer, C., ..., .., & Sutherland, W. (2020). Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences. Nature communications, 11(1), 6377. https://doi.org/10.1038/s41467-020-20142-y

Journal Article Type Article
Acceptance Date Nov 13, 2020
Online Publication Date Dec 11, 2020
Publication Date Dec 11, 2020
Deposit Date Dec 11, 2020
Publicly Available Date Dec 11, 2020
Journal Nature Communications
Print ISSN 2041-1723
Volume 11
Issue 1
Pages 6377
DOI https://doi.org/10.1038/s41467-020-20142-y
Publisher URL https://doi.org/10.1038/s41467-020-20142-y
Related Public URLs http://www.nature.com/ncomms/index.html
Additional Information Funders : Grantham Foundation for the Protection of the Environment;Kenneth Miller Trust;Australian Research Council Future Fellowship;Arcadia, MAVA, and The David and Claudia Harding Foundation;Natural Environment Research Council (NERC);Portuguese Foundation for Science and Technology (FCT);Koniambo Nickel SAS;Dirección General de Investigación Científica;Comunidad de Madrid;Spanish Government: MEC;U.S. Geological Survey;NextEra Energy;New York City Department of Environmental Protection;WINMON-BE via RBINS—OD Nature;Harold L. Castle Foundation;Clackamas County Water Environment Services River Health Stewardship Program and the Portland State University Student Watershed Research Project;New Zealand Department of Conservation (Te Papa Atawhai);Centre for Marine Environmental & Economic Research, Victoria University of Wellington, New Zealand;CNPq-CAPES grants;BNP Paribas Foundation;NOAA-NMFS;Bat Conservation International student research fellowship;Secretaría de Ciencia y Técnica (UNRC);Alaska Fisheries Science Center, NOAA Fisheries, and U.S. Department of Commerce;Madeira’s Regional Agency for the Development of Research, Technology and Innovation (ARDITI);Alabama Department of Conservation and Natural Resources;Spanish Ministry of Education;Strategic Science Investment Funding of the Ministry of Business, Innovation and Employment, New Zealand;Boreal Peatland LIFE;Parks and Wildlife Finland and Kone Foundation;Mexican National Council on Science and Technology;The Carl Tryggers Foundation;Natural Sciences and Engineering Research Council of Canada;French National Research Agency;Commonwealth Scientific and Industrial Research Organisation (CSIRO);Great Barrier Reef Marine Park Authority, the Fisheries Research and Development Corporation, the Australian Fisheries Management Authority, and Queensland Department of Primary Industries (QDPI)
Projects : The Cambridge Earth System Science DTP;Climate & Biodiversity Initiative, BIOCLIMATE project;Investment for the Future
Grant Number: FT180100354
Grant Number: NE/L002507/1
Grant Number: DL57/2016
Grant Number: UIDB/04326/2020
Grant Number: PB97-1252, BOS2002-01543, CGL2005-04893/BOS, CGL2008-02567
Grant Number: HENARSA-CSIC 2003469-CSIC19637
Grant Number: CGL2007-65176
Grant Number: 2018-T1/AMB-10374
Grant Number: SFRH/BD/78813/2011
Grant Number: UID/MAR/04292/2013
Grant Number: PELD site 23 403811/2012-0, PELD-RAS 441659/2016-0, BEX5528/13-5 and 383744/2015-6
Grant Number: NA16FM1031, NA06FM1001, NA16FM2416, and NA04NMF4720332
Grant Number: FCT PD/BD/52597/2014
Grant Number: 160049/2013-0
Grant Number: SFRH/BD/80488/2011
Grant Number: PTDC/BIA-BIC/111184/2009
Grant Number: M1420-09-5369-FSE-000002
Grant Number: AP2008-00577
Grant Number: CGL2008-02567
Grant Number: LIFE08 NAT/FIN/000596
Grant Number: CONACYT 242558
Grant Number: ANR-10-BTBR-04