TRUST IN RESEARCHJune 2019Researcher survey resultsBackground and approachResearch objective is to: Examine the drivers and influences on the communication of scholarly research.26.09.20192About the surveyTrust in research survey 3133 researchers responded to a survey of 98160 individuals randomly selected from database of 3.6 million researchers (3.2 % response rate). Survey tool: Co-branded (Elsevier and Sense About Science) online survey available in English only. Survey took 15 minutes to complete (median average). Fieldwork took place in May 2019.Search and discovery activities 1450 researchers responded to a survey of 105418 individuals randomly selected from database of 3.6 million researchers (1.4% response rate). Survey tool: Unbranded online survey available in English only. Survey took 17 minutes to complete (median average). Fieldwork took place in Jan-Feb 2019. Results: Responses have been weighted to be representative of the global researcher population by country (UNESCO 2014 data). Base sizes shown in this report are unweighted unless otherwise statedAbout Sense About ScienceSense about Science is an independent charity that challenges misrepresentation of science and evidence in public life.About ElsevierA global information analytics business specializing in science and health helping institutions and professionals progress science, advance healthcare and improve performance for the benefit of humanity. TRUST IN RESEARCHOVERVIEW OF RESULTS31%21%15%48%14%0%10%20%30%40%50%60%70%80%90%100%Research outputsNone of themSome of themAbout half of themThe majority ofthemAll of themTRUSTWORTHINESS OF RESEARCH OUTPUTS: Although 62% of researchers trust the majority of research outputs, a proportion doubt the quality of some of the research outputs they encounter. To compensate they check supplementary material/data carefully, read only information associated with peer reviewed journals or seek corroboration from other trusted sources.26.09.20194Thinking about the various research outputs that you interacted with (or encountered) last week what proportion of the outputs would you consider trustworthy?Which of the following mechanisms do you employ to compensate for any lack of confidence you have in the content you are considering reading/accessing?57%52%52%37%29%6%7%Check supplementary material or data carefullyOnly read/access content that is in or linked to apeer reviewed journalSeek corroboration from other trusted sources(e.g. see if research is cited in a known journal)Read/access research from researchers I knowRead/access research from specific institutesOnly read/access research that has beenpersonally recommendedOther (please specify)Base: All respondents (n=3133) Base: All respondents that do not think all research outputs are trustworthy(n=2715)37%62%REASONS RESEARCH OUTPUTS ARE REGARDED AS UNTRUSTWORTHY: for those researchers who rate at least some outputs as untrustworthy there are a variety of factors influencing them.26.09.20195You indicated that some/all of the research outputs are trustworthy. Why do you think all/some of the content you encounter is not trustworthy? “Content that comes from predatory journals is not trustworthy. It is not rigorously peer reviewed. This diminishes the trust in scientific research. As scientists we need to be held to a high standard. The traditional peer review system does that.“ (Physician, USA, prefer not to say age)“Not familiar with the journals or media/not sure what the peer review process might be, or how reviewers are selected“ (Psychology, United States of America, 26 to 35)“Experiments poorly designed, some analyses seemed suspect, areas that I know well improperly characterized“ (Environmental Sciences, Canada, Over 65)“There is published Research biased by financial or other Support to the authors and not properly declared.“ (Medicine and Allied Health, Switzerland, 56 to 65)“Authors often do not provide data/code/tools/proper description of the scenarios used for the evaluation contained in their papers. In particular, the correctness of code used for simulations reported in papers is often unverifiable.“ (Computer Sciences / IT, Brazil, 26 to 35)Lack of supplementary materialErrors: inflated statistical power/ grammatical/calculationsUnsupported claims: Poor conclusions drawn New channels: (social media, media outlets)Methodological flawsNew research outputs (preprints, data)Unclear if peer reviewedBiases(in peer review, funding, negative findings not published) Predatory journalsReduced focus on novel/ high-quality research Inhibitors of trustComponents of mistrustPressure to publishForces driving volume of researchNot peer reviewed Low quality peer review Peer reviewscopeGrowth in researchersOpen ScienceReasons research outputs are regarded as untrustworthy: of those researchers that rate at least some outputs as untrustworthy; the main reasons are poor interpretation, lack of clarity of the peer review process and flaws in the methodology 26.09.20196You indicated that some/all of the research outputs are trustworthy. Why do you think all/some of the content you encounter is not trustworthy?25%19%17%15%12%11%9%Poor conclusions drawn | Exaggerated findings | Lack ofinformation or detail providedLack of peer reviewMethodological flaws e.g. Reproducibility /generalisability / incorrect methods usedBias e.g. Peer review / funding source / data falsification/ pressure to publish = lower quality / negative findingsnot publishedLack of supplementary material | Unable to validateconclusionsNegative perception of source:Website/author(s)/predatory journalsErrors: Grammatical / citation / inflated statistical power /calculations / code49%43%41%38%37%33%28%26%21%13%11%33%30%37%37%33%30%35%37%32%25%24%14%21%17%17%20%28%23%30%25%30%26%4%6%6%8%9%8%15%7%23%31%39%Misinterpretation of research outcomes in media, policy or public discussionDeliberate misrepresentation by mediaIncreased low quality research available (i.e. research meets minimum technicalstandard only)Difficulty in distinguishing high quality researchToo many non-peer reviewed research publicationsDeliberate misrepresentation by researchers/their institutionsVolume of informationMisinterpretation of research outcomes by researchers/their institutionsContradictory research findings published (e.g. research from different groups haveopposing conclusions)Increased alternative sources (e.g. different versions of articles: preprint, accepted,published)Large number of competing platforms providing access to the same research outputsLarge Medium Small Not a problem at allPUBLIC CONFIDENCE IN RESEARCH EVIDENCE: biggest problem seen to be misinterpretation or deliberate misrepresentation by the media as well as difficulty identifying high quality research.26.09.20197Thinking about public confidence in research evidence, how much, if at all, do you believe any of the following are a problem?Base: All respondents (n=3133)70%40%38%35%35%30%Explain research context, findings and implications in lay termsEnable them to ask questions of the authors (e.g. discussion to beposted alongside the article)Clear explanation of peer review and make it easily accessible fromcontent (e.g. how many people involved, their role)Peer reviewed content is clearly marked as suchEnable updates to the article after publication (e.g. as more work isdone on the topic by author)Provide guidance on statistics (e.g. probability)COMMUNICATION TO THE PUBLIC: Explaining research in lay terms is seen as the best way to help people outside the research community judge the quality of research8To help people outside the research community judge the quality of research articles they view, which of the following would be most helpfulBase: All respondents (n=3133)9TRUST IN RESEARCHDemographicsDemographics 10N=313320%17%9%6%5%5%4%4%3%2%27%ChinaUSAJapanRussiaGermanyRep. of KoreaUKFranceIndiaCanadaOther countries5%6%8%16%19%5%4%12%8%17%ChemistryComputer ScienceEarth & Env. ScienceEngineeringLife SciencesMaterial ScienceMathsMedicine and AHPhysics & AstronomySSE + Arts HumSubject Country RegionOrganization PositionWestern Europe21%Eastern Europe10%Middle East3%APAC41%Africa 2%Latin America 4%North America 19%16%36%37%11%Head of Dept./ SeniorManagementSenior Researcher/Middle ManagementResearcher/ StaffMemberOther66%16%5%4%3%2%2%UniversityResearch InstituteHospital / Med. Schmercial / Corp.GovernmentCollegeOtherNumber of articles published8%18%17%21%11%8%10%5%5 or fewer6-1516-2526-5051-7576-100101-200Over 20048%21%16%5%4%2%1%3%Research and/or developmentR&D and teaching equallyTeachingPractitioner (clinical)Mgmt/Admin.Advisory/ConsultancyPractitioner (eng./tech.)OtherRoleAge0.5%16%27%24%18%12%2%Under 2626-3536-4546-5556-65Over 65Prefer not to say