Science in society
Incentives and obligations

We are learning how to think critically for ourselves

Active learning works best even if it sometimes feel challenging

• Active learning is what you do when you ask or answer questions
• We will prompt you sometimes with Activate

What we are going to do

We are going to reflect critically on how we practise science communication

1. The representation of credibility, uncertainty, and generalizability
2. The role of story telling
3. The use of clear or persuasive language or visualizations
4. The ethics of practice in science communication

Activate

Our primary learning question is this:

Science and society

We are going to work in the zone at the intersection between science and society

• All (most) scientists want to have impact

  • To influence policy makers and practitioners
  • To get funding, prestige, power
  • To sell books, ideas, talks, consulting services, tests, clicks

Science and society

• All (most) scientists will be affected by

  • The priorities of funders or policymakers
  • The concerns and interests of practitioners, patients and the public
  • What work accrues reputational credit
  • What work is accepted, published
  • What work sells

Science operating within society

We need to talk about communication

• Whereas we have reflected on science, theory, and hypothesis testing

• We now need to reflect critically on practices of communication in science

  • What and how we communicate to others about our motivations, what we do, what we find, what the implications of our findings are

We will engage with tensions we may not be able to resolve

We will look at credibility

• What kinds of methods, observations, or conclusions are represented as credible

We will engage with tensions we may not be able to resolve

We will look at our obligations to represent uncertainty in our theory, our methods, our observations, our conclusions

• The challenges we may face in seeking to represent uncertainty

• More or less effective ways of representing uncertainty

• The impacts of choices on claims on authority or credibility

We will engage with tensions we may not be able to resolve

We will look at our obligations to represent generalizability in our theory, our methods, our observations, our conclusions

• Challenges we may face in seeking to represent generalizability

We will engage with tensions we may not be able to resolve

We will, in looking at scientific communication, discuss the tension between

• Increasing expertise or specialization and its impact on the readability of what we say
• Recommendations to use plain language, to make things readable, to tell a story, to have impact

We will engage with tensions we may not be able to resolve

We will look at the ways in which using persuasive language or story-telling can be

• Recommended as a means to effect influence or impact
• Unethical or quesionable as a scientific practice
• Helpful or enlightening or stimulating as a means to think critically

Activate

Our learning questions then are these:

Reflections on recommendations for how we do and how we represent science

The normal business of science: solving puzzles

• Most of the time, scientists, working as members of a community or a tradition, work within a shared understanding of:

  • what problems are worth investigating
  • what solutions are possible
  • what methods can be used

• In the context of this paradigm (Kuhn, 1970), the scientist can spend their time figuring out solutions to the uncertainties that remain: solving puzzles

Anomalies and revolutionary change

But over time scientists may come to see a (Kuhn, 1970):

… persistent failure of the puzzles of normal science to come out as they should

A time of crisis ensues, leading to rejection of one paradigm, and acceptance of another

The credibility revolution

Vazire (2018) identifies a credibility revolution:

• Greater emphasis on transparency and openness
• Move toward preregistration of research
• More direct-replication studies
• Higher standards for the kinds of evidence needed to make claims

The science reform movement recommendations

A core set of recommendations can be found in manifestos like Munafò et al. (2017):

• Promote pre-registration
• Improve quality of reporting (checklists)
• Protect against conflicts of interest
• Encourage transparency and open science
• Diversify peer review
• Reward open and reproducible practices

Changes in practice

We do see changes in practice (Nosek et al., 2022):

Nosek et al. (2022): Growth in counts of users, sharing of files (research data, materials, code), and registration of studies

Critical self-awareness: peer review as a previous change to maintain credibility

• We see peer review emerging over time, in different forms in different places, and for different purposes (Spier, 2002)
• To understand why and how peer review emerges, we can follow the money (Baldwin, 2018):

Scientists and their supporters cast expert refereeing—or “peer review,” as it was increasingly called—as the crucial process that ensured the credibility of science as a whole.

Science reform as a narrative

Be aware of presentations of the credibility revolution or of the need for reform or of recommendations of open science practices, as elements of a narrative (Bartscherer, 2026):

• The Open Science story is “purposeful, influential, and by no means neutral”

  • Open science: the good (slow, equitable, inclusive, sustainable) science that counters the corrupting impacts of peverse incentives or human flaws which saves our science and thus public faith in science
  • Process fixes located in practices like confirmatory hypothesis testing

Historical conditions for specific modes of science can change

• Through the middle of the 20th century, we completely changed how we did research (Gigerenzer, 1993)

  • We stopped studying individual subjects (almost) and learned to focus on group averages

  • We started doing and reporting null hypothesis tests (using ANOVA or t-test methods)

    • Danziger (1990) argued that this shift was partly due to pressure felt by US academic psychologists to legitimize their work through showing its practical utility

Science reform language co-opted by authoritarians

• MAGA and “Gold standard science” (Delawalla et al., 2025):

  • Executive Order 14303 (Trump, 2025) mimics the language of the reform movement
  • To enable political appointees to designate research as scientific misconduct

Exposing uncertainty

Exposing uncertainty

Exposing uncertainty

Multiverse analyses of uncertainty

Multiverse analyses suggest estimates of psychological effects (phenomena) vary (Silberzahn & Uhlmann, 2015)

Silberzahn & Uhlmann (2015): Variation in effects estimates by different analysis teams

Multiverse analyses of uncertainty

Auspurg & Brüderl (2021) argue this happens when researchers don’t know how to interpret the research question

Silberzahn & Uhlmann (2015): Variation in effects estimates by different analysis teams

Levels of uncertainty

McElreath (2020) further distinguishes levels of uncertainty over model predictions (supposing data collection, sampling, pre-processing, and model choices):

• The ways that population outcomes are predicted to vary on average given a model
• The ways that specific responses can be produced given model average predictions

Levels of uncertainty

Hofman et al. (2020) explain

• Inferential uncertainty: the degree to which a summary statistic is known to the scientist

  • We can reduce inferential uncertainty by collecting more data e.g. gathering a larger sample to get to a more precise estimate of average height

• Outcome uncertainty: how much individual outcomes vary about the summart statistic regardless of how well it has been estimated

  • Even with a large sample and precise estimate of the average, individual heights will still vary

Visualizations of uncertainty

• Scientists often choose to display inferential uncertainty (precision of statistical estimates) using confidence intervals

  • But depicting inferential uncertainty alone causes participants to overstate beliefs in effects and to understate variability in individual outcomes (Hofman et al., 2020)
  • Readers think that results are more definitive than they really are (Zhang et al., 2023)

Effects of communicating uncertainty on public trust in facts and numbers

• van der Bles et al. (2020): Does openly communicating uncertainty around facts and numbers necessarily undermine audiences’ trust in the facts, or the communicators?

  • People do perceive greater uncertainty when it is communicated

    • through numerical ranges with a point estimate

  • Only small decrease in trust in numbers or in perceived trustworthiness of the information source

Generalizability: we can never test all the people on all the things

Researchers often make claims about the generalizability of their findings

Yarkoni (2022): we need to be careful in analyses and in presentations of analyses about the alignment between verbal hypotheses and statistical tests

• Is our operationalization of our constructs valid: do the things we measure capture the things we say they measure, and do those things actually exist (see also Borsboom et al., 2004)?
• Is our statement of theory, including who or what the theory applies to, adequately captured by our statistical model?

Constraints on generalizability

• Simons et al. (2017): recommend that we include Constraints on Generality (COG) statements in our reports

  • few papers identify their target populations
  • fewer papers justify how or why the tested samples are representative of broader populations

Constraints on generalizability

• Simons et al. (2017): there may be incentives to make the strongest possible claim of generality

• But failure to specify target populations contributes to problems in scientific communication, leaving uncertainty about

  • how or to whom reported findings generalize
  • what would be required for direct replications or for conceptual replications

Constraints on generalizability

Simons et al. (2017) recommend including a COG statement in discussions to explicate:

• Known empirical or theoretical boundary conditions

  • How is your sample of participants representative of a broader target population
  • The class of materials/stimuli to which your finding should generalize
  • What aspects of procedure are critical to reproducing the finding
  • Historical or temporal transparency

Transparency around claims on causality

• There is a taboo against addressing causal research questions and avoid drawing causal inference on the basis of nonexperimental evidence (Grosz et al., 2020)

  • But this results in a disconnect between original findings and how they are interpreted in subsequent work
  • Better to to talk openly about causal assumptions

Incentives

• Proulx & Morey (2021) criticize recent emphasis in normative psychological science practices on:

  • “interesting research” that is salient to the lay intuitions of the general public
  • findings that are “surprising”, novel and counterintuitive relative to common intuitions and established theory
  • research that is “groundbreaking”

No, it’s not the incentives, it’s you

Yarkoni (2018): we have to be careful

• An account of the prevalence of (perverse) incentives
• Does not absolve us of our personal responsibilities

Obligations and responsibilities for the scientist in society

• It is our responsibility to put things in the public domain and to make claims on attention for useful or helpful observations

  • Especially if our work was publicly funded

• But we must engage with our obligations also to be clear about limits on credibility: uncertainty, generalizability, and the identification of causal mechanisms

Activate

Our learning questions are this:

The complex link between science and story telling

Science is claimed to be getting less readable over time

Concerns over readability are one reason why science communication advice embraces “plain language” (DuBay, 2004; Schriver, 2017)

Plavén-Sigray et al. (2017): In a corpus consisting of 709,577 abstracts published between 1881 and 2015 from 123 scientific journals, the readability of science is steadily decreasing. This trend is indicative of a growing use of general scientific jargon.

Science communiction experts recommend plain language

• Use short, common, words
• Use short simple sentences, paragraphs
• Explain jargon but avoid jargon if you can
• Avoid the passive voice

The response to scientific communication relies on cognitive short-cuts

Scientific jargon can be satisfying but misleading (Cruz & Lombrozo, 2025)

• Jargon increases satisfaction because laypeople assume the jargon fills gaps in explanations that are otherwise incomplete

Stories are for telling, stories are for thinking

• Stories are central to engagement with the development and evaluation of theories (Gelman & Basbøll, 2014)

• But for a story to be useful in this way, it should be:

  • anomalous representing aspects of life that are not well explained by existing models
  • immutable carrying details that are well-enough established that they have the potential to indicate problems with a new model

Consider carefully how we may link research to policy

• This advice is not based on empirical evidence, or on good understandings of key literatures on policymaking or evidence use (Oliver & Cairney, 2019)

• A critical interrogation reveals important dilemmas:

  • Should you ‘co-produce’ research and policy and give policy recommendations?
  • What are the causes of unequal rewards and punishments for engagement?
  • Is engagement an event or (more likely) a career choice?

Consider carefully how we may link research to policy

What then are we to make of recommendations to:

• Engage in more effective dissemination

  • Avoid jargon but don’t over-simplify
  • Make simple and definitive statements
  • Produce good stories based on emotional appeals or humour to expand and engage your audience
  • Produce narrative through scene-setting, casting characters, establishing a plot, and equating the moral with a ‘solution to the policy problem’

How we may link research to policy

What uncertainties are inherent in the current limitations of evidence on what works?

• There is no consensus over what counts as good evidence for policy or how best to communicate good evidence

• Policy-makers may seek cognitive shortcuts to avoid decision-making ‘paralysis’ and allow them to process information efficiently enough to make choices

  • This means we need to understand how policy-makers use such shortcuts to interpret their world, pay attention to issues, define policy problems, and respond to proposed solutions

The ethics of persuasive language

Be aware of the ethics of using persuasive language or structure (Bajraktari & Munafò, 2026)

• Scientific communication is often assumed to prioritize objectivity, yet persuasive strategies may subtly distort research interpretation

The ethics of persuasive language

Bajraktari & Munafò (2026):

• Research reports are not just neutral reports of facts
• They are narratives: rhetorical constructions shaped by disciplinary norms, cultural contexts and social expectations

The ethics of persuasive language

Bajraktari & Munafò (2026):

• The pressure to publish is a well-documented feature of research culture

• Most researchers consider persuasion to be “Sometimes” or “Most of the time” necessary for publication

  • Because it is required by journal editors or by peer reviewers

• Journal expectations and peer review processes can encourage authors to emphasize the novelty or significance of their findings

Persuasive devices as questionable research practices

Bajraktari & Munafò (2026):

• Researchers recognize as attempts to persuade the use of devices like: “Misleading interpretation”, “selective reporting”, “inappropriate extrapolation”, “overgeneralization”, “hype” and “spin”
• Many of these devices overlap with questionable research practices (Nagy et al., 2025)
• But the QRP framing may be unhelpful

Critical reflections on the use of persuasive devices

Corneille et al. (2023) reflected on our own writing styles, and on the writing styles they encountered

• All writing that is intended for publication should be clear and engaging
• and the authors of scientific articles can use a wide range of persuasive communication devices to achieve these goals
• It is also possible for these devices to be mis-used in ways that can mislead readers

Critical reflections on the use of persuasive devices

Corneille et al. (2023):

• Researchers are at risk of misleading the reader about the novelty or strength of their research when they make inaccurate statements about the current state of scientific knowledge

  • failing to cite articles that are relevant
  • citing articles that are not relevant
  • misleading the reader about other work through selective quotation, misquotation, or quotation out of context

Critical reflections on the use of persuasive devices

Corneille et al. (2023):

• Authors can try to inflate the perceived importance of their own research

  • Use of attention-grabbing titles
  • Overgeneralizing their conclusions without sufficient evidence

Critical reflections on the use of persuasive devices

Corneille et al. (2023):

• The perceived coherence of the narrative can also be increased by

  • not reporting findings that would weaken the article
  • by claiming that the results were consistent with a hypothesis when, in fact, the hypothesis was generated after the data had been collected

• Writing that is cryptic, obscure or undecipherable can grant a false sense of expertise and prevent fair assessment

Critical reflections on the use of persuasive devices

Corneille et al. (2023):

• Superficial engagement with open research practices can also be used to mislead readers and reviewers about the strength of research
• An author may seek to overwhelm the reader with large volumes of poorly structured/explained supplementary material in order to prevent scrutiny of a paper’s conclusions (see also Towse et al., 2021)

Summary: Critical reflections

• We work as scientists in society
• This imposes incentives or pressures as well as obligations or responsibilities
• We may have to tell stories about our work
• We can choose what kinds of stories we tell, and how we tell them
• It would be useful to recognize our actions, to see ourselves in our actions, and to decide on what it is we want to see

End notes

Our learning questions are this: