Hypothesis testing, significance testing

Written by Rob Davies

NoteOverview

This week we work to develop our understanding of hypothesis testing and significance testing. We do this so that we can build our skills for thinking, in general, about how we should respond to evidence.

  • The discussion is anchored in the context of correlation analysis in Psychological research on the link between reading experience and vocabulary knowledge.

You can navigate to each in the side bar on the right.

Lectures

Interactive lecture – This week, we learn about hypothesis testing and significance testing as methods for responding to evidence

You can watch the lecture videos using the playback windows shown next. The lecture is broken into six parts, each about 5-25 minutes long.

  1. We are going to learn about hypothesis or significance testing
  2. What is NHST and where does it come from?
  3. Demonstration: significance testing
  4. Hypothesis testing
  5. Seeing the Neyman-Pearson approach at work
  6. Summary
Tip

During the lecture, I suggest some questions you can think about. Reflecting on your responses to these questions will help you to learn.

You can download the lecture slides as a .html file you can open in any browser.

The slides end with a references list but don’t worry about reading what’s in the references list, we give you recommended readings in Section 2.

Tip

The slides presented in the next videos can be downloaded as a web page: hypothesis-testing-significance.html.

When you download the file, double click on it to open it in your usual web browser for viewing.

There are many slides but not all slides are discussed in-depth in the lecture, some are there to help to visually support transitions between points.

Next, you can see the lecture elements. We recommend that you watch them in sequence.

  1. We are going to learn about hypothesis or significance testing (~5 minutes). This is a challenging topic. We can meet this challenge by enriching our perspectives to take in historical, philosophical, and psychological viewpoints.

  1. What is NHST and where does it come from? (~13 minutes). We are going to learn about hypothesis testing and significance from two different perspectives: (1.) The perspective of an individual researcher, asking what they can learn from the evidence from their study. (2.) The perspective of someone looking at errors that we (in general) could make as we make decisions based on the evidence from multiple repeated studies. We begin with Ronald Fisher’s ideas.

  1. Demonstration: significance testing (~20 minutes). We have outlined the logic of the significance test in words. But it is often easier to understand the logic if we see how the ideas work in practice.

  1. Hypothesis testing (~10 minutes). We switch to a different philosophy and a different view of the psychology of researchers: we look at the ideas of Jerzy Neyman and Egon Pearson.

  1. Seeing the Neyman-Pearson approach at work (24 minutes). To help us to build understanding, we work through demonstrations based on simulations of Psychology studies of correlation.

  1. Summary (8 minutes). Two ways of thinking clearly about evidence. The impacts on how we write about and think about research as psychologists.

Reading

TipEasy access to readings

All reading is accessible via the library.

  • Essential: We expect you to be familiar with these reading. You will be assessed on this.
  • Recommended: We highly recommend these readings. They often provide a different perspective, or explain things in a different way.
  • Optional: Further readings for those interested.

This week, the reading is optional but Recommended:

  • Gigerenzer, G. (2018). Statistical Rituals: The Replication Delusion and How We Got There. Advances in Methods and Practices in Psychological Science, 1(2), 198–218. https://doi.org/10.1177/2515245918771329 Recommended
Back to top