Linear models and interaction effects

Rob Davies

Department of Psychology, Lancaster University

PSYC122: Classes in weeks 16-20

  • My name is Dr Rob Davies, I am an expert in communication, individual differences, and methods

Tip

Ask me anything:

  • questions during class in person or anonymously through slido;
  • all other questions on discussion forum

Week 19: Linear models – interactions and variation in effects

Figure 1: Plot showing how the effect of reading strategy on accuracy varies at different education levels, given an interaction between the effects of reading strategy and education level

Targets for weeks 16-19: Concepts

We are working together to develop concepts:

  1. Week 16 — Hypotheses, measurement and associations
  2. Week 17 — Predicting people using linear models
  3. Week 18 — Everything is some kind of linear model
  4. Week 19 — The real challenge in psychological science

Targets for weeks 16-19: Skills

We are working together to develop skills:

  1. Week 16 — Visualizing, estimating, and reporting associations
  2. Week 17 — Using data to predict people
  3. Week 18 — Going deeper on linear models
  4. Week 19 — Evaluating how effects vary

Learning targets for this week

Concepts – To engage with the real challenge in psychological science:

  1. People vary
  2. Effects vary
  3. Results vary
  • This means we have to think about what Psychologists call interaction effects

Black and white image of cartoon profile pictures.

Crowd of people; open-peeps

Learning targets for this week

Skills – To be able to code linear models that include interaction effects:

lm(outcome.Y ~ predictor.X*predictor.Z)

Learning targets for this week

Skills – To be able to visualize, interpret, and report tests of interaction effects:

# -- fit model
data.lm.acc <- lm(mean.acc ~ strategy_centered*EDUCATION, data = data)

# -- make plot of predictions
plot_model(data.lm.acc, type = "pred", 
           terms = c("strategy_centered", "EDUCATION"))

Why are these our targets? Because people vary, effects vary, results vary

Picture shows a crowd of people seen from above with a variety of colours of clothing.

flickr, Cat Walker ‘crowd’

Critical thinking: Variation

Psychological and social processes show much more variability than the usual phenomena in the physical sciences (Gelman, 2015)

  • The patterns that interest us will vary between places, people, and contexts
  • Because the effects of interventions, treatments or experimental manipulations can be expected to vary
  • We must build the ability to investigate how those effects vary

Critical thinking: Uncertainty

Human variation means that there are three levels of uncertainty when we look at sample data (McElreath, 2020):

  1. The nature of the expected change in outcome \(\rightarrow\) what variables predict average outcome variation?
  2. The ways that expected changes might vary between individual participants or between groups of participants \(\rightarrow\) what differences are there between individuals or groups?
  3. The random ways that specific responses can be produced \(\rightarrow\) how much can we predict anyone’s behaviours?

Critical thinking: Working with samples

We often test who we can – convenience sampling – and who we can test has an impact on the quality of our evidence (Bornstein et al., 2013)

Critical evaluation

  • If the people in our samples vary in age, education, ethnicity or gender \(\rightarrow\)
  • Does the presence of sub-groups or of individual differences within or between our samples matter?
  • If it does matter, how or why does it matter?
  • We examine these questions in the context of the health comprehension project

Let’s take a break

  • End of part 1

Health comprehension project – finding answers to our questions

  • We have been working in the context of a live research project: What makes it easy or difficult to understand written health information?

flickr: Sasin Tipchair 'Senior woman in wheelchair talking to a nurse in a hospital'

flickr, Sasin Tipchair ‘Senior woman in wheelchair talking to a nurse in a hospital’

Health comprehension project: questions and analyses

  • Our research questions are:

Note

  1. What person attributes predict success in understanding?
  2. Can people accurately evaluate whether they correctly understand written health information?

Theory: Models of comprehension accuracy should include as predictors:

(1.) experience HLVA, SHIPLEY and (2.) reasoning ability (strategy, reading strategy) (Freed et al., 2017)

Q nd_1_l Language experience nd_2 Comprehension outcome nd_1_l->nd_2 nd_1_r Reasoning capacity nd_1_r->nd_2
Figure 2: Understanding text (Freed et al., 2017)

Multiple potential predictor variables

It will become important that we distinguish between numeric variables (like age) and categorical variables (factors like education level)

The figure presents a grid of scatterplots indicating the association between outcome mean accuracy (on y-axis) and (x-axis) scores on a range of predictor variables. The points are shown in grey, and higher points are associated with higher accuracy scores. The grid includes as predictors: self-rated accuracy; vocabulary (SHIPLEY); health literacy (HLVA); reading strategy (strategy); age (years); gender; education, and ethnicity. The plots indicate that mean accuracy increases with increasing self-rated accuracy, vocabulary, health literacy, and reading strategy scores. Trends are indicated by red lines.

Figure 3: Scatterplots showing the potential association between accuracy of comprehension and variation on each of a series of potential predictor variables.

What effects could vary?

  • The effect of an intervention or treatment
  • The effect of an experimental manipulation
  • The effect of the difference between groups
  • The characteristics of an association between an outcome and a predictor
Figure 4: How the effect of reading strategy on accuracy varies at different education levels

How could effects vary?

  • We are focusing, here, on the shape and size of an effect
  • The shape and size of an effect is shown by the coefficient estimate
  • The coefficient estimate indicates the slope of the line
  • The slope predicts how the outcome will change, given variation in predictor values
Figure 5: Plot: how simulated data values on mean accuracy and mean self-rated accuracy could vary together

How could effects vary?

  • Is the effect (the slope) positive or negative in direction (see Figure 6): does the line go up (effect is positive) or down (effect is negative)?
  • Is the effect small, or large, or moderate in size: does the slope go up or down steeply (larger effect) or at a shallow angle (smaller effect)?
Figure 6: Plot: how simulated data values on mean accuracy and mean self-rated accuracy could vary together

How could effects vary?

Tip

Is the shape or size of an effect different in different groups of people, in different contexts, or for different values of a third variable (a moderator)?

  • We can answer questions like these by building models that include interaction effects
  • We learn how to do that next

How do we code a model so that it includes an interaction effect?

  • If we think that the shape or size of an effect is different in different groups of people, or in different contexts, or for different values of a third variable (a moderator)
  • Then we think that there is an interaction
  • It helps to understand what this means by first working with an example and then working with a little algebra

Theory of an interaction

  • Hypothesis: the impact of knowing about health matters (health literacy, measured using the HLVA) will interact with the impact of having broad vocabulary knowledge (measured using the SHIPLEY)
  • Theory 1: the effect (slope) of health literacy will be amplified (made steeper) for people who already know about language (and score high on the SHIPLEY)
  • because wider general knowledge improves the capacity to work with specialist (health) information

Theory of an interaction

  • Hypothesis: the impact of knowing about health matters (health literacy, measured using the HLVA) will interact with the impact of having broad vocabulary knowledge (measured using the SHIPLEY)
  • Theory 2: the effect (slope) of health literacy will be reduced (made more shallow) for people who already know about language (and score high on the SHIPLEY)
  • because if you are already doing well (on outcome comprehension) the room for improvement (with more specialist health knowledge) decreases

Visualisation of an interaction

  • Psychologists often talk about interactions as joint effects:
  • Figure 7 shows how the relation between outcome mean.acc and predictor HLVA could be different for different values of a third variable (SHIPLEY)
Figure 7: The relation between HLVA and mean.acc for different ranges of SHIPLEY scores: left plot (scores between 19-33); middle (scores between 33-37); and right (scores between 37-40)

Visualisation of an interaction

The way that

  1. the slope for the relation between outcome mean.acc and predictor HLVA
  2. differs depending on
  3. differences in values of the third variable SHIPLEY

\(\rightarrow\) This is what makes a possible interaction

Figure 8: The relation between HLVA and mean.acc for different ranges of SHIPLEY scores

Let’s take a break

  • End of part 2

Coding models with interactions

lm(mean.acc ~ HLVA*SHIPLEY, data = data)

Revision: Using linear models for prediction

  • With the linear model lm(mean.acc ~ SHIPLEY)
  • We can predict change in mean comprehension accuracy (mean.acc)
  • Given variation in vocabulary scores (SHIPLEY)

Revision: Differences between predictions and observations

  • Higher vocabulary scores are associated with higher accuracy scores
  • Model predictions are circled in black along the blue line
  • Light grey lines indicate differences between predicted and observed outcomes

Revision: Assume residuals are normally distributed

  • The plot shows the distribution of residuals, as a histogram in grey
  • The peak of the distribution is centered at residuals = 0
  • The red curve show what the distribution of residuals would look like if it was perfectly normal

Revision: Previously, we have looked at coding the linear model with multiple predictors

lm(mean.acc ~ SHIPLEY + HLVA + strategy + AGE + NATIVE.LANGUAGE, ...)
  • The code represents a linear model with multiple predictors:
  • \(y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots + \epsilon\)

Revision: For a linear model with multiple predictors, we see


Call:
lm(formula = mean.acc ~ SHIPLEY + HLVA + strategy + AGE, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.38654 -0.07169  0.01129  0.07683  0.31901 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  3.891e-01  4.856e-02   8.013 8.80e-15 ***
SHIPLEY      4.975e-03  1.325e-03   3.754 0.000196 ***
HLVA         1.831e-02  2.582e-03   7.090 4.92e-12 ***
strategy     1.736e-03  7.974e-04   2.177 0.029984 *  
AGE         -7.984e-05  3.954e-04  -0.202 0.840082    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1124 on 473 degrees of freedom
Multiple R-squared:  0.2082,    Adjusted R-squared:  0.2015 
F-statistic: 31.09 on 4 and 473 DF,  p-value: < 2.2e-16
  1. The Coefficient Estimate: for the slope of the effect of variation in each predictor
  2. The Std. Error (standard error) for the estimate of each effect
  3. The t value and Pr(>|t|) p-value for null hypothesis tests of each coefficient

Revision: Thinking about the linear model with multiple predictors

\(y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots + \epsilon\)

Outcome \(y\) is calculated as a sum by adding together:

  • The intercept \(\beta_0\) plus
  • The product of the coefficient of the effect of e.g. AGE \(\beta_1\) multiplied by \(x_1\) a person’s age +
  • + any number of other variables +
  • The error \(\epsilon\): mismatches between observed and predicted outcomes

New: Thinking about the linear model with multiple predictors as an additive model

\(y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots + \epsilon\)

Note

Models like this are said to be additive because

  • The relation between the outcome variable (here, mean.acc) for any one predictor (e.g., the relation between mean.acc and HLVA)
  • Is constant over all values of the other predictor
  • In our example, this means the relation between mean.acc and HLVA would be the same whatever the value of SHIPLEY.

New: Now, we code models with interactions

  • For discussion, we consider an example model in which we focus on the interaction between the effects of health literacy (HLVA) and vocabulary knowledge (SHIPLEY)

  • We can specify interactions in two different ways:

  1. Using *
lm(mean.acc ~ HLVA*SHIPLEY, data = data)
  1. Using :
lm(mean.acc ~ HLVA + SHIPLEY + HLVA:SHIPLEY, data = data)

Identifying key information in results: for a model including an interaction

model <- lm(mean.acc ~ HLVA*SHIPLEY, data = all.data)
summary(model)

Call:
lm(formula = mean.acc ~ HLVA * SHIPLEY, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.37986 -0.06840  0.01342  0.07652  0.29872 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.4806201  0.1445375   3.325 0.000952 ***
HLVA         0.0153042  0.0168363   0.909 0.363810    
SHIPLEY      0.0045132  0.0043491   1.038 0.299919    
HLVA:SHIPLEY 0.0001132  0.0004922   0.230 0.818173    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1128 on 474 degrees of freedom
Multiple R-squared:  0.2004,    Adjusted R-squared:  0.1953 
F-statistic: 39.59 on 3 and 474 DF,  p-value: < 2.2e-16

Identifying key information in results: for a model including an interaction

You see each Estimate for the Coefficients: – the effects of:

  1. HLVA
  2. SHIPLEY
  3. The HLVA:SHIPLEY coefficient for the interaction tells us how the slope of the HLVA effect varies – depending on SHIPLEY scores
model <- lm(mean.acc ~ HLVA*SHIPLEY, data = all.data)
summary(model)

Call:
lm(formula = mean.acc ~ HLVA * SHIPLEY, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.37986 -0.06840  0.01342  0.07652  0.29872 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.4806201  0.1445375   3.325 0.000952 ***
HLVA         0.0153042  0.0168363   0.909 0.363810    
SHIPLEY      0.0045132  0.0043491   1.038 0.299919    
HLVA:SHIPLEY 0.0001132  0.0004922   0.230 0.818173    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1128 on 474 degrees of freedom
Multiple R-squared:  0.2004,    Adjusted R-squared:  0.1953 
F-statistic: 39.59 on 3 and 474 DF,  p-value: < 2.2e-16

We need to use algebra to make sense of what we see

The standard way to represent an interaction in algebra is:

\[y \sim \beta_0 + \beta_1X + \beta_2Z + \color{red}{\beta_3XZ} + \epsilon\]

For this model, the average change in the outcome, \(y\), is calculated by adding together:

  • the values for \(\beta_0\) the intercept plus
  • \(\beta_1X\) the effect of X (for some value of X) plus
  • \(\beta_2Z\) the effect of Z (for some value of Z) plus
  • \(\color{red}{\beta_3XZ}\) the effect of XZ the interaction

What is the interaction term doing?

\[y \sim \beta_0 + \beta_1X + \beta_2Z + \color{red}{\beta_3XZ} + \epsilon\]

To make this concrete, we can translate this model using the variables from our example data:

\[ \begin{aligned} mean.acc &= {\beta_0} + {\beta}_{1}(HLVA) + {\beta}_{2}(SHIPLEY)\ + \\ &\quad {\beta}_{3}(HLVA \times SHIPLEY) + \epsilon \end{aligned} \]

  • \(\beta_0\) the intercept
  • \(\beta_1X\) the effect of HLVA
  • \(\beta_2Z\) the effect of SHIPLEY
  • \(\color{red}{\beta_3XZ}\) the effect of HLVA:SHIPLEY the interaction

What is the interaction term doing?

\[y \sim \beta_0 + \beta_1X + \beta_2Z + \color{red}{\beta_3XZ} + \epsilon\]

\[ mean.acc = \alpha + \beta_{1}(HLVA) + \beta_{2}(SHIPLEY) + \beta_{3}(HLVA \times SHIPLEY) + \epsilon \]

  • Here, \(\beta_1\) reflects the influence of HLVA on outcome mean.acc when SHIPLEY = 0.
  • \(\beta_2\) reflects the influence of SHIPLEY on outcome mean.acc when HLVA = 0.
  • \(\color{red}{\beta_3}\) reflects the change in the effect of HLVA on outcome mean.acc, given different values of SHIPLEY

What is the interaction term doing?

The key idea here is that the effect of one predictor variable (HLVA) on the outcome is a function of another predictor variable (SHIPLEY)

  • The slope \(\beta_1\) of the effect of the predictor \(HLVA\) changes
  • by the amount \(\beta_3\), for unit change in \(SHIPLEY\) values

What is the interaction term doing?

We can state an equation for calculating the slope of the predicted effect of \(HLVA\) on outcome \(mean.acc\) at any value of \(SHIPLEY\) as:

\[The \space HLVA \space slope \space at \space some \space SHIPLEY \space score = \beta_1 + \beta_3 \times SHIPLEY\] In the presence of an interaction, the effect of the predictor HLVA is a composite reflecting

  • the impact of variation in health literacy (\(\beta_1\))
  • and how that effect is modified (\(\beta_3\)) for different values of the moderator SHIPLEY

Interactions are symmetrical

  • In our example, we started by looking at the HLVA*SHIPLEY interaction
  • But equally we could consider the SHIPLEY*HLVA interaction
model <- lm(mean.acc ~ SHIPLEY*HLVA, data = all.data)
summary(model)

Call:
lm(formula = mean.acc ~ SHIPLEY * HLVA, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.37986 -0.06840  0.01342  0.07652  0.29872 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.4806201  0.1445375   3.325 0.000952 ***
SHIPLEY      0.0045132  0.0043491   1.038 0.299919    
HLVA         0.0153042  0.0168363   0.909 0.363810    
SHIPLEY:HLVA 0.0001132  0.0004922   0.230 0.818173    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1128 on 474 degrees of freedom
Multiple R-squared:  0.2004,    Adjusted R-squared:  0.1953 
F-statistic: 39.59 on 3 and 474 DF,  p-value: < 2.2e-16
Figure 9: The relation between SHIPLEY and mean.acc for different ranges of HLVA scores

Evaluation of an interaction

  • In our example, there is some variation in the slope of the HLVA effect for different ranges of SHIPLEY
  • And some variation in the slope of the SHIPLEY effect for different ranges of HLVA
  • But either way the interaction is not significant: p > .05 for the interaction
model <- lm(mean.acc ~ HLVA*SHIPLEY, data = all.data)
summary(model)

Call:
lm(formula = mean.acc ~ HLVA * SHIPLEY, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.37986 -0.06840  0.01342  0.07652  0.29872 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.4806201  0.1445375   3.325 0.000952 ***
HLVA         0.0153042  0.0168363   0.909 0.363810    
SHIPLEY      0.0045132  0.0043491   1.038 0.299919    
HLVA:SHIPLEY 0.0001132  0.0004922   0.230 0.818173    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1128 on 474 degrees of freedom
Multiple R-squared:  0.2004,    Adjusted R-squared:  0.1953 
F-statistic: 39.59 on 3 and 474 DF,  p-value: < 2.2e-16
Figure 10: The relation between HLVA and mean.acc for different ranges of SHIPLEY scores: left plot (scores between 19-33); middle (scores between 33-37); and right (scores between 37-40)

Let’s take a break

  • End of part 3

Interactions between numeric variables and factors

data.lm.acc <- lm(mean.acc ~ EDUCATION*strategy, 
                  data = all.data)
summary(data.lm.acc)

Call:
lm(formula = mean.acc ~ EDUCATION * strategy, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.43422 -0.06111  0.01700  0.09208  0.23754 

Coefficients:
                          Estimate Std. Error t value Pr(>|t|)    
(Intercept)               0.692571   0.054831  12.631   <2e-16 ***
EDUCATIONHigher          -0.199280   0.083325  -2.392   0.0172 *  
strategy                  0.002210   0.001100   2.008   0.0452 *  
EDUCATIONHigher:strategy  0.004104   0.001656   2.479   0.0135 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1223 on 474 degrees of freedom
Multiple R-squared:  0.06106,   Adjusted R-squared:  0.05512 
F-statistic: 10.28 on 3 and 474 DF,  p-value: 1.45e-06

The interaction between the effect of reading strategy on comprehension accuracy and the effect of differences in education level

Interactions between numeric variables and factors

Tip

  • While interactions between numeric variables and factors work in the same way
  • As the interactions between numeric variables (like HLVA and SHIPLEY) that we have been working with so far
  • You will see some important differences

Interactions between numeric variables and factors

To explore interactions between numeric variables and factors, we take as an example, the interaction between the effect on outcomes (mean.acc) of:

  • Variation in reading strategy (measured as scores on the strategy reading strategy questionnaire), a numeric variable
  • Variation in level of education (measured by self-report on EDUCATION as Further or Higher), a factor

Numeric variables and factors

Two kinds of variables are treated differently in R:

  1. Numeric variables like health literacy score (HLVA) or reading strategy score (strategy) where
  • the column values are numbers you can do arithmetic with
Figure 11: Scatterplots showing the potential association between accuracy of comprehension and variation on (left) health literacy, (middle) reading strategy, or (right) education level

Numeric variables and factors

Two kinds of variables are treated differently in R:

  1. Categorical variables or factors like EDUCATION
  • where values may be numbers or words or letters
  • and you use different values to code for group or category e.g. EDUCATION level can be Further or Higher
Figure 12: Scatterplots showing the potential association between accuracy of comprehension and variation on (left) health literacy, (middle) reading strategy, or (right) education level

The effects of factors in linear models

  • A factor like EDUCATION encodes differences between levels or between groups
  • A participant can be coded as having one level only
  • A participant can be a member of one group only
data.lm.acc <- lm(mean.acc ~ EDUCATION + strategy, 
                  data = all.data)
summary(data.lm.acc)

Call:
lm(formula = mean.acc ~ EDUCATION + strategy, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.42828 -0.06774  0.02177  0.09198  0.31055 

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)     0.6032460  0.0415482  14.519  < 2e-16 ***
EDUCATIONHigher 0.0053666  0.0113014   0.475    0.635    
strategy        0.0040226  0.0008266   4.866 1.55e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1229 on 475 degrees of freedom
Multiple R-squared:  0.04889,   Adjusted R-squared:  0.04489 
F-statistic: 12.21 on 2 and 475 DF,  p-value: 6.75e-06

The effects of factors in linear models

  • If you include a factor like EDUCATION in a linear model
  • The model will give you an estimate of the effect of differences between the levels of the factor
  • R will pick one level as the baseline or reference level
  • and the coefficient estimate will tell you the difference in outcome on average
  • between the reference level and the other factor level(-s)
data.lm.acc <- lm(mean.acc ~ EDUCATION + strategy, 
                  data = all.data)
summary(data.lm.acc)

Call:
lm(formula = mean.acc ~ EDUCATION + strategy, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.42828 -0.06774  0.02177  0.09198  0.31055 

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)     0.6032460  0.0415482  14.519  < 2e-16 ***
EDUCATIONHigher 0.0053666  0.0113014   0.475    0.635    
strategy        0.0040226  0.0008266   4.866 1.55e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1229 on 475 degrees of freedom
Multiple R-squared:  0.04889,   Adjusted R-squared:  0.04489 
F-statistic: 12.21 on 2 and 475 DF,  p-value: 6.75e-06

The effects of factors in linear models

  • Here, the Coefficients: Estimate for the effect of EDUCATION is 0.0053666
  • The summary shows the coefficient as EDUCATIONHigher
  • Because the estimate 0.0053666 tells us that
  • compared to Further education (the reference level)
  • people with Higher education are predicted to score 0.0053666 more on outcome mean.acc
data.lm.acc <- lm(mean.acc ~ EDUCATION + strategy, 
                  data = all.data)
summary(data.lm.acc)

Call:
lm(formula = mean.acc ~ EDUCATION + strategy, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.42828 -0.06774  0.02177  0.09198  0.31055 

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)     0.6032460  0.0415482  14.519  < 2e-16 ***
EDUCATIONHigher 0.0053666  0.0113014   0.475    0.635    
strategy        0.0040226  0.0008266   4.866 1.55e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1229 on 475 degrees of freedom
Multiple R-squared:  0.04889,   Adjusted R-squared:  0.04489 
F-statistic: 12.21 on 2 and 475 DF,  p-value: 6.75e-06

Coding an interaction

We can code the effect of the interaction between the effects of:

  • education level (EDUCATION)
  • reading strategy (strategy)
data.lm.acc <- lm(mean.acc ~ EDUCATION*strategy, 
                  data = all.data)
summary(data.lm.acc)

What is the interaction term doing?

\[y \sim \beta_0 + \beta_1X + \beta_2Z + \color{red}{\beta_3XZ} + \epsilon\]

\[ \begin{aligned} mean.acc &= {\beta_0} + {\beta}_{1}(EDUCATION_{Higher}) + {\beta}_{2}(strategy)\ + \\ &\quad {\beta}_{3}(EDUCATION_{Higher} \times strategy) + \epsilon \end{aligned} \]

  • Here, \(\beta_1\) reflects the influence of strategy on outcome mean.acc when EDUCATION = Further (the baseline)
  • \(\beta_2\) reflects the influence of EDUCATION (Further versus Higher) on outcome mean.acc when strategy = 0

What is the interaction term doing?

\[y \sim \beta_0 + \beta_1X + \beta_2Z + \color{red}{\beta_3XZ} + \epsilon\]

\[ \begin{aligned} mean.acc &= {\beta_0} + {\beta}_{1}(EDUCATION_{Higher}) + {\beta}_{2}(strategy)\ + \\ &\quad {\beta}_{3}(EDUCATION_{Higher} \times strategy) + \epsilon \end{aligned} \]

  • \(\beta_3\) reflects the change in the effect of strategy on outcome mean.acc
  • given different values of EDUCATION:
  • here, when EDUCATION is Higher instead of Further

Identifying key information in results: for a model including an interaction

You see each Estimate for the Coefficients: – the effects of:

  1. EDUCATION
  2. strategy
  3. The EDUCATION:strategy coefficient for the interaction tells us how the slope of the strategy effect varies – depending on EDUCATION level
data.lm.acc <- lm(mean.acc ~ EDUCATION*strategy, 
                  data = all.data)
summary(data.lm.acc)

Call:
lm(formula = mean.acc ~ EDUCATION * strategy, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.43422 -0.06111  0.01700  0.09208  0.23754 

Coefficients:
                          Estimate Std. Error t value Pr(>|t|)    
(Intercept)               0.692571   0.054831  12.631   <2e-16 ***
EDUCATIONHigher          -0.199280   0.083325  -2.392   0.0172 *  
strategy                  0.002210   0.001100   2.008   0.0452 *  
EDUCATIONHigher:strategy  0.004104   0.001656   2.479   0.0135 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1223 on 474 degrees of freedom
Multiple R-squared:  0.06106,   Adjusted R-squared:  0.05512 
F-statistic: 10.28 on 3 and 474 DF,  p-value: 1.45e-06

Evaluation of an interaction

Interactions are easier to interpret if we plot them:

  1. Fit the model using lm()
  2. Feed the model estimates into plot_model() to show predictions
# -- fit a model
data.lm.acc <- lm(mean.acc ~ EDUCATION*strategy, 
                  data = all.data)

# -- set colour values for manual colour
cols.studies <- c("#BAE4B3", "#6BAED6") 

# -- make plot
plot_model(data.lm.acc, type = "pred", 
           terms = c("strategy", "EDUCATION"),
           colors = cols.studies) +
  theme_bw() +
  ylim(0, 1)

Interactions between numeric variables and factors

The resulting plot shows how the slope of the strategy effect is steeper (the effect is larger) for people with Higher education than with Further education

data.lm.acc <- lm(mean.acc ~ EDUCATION*strategy, 
                  data = all.data)
summary(data.lm.acc)

Call:
lm(formula = mean.acc ~ EDUCATION * strategy, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.43422 -0.06111  0.01700  0.09208  0.23754 

Coefficients:
                          Estimate Std. Error t value Pr(>|t|)    
(Intercept)               0.692571   0.054831  12.631   <2e-16 ***
EDUCATIONHigher          -0.199280   0.083325  -2.392   0.0172 *  
strategy                  0.002210   0.001100   2.008   0.0452 *  
EDUCATIONHigher:strategy  0.004104   0.001656   2.479   0.0135 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1223 on 474 degrees of freedom
Multiple R-squared:  0.06106,   Adjusted R-squared:  0.05512 
F-statistic: 10.28 on 3 and 474 DF,  p-value: 1.45e-06

The interaction between the effect of reading strategy on comprehension accuracy and differences in education level


Call:
lm(formula = mean.acc ~ EDUCATION * strategy, data = all.data)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.43422 -0.06111  0.01700  0.09208  0.23754 

Coefficients:
                          Estimate Std. Error t value Pr(>|t|)    
(Intercept)               0.692571   0.054831  12.631   <2e-16 ***
EDUCATIONHigher          -0.199280   0.083325  -2.392   0.0172 *  
strategy                  0.002210   0.001100   2.008   0.0452 *  
EDUCATIONHigher:strategy  0.004104   0.001656   2.479   0.0135 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.1223 on 474 degrees of freedom
Multiple R-squared:  0.06106,   Adjusted R-squared:  0.05512 
F-statistic: 10.28 on 3 and 474 DF,  p-value: 1.45e-06

We fitted a linear model with mean comprehension accuracy as the outcome and education and reading strategy as predictors. The model is significant overall, \(F(3, 474) = 10.28, p < .001\), and explains 6% of variance (\(\text{adjusted } R^2 = .06\)). Accuracy of comprehension was lower for people with higher levels of participant education (\(\beta = -.199, t = -2.39, p = .017\)). But comprehension was more accurate for people with better reading strategy (\(\beta = .002, t = 2.01, p = .045\)). A significant interaction (\(\beta = .004, t = 2.48, p = .014\)) suggests the strategy effect was larger for participant with higher education.

The language of results reporting: Look at what we do with the text

We fitted a linear model with mean comprehension accuracy as the outcome and education and reading strategy as predictors. The model is significant overall, \(F(3, 474) = 10.28, p < .001\), and explains 6% of variance (\(\text{adjusted } R^2 = .06\)). Accuracy of comprehension was lower for people with higher levels of participant education (\(\beta = -.199, t = -2.39, p = .017\)). But comprehension was more accurate for people with better reading strategy (\(\beta = .002, t = 2.01, p = .045\)). A significant interaction (\(\beta = .004, t = 2.48, p = .014\)) suggests the strategy effect was larger for participant with higher education.

  1. Explain: the method (linear model); the outcome (accuracy) and the predictors
  2. Report the model fit statistics overall (\(F, R^2\))
  3. Report the significant effects (\(\beta, t, p\)) and describe the effects

Health comprehension project: answers

Note

  1. What person attributes predict success in understanding?
  • Health literacy, vocabulary, and reading strategy
  1. Can people accurately evaluate whether they correctly understand written health information?
  • Yes but not very well

Summary: key lessons

Understanding key concepts

  • Psychologists often assume, or they are interested in, the effects of experimental manipulations, of the effects of treatments or interventions, or the effects of associations between variables
  • The key lesson here is that people vary, and effects may vary

Understanding key concepts

  • Effects can be different between different population groups
  • Effects can be different for different values of a third variable (that third variable is often called a moderator)
  • Where we see that effects are different between different sub-groups or where we see that the effect of an association is different for different levels of a moderator then there we are looking at an interaction

Understanding key coding skills

We typically code models like those we have seen using the structure:

lm(outcome.Y ~ predictor.X*predictor.Z)

End of week 19

References

Bornstein, M. H., Jager, J., & Putnick, D. L. (2013). Sampling in developmental science: Situations, shortcomings, solutions, and standards. Developmental Review, 33(4), 357–370. https://doi.org/10.1016/j.dr.2013.08.003
Freed, E. M., Hamilton, S. T., & Long, D. L. (2017). Comprehension in proficient readers: The nature of individual variation. Journal of Memory and Language, 97, 135–153. https://doi.org/10.1016/j.jml.2017.07.008
Gelman, A. (2015). The Connection Between Varying Treatment Effects and the Crisis of Unreplicable Research: A Bayesian Perspective. Journal of Management, 41(2), 632–643. https://doi.org/10.1177/0149206314525208
McElreath, R. (2020). Statistical rethinking. Chapman; Hall/CRC. https://doi.org/10.1201/9780429029608