Better understanding the linear model

PSYC122: Classes in weeks 16-20

• My name is Dr Rob Davies, I am an expert in communication, individual differences, and methods

Week 17: Better understanding the linear model

Objectives: 1. Link together ideas on how to do psychological science

Objectives: 2. Strengthen your practice and build your independence

Targets for weeks 16-19: Concepts

We are working together to develop concepts:

1. Week 16 — Hypotheses, measurement and associations
2. Week 17 — Predicting people using linear models
3. Week 18 — Everything is some kind of linear model
4. Week 19 — The real challenge in psychological science

Targets for weeks 16-19: Skills

We are working together to develop skills:

1. Week 16 — Visualizing, estimating, and reporting associations
2. Week 17 — Using data to predict people
3. Week 18 — Going deeper on linear models
4. Week 19 — Evaluating evidence across multiple studies

Learning targets for this week

• Skills – Understand how to code: lm(mean.acc ~ SHIPLEY)
• Concepts – To answer questions like: Is comprehension success influenced by vocabulary knowledge?

Learning targets for this week

• We will learn how to do analysis in the context of a live research project: the health comprehension project

1. code linear models
2. identify and interpret model statistics
3. critically evaluate the results
4. communicate the results

Thinking about relationships in psychologial science

We often want to know about relationships

• Does variation in X predict variation in Y?
• What are the factors that influence outcome Y?
• Is a theoretical model consistent with observed behaviour?

Now consider our research aims in the context of the health comprehension project

Health comprehension project: questions and analyses

1. We want to know: What makes it easy or difficult to understand written health information?
2. So our research questions are:

• What person attributes predict success in understanding?
• Can people accurately evaluate whether they correctly understand written health information?

3. These kinds of research questions can be answered using methods like linear models

Context: Individual differences theory of comprehension success

• Understanding text depends on (1.) language experience and (2.) reasoning ability (Freed et al., 2017)

The measurement context: Where the data come from

• We measure reading comprehension: asking people to read text and then answer multiple choice questions
• We measure background knowledge: vocabulary knowledge (Shipley); health literacy (HLVA)

Reflect: The kinds of critical evaluation questions you can ask yourself

• Are multiple choice questions good ways to probe understanding? What alternatives are there?
• Are tests like the Shipley good measures of language knowledge? What do we miss?

Reflect: As we move into thinking about the data analysis, we need to identify our assumptions

1. validity: that differences in knowledge or ability cause differences in test scores
2. measurement: that this is equally true across the different kinds of people we tested
3. generalizability: that the sample of people we recruited looks like the population

We need to think about the derivation chain

Questions, assumptions, predictions

Link: concepts, questions \(\rightarrow\) assumptions \(\rightarrow\) testable predictions

1. concepts, questions: Can people accurately understand health guidance? \(\rightarrow\)
2. assumptions: People who know more about language should also present more accurate understanding \(\rightarrow\)
3. testable predictions: Higher levels of vocabulary should be associated with higher levels of comprehension accuracy: we expect to estimate a positive coefficient

One way of thinking about the association is to visualize it

• For each value of the predictor vocabulary
• Does the the value of the outcome accuracy
• Increase or decrease?

Let’s take a break

How do we estimate the association between two variables?

model <- lm(mean.acc ~ SHIPLEY, 
            data = clearly.one.subjects)
summary(model)

1. Specify the lm function and the model mean.acc ~ SHIPLEY
2. Specify what data we use data = clearly.one.subjects
3. Get the results summary(model)

The sentence structure of models in R

Take a good look:

lm(mean.acc ~ SHIPLEY ...)

You will see this sentence structure in coding for many different analysis types

• method(outcome ~ predictors)
• method could be aov, brm, lm, glm, glmm, lmer, t.test, cor.test

These coefficients build a line

• The line represents:
• our prediction for how the outcome varies on average
• given change in the predictor

So now we need to think about straight lines

1. You may remember from school that to draw a straight line you need four numbers:

\[y = a + bx\]

2. We calculate the height \(y\) by adding

• \(a\) the intercept, the value of y when \(x = 0\)
• to the product of \(b\) the coefficient for the slope of the line
• multiplied by \(x\) the value of the predictor variable

Let’s draw it

• Look at what we get if we draw the line using the linear model coefficients:
• \(= 0.449\) for the intercept, \(a\)
• \(= 0.011\) for the slope, \(b\)
• In the formula: \(y = 0.449 + 0.011x\)
• (I round the numbers to three decimal places.)

We can understand the line as representing a set of predictions

• To see how — we use the coefficients to predict just one potential outcome:
• the expected accuracy for someone with a vocabulary score of 20
• We do this using the formula:

\(\text{predicted y} = 0.449 + \text{0.011 } \times \text{ Shipley score of } 20\)

We can understand the line as representing a set of predictions

• Let’s expand our predictions

1. Predict accuracy given a Shipley score of 20

• \(y = 0.449 + 0.011 \times 20\)

2. Predict accuracy given a Shipley score of 30

• \(y = 0.449 + 0.011 \times 30\)

The linear model allows us to predict the average outcome we can expect given any value of the predictor

Figure 9: The predicted change in mean comprehension accuracy, given variation in vocabulary scores

Let’s take a break

We could draw a variety of different model prediction lines: how do we pick the right one?

We could draw a variety of different model prediction lines: how do we pick the right one?

We need to go back to the prediction model

• To calculate a predicted outcome value, we calculated it as: \(\text{predicted y} = 0.449 + 0.011 \times \text{ Shipley score } 20\)
• Assuming the linear model \(\text{predicted y} = intercept + \text{slope } \times \text{ vocabulary}\)
• But we missed a bit: error

Linear models are typically estimated given sample data

• Maybe you noticed that I talked about how the model allows us to predict
• how the outcome varies on average given different values of the predictor
• When we use a linear model to estimate the intercept and slope – to build the predictions – we fit a model to the sample data
• And no model will fit sample data perfectly

Linear models are typically estimated given sample data

Usually, this means there are differences between the expected outcomes that the model predicts and the observed outcomes

• So we often write the linear model like this: \(y = a + bx + \epsilon\)

1. The observed outcome \(y\) equals
2. the intercept \(a\)
3. plus the difference associated with a specific predictor value \(bx\)
4. plus some amount of mismatch or error \(\epsilon\), the difference between the observed outcome and the predicted outcome

We can derive the formulas used to calculate the estimates using calculus

• But we won’t
• Because the linear model calculations are done using matrix solution algorithms in R so we don’t have to

What do the prediction errors – the residuals – look like?

Figure 10: The predicted change in mean comprehension accuracy, given variation in vocabulary scores. Observed values are shown in orange-red. Predicted values are shown in blue

We typically assume that the residuals are normally distributed

• Some are positive: observed outcome larger than predicted outcome
• Some are negative: observed outcome smaller than predicted outcome
• The average of the residuals will be zero overall

So: We pick the line that minimizes the residuals – the mismatch between predicted and observed outcomes

Let’s take a break

The t-tests in the linear model

\[t = \frac{\beta_j}{s_{\beta_j}}\]

• For each coefficient, the t-test is used to evaluate if the coefficient \(\beta_j\) is significantly different from zero
• We assume the null hypothesis that the coefficient \(\beta_j\) is zero
• We do the test by comparing the estimated coefficient \(\beta_j\) with the standard error of the estimate

The t-tests in the linear model

\[t = \frac{\beta_j}{s_{\beta_j}}\]

• The standard error \(s_{\beta_j}\) indicates our uncertainty about the estimate
• Larger standard errors represent greater uncertainty

The t-tests in the linear model

\[t = \frac{\beta_j}{s_{\beta_j}}\]

• Standard errors can be calculated using information about:
• Error in the model — think of the distribution of residuals
• Variation of values in the predictor — how widely they range
• The sample size
• Standard errors will be smaller for the coefficients of effects that appear to have bigger impacts, in models that describe outcomes better, in larger samples

End of week 9