Week 6. The structured research report – Quick start

Written by Rob Davies

6.1 Overview: Quick start

The structured research report assignment: We want you to analyse previously collected data and write a report about your findings.

Here, we present a guide in five parts:

Tip

What you have to write.
How you can do the analysis work.
Why we are asking you to do this.
Where you can get in-depth information.
How you can expect us to grade your work.

We provide a summary guide to What you have to write next, and then a quick guide to How you can do the analysis work you need to do, so that you can do the writing.

We share a lecture on Why we are asking you to do this work: what you will learn and how you will benefit.

At the end of this page, in How we will mark your work, we detail what work of different grades looks like, so that you can both understand what we want you to do, and how you can do it.

If you want more in-depth guidance or support, we provide detailed step-by-step notes on the work you can plan to do in a chapter we have written to support you: How you can do the analysis work. If you want an explanation of why we are asking you to do this work, so you can think about best practice in science, we have got that for you in our chapter: Why we are asking you to do this.

6.2 What you have to write

Let’s begin with what you need to write to complete this coursework.

Submission deadline: On Friday 17th January 2025
You will submit a structured report.
Presenting an analysis: what you did; what you found; the context; and the implications.

Tip

You will be able to find the submission point on Moodle here when it is revealed.

What is a structured report?

You will submit short answers to a series of questions:

Describe the questions or the predictions you examine in your analysis.
Explain the research background: you are analyzing previously collected data, so briefly explain why the people who originally collected the data did that work.
Explain if you are attempting to repeat the analysis that the original researchers did or if you are doing something different.
Explain the motivation for the questions or the predictions your analysis examines.
Summarize the methods that were used to collect the data you analyzed.
Identify what variables are included in the analysis: what variable is the outcome (or dependent) variable, and what variables are the predictor (or independent) variables.
Describe the analysis method you use to address the question or test the predictions.
Explain your analysis method choices: why are you using this method?
Identify the model you use in your analysis.
Present a summary of the analysis results: use text and plots to show what you found.
Explain the implications of the results, in theoretical or in policy terms.
Critically evaluate your findings: reflect on the strength or the limitations of the evidence you present, as answers to the questions or as tests of the predictions you outlined.
Predict: how can future research build on the work you have done?

Tip

You will write each answer in sentences, organized in one or more paragraphs.
You will submit your analysis code in an appendix.
Where appropriate, you will include plots.
Submit your report as a series of answers to the questions listed in a single document.
You can use each question as a heading in the document.
Word count limit: no more than 1500 words are allowed for all materials except references, appendices, and the content of tables.

6.3 How you can do the analysis work

Reports will concern, usually, findings from analyses of data collected in previous studies or data accessed from online sources. These data will usually be associated with a published report in a journal like Psychological Science or a pre-print archive like PsyArXiv.

We expect students to use one of the analysis methods taught in the module.

Tip

If you want to know more about how to do this work, you can read in-depth step-by-step guidance in the how-to chapter.

You can see information on APA formatting of statistics and numbers in the OWL Purdue guide. Though the APA guidelines are the authoritative guide.

6.3.1 Report checklist

Checklist

To help you organize your work in preparing for the research report assignment, we share a downloadable checklist: PSYC411-research-report-checklist.

6.4 Why we are asking you to do this

We are asking you to do the structured research report because doing the work — the analysis and the writing — will build your awareness and understanding of how psychological science really works, and will strengthen your sense of what best practice looks like in modern science.

6.5 Lectures and slides

6.5.1 Lecture recordings – videos

Click on a link and your browser should open a tab showing the Panopto video for the lecture part. You should be able to access the videos anywhere; you should not need to be on campus or logged on to the university VPN to view the videos.

Overview (20 minutes): the key ideas, the scientific context

Overview (20 minutes): the research workflow, multiverse analyses

Overview (20 minutes): kinds of reproducibility, open data, and doing better science.

6.5.2 Lecture recordings – slides

You can download the lecture slides as a single downloadable .html file that you can open in any browser: 411-research-report.html. This can be opened in a browser and presents the slides as they are delivered.

6.6 How we will mark your work

A distinction requires the following.

A. Background and methods

There should be a coherent, logical, argument for how the background or context leads to the questions or predictions.
There should be a concise explanation of the background or context for the research: Why did the original study authors do the research? Why are you doing the analysis you report?
There should be a specific concrete statement of the research question that your analysis addresses, or the hypothesis that your analysis tests.
In the method information, there should be a concise summary of the properties of the variables — how data were collected, how variable values were coded or scored — so that the reader can understand the structure of the data.

B. Analysis and results

There should be a clear account of which variables are included in the analyses, so that the reader can understand what outcome (dependent) variable is being analyzed using what predictor (independent) variables.
There should be a scholarly explanation of the analysis you did, including an account of why you chose to use the method you use, given possible alternatives and, if relevant, referring to justifications for your choice that are identified in the statistical literature.
There should be a clear identification of what analysis you did, using what method, outlining the elements and structure of the model whose results you present. -There should be a clear presentation of the results, presenting a correct interpretation of statistical information concerning the size, the direction (or sign), and the significance of any effect you estimate or difference you test. The presentation should explain how the outcome is expected to vary, on average, given the differences or the effects estimated using your analyses.
There should be full and correct use of APA conventions in the presentation of statistical results.
Where appropriate, results are presented using both text and relevant, informative, visualizations.

C. Implications and critical reflection

There should be an accurate account of whether or how the analysis results address the questions, or how the analysis results support or disconfirm the predictions stated.
There should be a scholarly explanation of the implications of the results: what do the results suggest we now know that we did (or did not) know before?
There should be a critical analysis of the strengths or limitations of the evidence: how confident can we be, how uncertain should we be, that the results that are presented do or do not suggest what you think they suggest.
There should be a constructive analysis of how future research could build on your findings to further extend understanding or evaluate methodological concerns.

A merit will be awarded, by comparison, if:

A. Background and methods

There is a concise explanation of the background or context for the research. There is limited or partial information on why the original study authors did the research, or why you are you doing the analysis you report. We do not see an argument explaining the reasons motivating your research.
There is a clear statement of the research question, or the hypothesis or prediction that your analysis tests. But the question or the prediction may be vague, or stated in general not specific terms.
The summary of the properties of the variables may not be sufficiently clear or informative about the structure of the data that the reader can understand what data are involved in analyses.

B. Analysis and results

There is a clear account of which variables are included in the analyses, so that the reader can understand what outcome (dependent) variable is being analyzed using what predictor (independent) variables.
There is an explanation of the analysis you did, but there is limited explanation of why you chose to use the method you use. There is no or there is limited awareness of possible alternatives. No informed justification is presented for analysis choices.
There is a clear identification of what analysis you did, using what method, outlining the elements and structure of the model whose results you present.
There is evidence that the analysis method you chose was used correctly, that the method was appropriate to the question and the data, and that the code you used could do the analysis you say it did.
There is a clear presentation of the results, presenting a correct interpretation of statistical information concerning the size, the direction (or sign), and the significance of any effect you estimate or difference you test. The presentation should explain how the outcome is expected to vary, on average, given the differences or the effects estimated using your analyses.
There is full and correct use of APA conventions in the presentation of statistical results.
Where appropriate, results are presented using both text and relevant, informative, visualizations.

C. Implications and critical reflection

There is an accurate account of whether or how the analysis results address the questions, or how the analysis results support or disconfirm the predictions stated.
There is some explanation of the implications of the results but this account is not linked to previous research (in the literature).
There is limited reflection on the strengths or limitations of the evidence. We may see generic but not well informed discussion of the evidence specific to your analysis or your data.
There is limited discussion of future research.

A pass will be awarded, by comparison, if:

A. Background and methods

There is a concise explanation of the background or context for the research. There is limited or no information on why the original study authors did the research or why you are you doing the analysis you report.
There is a statement of the research question, or the hypothesis or prediction that your analysis tests. But the question or the prediction may be vague, or stated in general terms only.
The summary of the properties of the variables provides information on the structure of the data but it may be unclear how one or more variables were coded or scored. There is limited engagement with questions of measurement reliability or validity.

B. Analysis and results

There is a clear account of which variables are included in the analyses, so that the reader can understand what outcome (dependent) variable is being analyzed using what predictor (independent) variables.
There is an explanation of the analysis you did, but there is no explanation of why you chose to use the method you use. Limited or no informed justification is presented for analysis choices.
There is a clear identification of what analysis you did, using what method, outlining the elements and structure of the model whose results you present.
There is evidence that the analysis method you chose was used correctly, that the method was appropriate to the question and the data, and that the code you used could do the analysis you say it did.
There is a clear presentation of the results, presenting a correct interpretation of statistical information concerning the direction (or sign) and the significance of any effect you estimate or difference you test. The presentation should explain how the outcome is expected to vary, on average, given the differences or the effects estimated using your analyses.
There may be partial use of APA conventions in the presentation of statistical results.
Appropriate visualizations may be included but without comment or discussion.

C. Implications and critical reflection

There is a summary account of the results but it may not clearly explain whether or how the analysis results address the questions, or how the analysis results support or do not support the predictions stated.
There is limited or no explanation of the implications of some of the results.
There is limited or no reflection on the strengths or limitations of the evidence.
There is no discussion of future research.