Home/Insights/Methodology reference/Psychology research methods
Psychology research methods
Psychology’s methods are shared with much of the life sciences; what is distinctive is how publicly the field has had to confront what happens when they are applied loosely.
The main designs
Experimental, with random assignment to condition, supporting causal inference. Quasi-experimental, where the grouping is pre-existing — age, diagnosis, gender — since these cannot be assigned. Correlational, establishing association. Observational and qualitative, for behaviour in context and for meaning.
Within-subject designs give greater power because each participant is their own control, at the cost of order effects, which counterbalancing addresses.
Measurement is the hard part
Psychology measures constructs that cannot be observed directly, so an instrument’s validity is a claim requiring evidence: content, construct, criterion. Reliability covers internal consistency, test–retest stability and inter-rater agreement.
Cronbach’s alpha is routinely over-interpreted. It depends on the number of items as well as their intercorrelation, a high value does not establish unidimensionality, and it is a property of the scores in this sample rather than of the scale itself. It should be reported for the present data, not quoted from the original validation paper.
What the replication crisis established
Large-scale replication efforts found that a substantial proportion of published psychological findings did not replicate at the original effect size. The causes identified were structural rather than individual: analytic flexibility, selective reporting of outcomes and conditions, underpowered studies, and publication bias toward positive results.
The remedies now expected by many journals are preregistration of hypotheses and analysis plans, registered reports reviewed before data collection, power analysis conducted in advance, open data and materials, and reporting of effect sizes with confidence intervals rather than p-values alone.
This matters beyond psychology; the same practices have since been adopted across the biomedical and social sciences.
Ethics
Informed consent, the right to withdraw, debriefing where any concealment was used, and particular care with vulnerable groups. Deception requires specific justification and ethics committee approval.
In practice
Fix, flag or query.
The distinction that keeps an edit defensible: what an editor may change silently, what must be shown to the author, and what only the author can answer.
- Report: alpha for the present sample, not from the validation study.
- Conduct: power analysis before data collection, not after.
- Preregister: hypotheses and analysis plan where the design allows.
Reporting standards
- APA Journal Article Reporting Standards (JARS)
- Open Science Framework, on preregistration and registered reports
Reporting guidelines named here are hosted by the EQUATOR Network. Writing to the checklist for your design before submission is the cheapest available improvement to a methods section.
Related