Research Methods & Statistics
Study design, variable types, and statistical test selection for psychological research. Based on graduate-level methodology curriculum.
Variable Types
Categorical Variables
Variables that place participants into distinct groups or categories. Also called nominal or grouping variables. Examples: diagnostic group (MDD vs. control), sex assigned at birth, attachment category (secure, anxious, avoidant, disorganized).
Categorical outcome examples: When your dependent variable is categorical, you’re asking whether group membership differs based on predictors. Appropriate analyses: chi-square, logistic regression, discriminant function analysis.
Dimensional Variables
Variables measured on a continuous scale with a range of values. Also called continuous, interval, or ratio variables. Examples: PHQ-9 total score, IQ composite, reaction time, defense maturity index.
Dimensional outcome examples: When your dependent variable is dimensional, you’re examining how scores vary across conditions or predictors. Appropriate analyses: t-test, ANOVA, multiple regression, HLM.
Study Designs
Between-Subjects Design
Different participants are assigned to different conditions or groups. Controls for order effects and practice effects, but requires larger samples and introduces individual-difference noise. Used in randomized controlled trials, group comparison studies.
Within-Subjects (Repeated Measures) Design
The same participants provide data across all conditions. More statistically powerful than between-subjects for the same N; eliminates individual-difference confounds. Introduces order effects and carryover effects that require counterbalancing or washout periods.
Mixed Design
At least one between-subjects factor and at least one within-subjects factor. Common in longitudinal clinical trials where treatment group is between-subjects and time of assessment is within-subjects. Analyzed with mixed-model ANOVA or linear mixed models (HLM/LME).
Statistical Test Selection
| Design | DV Type | Test |
|---|---|---|
| 2 independent groups | Continuous | Independent-samples t-test |
| 2 independent groups | Categorical | Chi-square / Fisher’s exact |
| 3+ independent groups | Continuous | One-way ANOVA |
| Pre/post (1 group) | Continuous | Paired-samples t-test |
| Factorial (2+ IVs) | Continuous | Factorial ANOVA / HLM |
| Multiple predictors | Continuous | Multiple regression |
| Multiple predictors | Categorical | Logistic regression |
| Latent structure | Continuous | EFA / CFA / SEM |
| Longitudinal multilevel | Continuous | Linear Mixed Effects Model |
Effect Sizes
- Cohen’s d: Standardized mean difference. Small = 0.2, medium = 0.5, large = 0.8.
- r / R²: Correlation and proportion of variance explained. r = .10 small, .30 medium, .50 large.
- η² / partial η²: Variance explained in ANOVA. η² = .01 small, .06 medium, .14 large.
- OR (odds ratio): Categorical outcomes. OR = 1 means no effect; OR = 2 means twice the odds.
Construct Validity Framework
Cronbach and Meehl’s (1955) construct validity framework requires that a psychological measure demonstrate: (1) convergent validity — correlates with theoretically related measures; (2) discriminant validity — does not correlate with theoretically unrelated measures; (3) internal consistency — items that should hang together do; (4) criterion validity — predicts meaningful real-world outcomes.
The multitrait-multimethod (MTMM) matrix operationalizes this framework, separating method variance from trait variance. Acceptable MTMM patterns show convergent validity coefficients that exceed both heterotrait-monomethod and heterotrait-heteromethod coefficients.