Teaching Item Bank

Random assignment to conditions primarily protects which type of validity?

1. External validity
2. Internal validity
3. Construct validity
4. Statistical conclusion validity

Random assignment equates groups in expectation, ruling out confounds and supporting causal (internal-validity) claims.

A researcher studies whether income predicts well-being using survey data with no manipulation. This is best described as:

1. A true experiment
2. An observational (correlational) design
3. A randomized controlled trial
4. A factorial experiment

With no manipulation or random assignment, the design is observational and supports association but not strong causal claims.

Selecting participants because they scored extremely low and retesting them later risks which threat?

1. Maturation
2. Regression to the mean
3. Instrumentation
4. Demand characteristics

Extreme scores tend to move toward the mean on retest regardless of any intervention.

The degree to which findings generalize to other people and settings is:

1. Internal validity
2. External validity
3. Face validity
4. Reliability

External validity concerns generalization across populations, settings, and times.

Differential attrition across conditions is a threat mainly because it can:

1. Increase statistical power
2. Make groups non-equivalent by the end of the study
3. Improve construct validity
4. Eliminate confounds

If dropout differs by condition, the groups are no longer equivalent, reintroducing confounding.

A p-value of .03 means:

1. There is a 3% chance the null hypothesis is true
2. Assuming the null is true, data this extreme (or more) occur 3% of the time
3. The effect is large
4. The result will replicate 97% of the time

A p-value is computed assuming the null is true; it is the probability of data at least as extreme as observed.

Failing to reject a false null hypothesis is a:

1. Type I error
2. Type II error
3. Sampling error
4. Measurement error

A Type II error (beta) is a false negative — missing a true effect.

Statistical power is defined as:

1. 1 − alpha
2. 1 − beta
3. The p-value
4. The effect size

Power = 1 − beta, the probability of detecting a true effect of a given size.

Which Cohen's d is conventionally considered a 'medium' effect?

1. 0.20
2. 0.50
3. 0.80
4. 1.00

Cohen's conventions: d ≈ .2 small, .5 medium, .8 large.

A 95% confidence interval is best interpreted as:

1. A 95% probability the true value lies in this specific interval
2. A range of plausible parameter values; 95% of such intervals capture the parameter over repeated sampling
3. The range containing 95% of the data
4. The standard error times 95

The 95% refers to the long-run capture rate of the procedure across repeated samples.

Cronbach's alpha is an index of:

1. Criterion validity
2. Internal-consistency reliability
3. Test difficulty
4. External validity

Alpha summarizes how consistently a set of items measure the same thing.

Reliability and validity are related such that:

1. A valid test must be reliable
2. A reliable test must be valid
3. They are unrelated
4. Validity caps reliability

Reliability sets a ceiling on validity; a measure cannot be valid for a purpose if it is not reliable.

Convergent and discriminant evidence are forms of:

1. Content validity
2. Construct validity
3. Inter-rater reliability
4. Face validity

Convergent (relates to similar constructs) and discriminant (unrelated to different constructs) evidence support construct validity.

An item-total correlation near zero suggests the item:

1. Is too difficult
2. Does not cohere with the rest of the scale
3. Has high discrimination
4. Is perfectly reliable

Low item-total correlation indicates the item is not measuring the same construct as the scale.

A T-score has a mean and standard deviation of:

1. 0 and 1
2. 50 and 10
3. 100 and 15
4. 5 and 2

T-scores are standardized to M = 50, SD = 10.

A score above a depression screener's cut-off indicates:

1. A confirmed diagnosis
2. That a fuller clinical evaluation may be warranted
3. Treatment is unnecessary
4. The person is malingering

Screeners flag the possible need for further assessment; they do not diagnose.

Specificity of a screening test refers to:

1. The proportion of true cases correctly identified
2. The proportion of non-cases correctly identified
3. The total accuracy
4. The base rate of the disorder

Specificity is the true-negative rate — non-cases correctly screened out.

Combining self-report with a performance task is an example of:

1. Mono-method assessment
2. Multi-method assessment
3. Criterion contamination
4. Norm referencing

Using different methods reduces shared-method bias and strengthens conclusions on convergence.

Reporting a confidence range around a score rather than a single number reflects attention to:

1. Measurement error
2. Random assignment
3. Demand characteristics
4. Publication bias

Every observed score contains measurement error; a confidence range communicates that uncertainty.

Norm-referenced interpretation is valid only when:

1. The test is brand new
2. The examinee resembles the standardization sample
3. The score is above average
4. No cut-off is used

Norms generalize only to populations resembling the sample on which they were established.