ANOVA

Purpose:

An analysis of variance, used to determine if there are any statistically significant differences between the means of three or more independent groups. It does not specify "how" they differ from each other (a post-hoc test can answer this!). There are three kinds of ANOVA: a one-way ANOVA (for between-subjects manipulation where the IV has 3+ levels), repeated measures ANOVA (for within-subjects manipulation where the IV has 3+ levels), and factorial ANOVA (where there are 2+ IVs).

Note: Between-group manipulation (between-subjects study design) means that each group received a different IV treatment condition (level). Within-group manipulation (within-subjects study design) means that every participant experiences every IV treatment condition (level).

Context Used:

  • There must be ONE CATEGORICAL IV with 3+ levels, and ONE CONTINUOUS DV, OR
  • There must be TWO OR MORE CATEGORICAL IVs with 2+ levels, and ONE CONTINUOUS DV
Do NOT use ANOVA if: Your IV has only two levels → a t-test is sufficient | Your DV is categorical → use chi-square | You want to predict rather than compare means → use regression | You have a continuous IV → use regression or correlation instead

Assumptions:

  • Normality: The DV should be approximately normally distributed within each group.
  • Independence: Observations are independent of each other.
  • Homogeneity of variances: Check with Levene's test. If p < .05, use Welch's ANOVA.
  • Sphericity (repeated measures only): Variances of the differences between all pairs of conditions should be equal. Check with Mauchly's test. If violated, apply Greenhouse-Geisser correction.

Jamovi Walkthrough:

One-Way ANOVA:

Data setup: Column 1 = IV levels (coded numerically), Column 2 = DV scores.
  1. Click "ANOVA"
  2. Click "ANOVA"
  3. Move the DV into the "Dependent Variable" box
  4. Move the IV into the "Fixed Factors" box
  5. Check "η²" under "Effect Size"
  6. Check "Homogeneity test" in the "Assumption Checks" menu
  7. Move the IV into the right box in the "Post Hoc Tests" menu
  8. Check "Tukey" under "Corrections" and "Cohen's d" under "Effect Size" in the "Post Hoc Tests" menu

Repeated Measures ANOVA:

Data setup: Each column represents DV scores at one IV level (one column per condition or time point).
  1. Click "ANOVA"
  2. Click "Repeated Measures ANOVA"
  3. Label the IV in the bolded text box under "Repeated Measures Factors"
  4. Label the IV levels in the remaining text boxes under "Repeated Measures Factors"
  5. Move the IV levels from the left box into the corresponding box under "Repeated Measures Cells"
  6. Label the DV in the text box under "Dependent Variable Label"
  7. Check "η²" under "Effect Size"
  8. Check "Homogeneity test" and "Greenhouse-Geisser" under "Assumption Checks"
  9. Move the IV into the right box in the "Post Hoc Tests" menu; check "Tukey"

Factorial ANOVA:

Data setup: Column 1 = IV1 levels, Column 2 = IV2 levels, Column 3 = DV scores.
  1. Click "ANOVA"
  2. Click "ANOVA"
  3. Move the DV into the "Dependent Variable" box
  4. Move both IVs into the "Fixed Factors" box
  5. Check "η²" under "Effect Size"
  6. If any IV has 3+ levels or you reject the null hypothesis: move that IV into the right box in "Post Hoc Tests" and check "Tukey"
  7. Click "Exploration" → "Descriptives"; move DV into "Variables," move all IVs into "Split by" for cell means
  8. Repeat step 7 with only ONE IV in "Split by" at a time for marginal means

R Walkthrough:

One-Way ANOVA:

model <- aov(DV ~ IV, data = df)
summary(model)

effectsize::eta_squared(model)

model_aov <- aov(DV ~ Group, data = df)
PostHocTest(model_aov, method = "hsd")  # DescTools package

Repeated Measures ANOVA:

model <- aov(DV ~ Time + Error(ID/Time), data = df)
summary(model)

Factorial ANOVA:

model_2way <- aov(DV ~ Factor1 * Factor2, data = df)
summary(model_2way)

effectsize::eta_squared(model_2way, partial = TRUE)

PostHocTest(model_2way, method = "hsd")

Output Interpretation:

Levene's Test: Tests homogeneity of variances assumption.

  • If p < .05 → assumption violated; select Welch's test under "Variances" to continue
  • If p > .05 → assumption satisfied (this is what we want; continue as normal)

F-value: The ratio of variation between sample means to variation within the samples.

  • If the f-value is high, there is high variability and a lower p-value.
  • If the f-value is low, there is low variability.

p-value: The probability of detecting a meaningful difference when there is none. Looking for a small value (p < .05).

  • If p < .05 → reject the null hypothesis. There IS a difference.
  • If p > .05 → retain the null hypothesis. There is NO difference.

η² (eta-squared): The proportion of variance in the DV explained by the IV. Even a medium effect (.06) is considered meaningful in human subjects research, where many variables influence behavior simultaneously.

η²Interpretation
≈ .01Small effect
≈ .06Medium effect
≈ .14Large effect

"% of variance in [DV] is explained by [IV]."

Post-Hoc Comparisons:

Cohen's d: A measure of effect size between specific group pairs.

Cohen's dInterpretation
.20 – .49Small
.50 – .79Medium
≥ .80Large

p-value: Looking for a small value (p < .05).

  • If p < .05 → reject the null hypothesis. There IS a difference between this pairwise comparison.
  • If p > .05 → retain the null hypothesis. There is NO difference between this pairwise comparison.

APA Format:

Appropriate data visualization: Bar graphs (with error bars).

Sample table: https://apastyle.apa.org/style-grammar-guidelines/tables-figures/sample-tables#anova

Sample write-up:

A one-way ANOVA examined differences in [DV] across [levels of IV]. The test was statistically significant, F(, ) = , p < .05, η² = . A Tukey HSD post-hoc test indicated that [Group A] (M = , SD = ) scored significantly higher than [Group B] (M = , SD = ). [Group B] and [Group C] did not differ significantly.

Note: Plug in the appropriate test used, means, standard deviations, whether the test was significant, f-value, df, alpha and p-level, the strength of the relationship (η²), and Tukey test results.

↑