T-Tests

Purpose:

Compare the means between two groups. There are three kinds of t-tests: an independent samples t-test (for between-group manipulation), a paired samples t-test (for within-group manipulation), and one-sample t-tests (for one sample being compared to the population).

Note: Between-group manipulation (between-subjects study design) means that each group received a different IV treatment condition (level). Within-group manipulation (within-subjects study design) means that every participant experiences every IV treatment condition (level).

Context Used:

There must be only ONE CATEGORICAL IV, with only TWO LEVELS.
Additionally, there must be only ONE QUANTITATIVE DV.

Note: Categorical variables are variables where the data represents groups with no real numerical value (e.g., gender, car brand, hair color, education level). Quantitative variables are variables where the data represents amounts with a real numerical value (e.g., height, weight, age, time, speed, distance).

Do NOT use a t-test if: Your IV has more than two levels → use ANOVA | Your DV is categorical → use chi-square | You have multiple DVs → use MANOVA | You have covariates you need to control for → use ANCOVA or regression

Assumptions:

Before running a t-test, three assumptions should be met:

Normality: The DV should be approximately normally distributed within each group. In Jamovi, check this under "Assumption Checks" → "Normality test (Shapiro-Wilk)." If p < .05, normality is violated — use a non-parametric test instead.
Independence: Observations must be independent of one another (each participant contributes one score).
Homogeneity of variances (independent samples only): The two groups should have similar variances. Jamovi tests this with Levene's test. If p < .05, select "Welch's" correction under "Variances."

When to use a non-parametric alternative: If the normality assumption is violated, use: Independent samples → Mann-Whitney U | Paired samples → Wilcoxon Signed-Rank Test. Both are available in Jamovi under the T-Tests menu.

Jamovi Walkthrough:

Independent Samples T-Test:

Click "T-Tests"
Click "Independent Samples T-Test"
Enter the quantitative dependent variable into the "Dependent Variables" box
Enter the categorical independent variable in the "Grouping Variable" box
Check "Effect size" and "Confidence interval" under "Additional Statistics"

Paired Samples T-Test:

Click "T-Tests"
Click "Paired Samples T-Test"
Enter the two IV level score columns into the "Paired Variables" box
Check "Effect size" and "Confidence interval" under "Additional Statistics"

One-Sample T-Test:

Click "T-Tests"
Click "One Sample T-Test"
Enter the sample's DV scores into the "Dependent Variables" box
Enter the known population mean into the "Test value" field
Check "Effect size" and "Confidence interval" under "Additional Statistics"

R Walkthrough:

Independent Samples T-Test:

t.test(DV ~ grouping_variable, data = df,
       paired = FALSE, var.equal = TRUE, conf.level = .95)

effsize::cohen.d(DV ~ grouping_variable, data = df,
                 pooled = TRUE, hedges.correction = FALSE)

Paired Samples T-Test:

t.test(df$before, df$after,
       paired = TRUE, alternative = "two.sided", conf.level = .95)

effsize::cohen.d(df$before, df$after, paired = TRUE, hedges.correction = FALSE)

One-Sample T-Test:

t.test(df$variable, mu = 100, conf.level = .95)

effsize::cohen.d(df$variable ~ 1, mu = 100, conf.level = .95)

Output Interpretation:

p-value: The probability of detecting a meaningful relationship/difference when there is none. We are typically looking for a small value (p < .05).

If p < .05, reject the null hypothesis. There IS a significant difference.
If p > .05, retain the null hypothesis. There is NO significant difference.

t-value: The number of estimated standard error of the mean (SEM) units the sample is from the population.

If t-value is small, there is more similarity between the two sample sets.
If t-value is large, a large difference exists between the two sample sets.
If t-value is negative, it has no bearing on the significance of the difference. It only indicates a reversed directionality of the effect.

SEM: Average deviation of sample means from the population mean. A measure of sampling error.

If SEM is small, sample means are similar to the population mean, with little sampling error (this is what we want).
If SEM is large, sample means are more variable.

df: Values in a study that have the freedom to vary. Essential for assessing the importance and validity of the null hypothesis.

Effect size - Cohen's d: A measure of the magnitude of the mean difference. Bigger numbers mean a bigger difference between groups in terms of standard deviation.

Cohen's d	Interpretation
< .20	Very small effect size
.20 – .49	Small effect size
.50 – .79	Medium effect size
≥ .80	Large effect size

Confidence Interval (CI): The 95% CI refers to the interval around the mean difference. If the CI does not include zero, the difference is statistically significant at p < .05. If it includes zero, the difference is not significant. In Jamovi, check "Confidence interval" under "Additional Statistics" to get these values. Report the lower and upper bounds in brackets.

APA Format:

Appropriate data visualization: Bar graphs (with error bars).

Sample table: https://apastyle.apa.org/style-grammar-guidelines/tables-figures/sample-tables#tests

Sample write-up:

A paired-samples t-test compared the average [DV] during [Condition 1] (M = , SD = ) and [Condition 2] (M = , SD = ). There was a significant difference in [DV], t() = , p < .05, d = , 95% CI [, ]. These results suggest that [DV] increases/decreases from [Condition 1] to [Condition 2].

Note: Plug in the appropriate test used, means, standard deviations, whether the test was significant, t-value, df, p-level, the strength of the relationship (d), confidence interval, and a conclusion.