Student t-test: definition

Parametric statistical test (Gosset 1908, "Student" pseudonym) for comparing means in small samples when population variance is unknown and estimated from data. Under normality, t-statistic follows Student t-distribution with df degrees of freedom:

t = (statistic - H0_parameter) / SE(statistic)

Test types

• One-sample: t = (xbar - mu_0)/(s/sqrt(n)), df = n - 1.
• Two-sample Welch: t = (xbar1 - xbar2)/sqrt(s1^2/n1 + s2^2/n2), Welch-Satterthwaite df. Robust to unequal variances.
• Pooled Student: assumes equal variances. df = n1+n2-2. Use Welch instead.
• Paired: t = d_bar/(s_d/sqrt(n)), df = n-1. For repeated measures.

One-sample example

n=25 students, xbar=102, s=10, mu_0=100. SE=2. t=1.0. df=24. p-value ~ 0.327. Not significant at alpha=0.05.

Two-sample example

Group A: n=25, xbar=102, s=10. Group B: n=30, xbar=98, s=12. Diff=4. SE_W=2.966. t=1.349. df Welch ~ 52.6. p ~ 0.183.

Student t-distribution

• Symmetric around 0.
• Heavier tails than normal for small df.
• df = 1: Cauchy.
• df -> infinity: standard normal.
• Variance: df/(df-2) for df > 2.

p-value computation

Two-tailed: p = 2*P(T > |t|). Using beta function: P(T <= t | df) = 1 - 0.5 I_{df/(df+t^2)}(df/2, 1/2). Numerical Recipes 6.4 algorithm.

Critical values (alpha=0.05, two-tailed)

• df=1: 12.706.
• df=5: 2.571.
• df=10: 2.228.
• df=20: 2.086.
• df=30: 2.042.
• df=infty: 1.96.

Assumptions

1. Independence.
2. Normality (or large n via CLT).
3. Interval scale.
4. Equal variance for pooled Student.

Type I/II errors and power

• Type I (alpha): reject H0 when true. Conventional 0.05.
• Type II (beta): fail to reject when false.
• Power (1-beta): probability to reject when false. Increases with n, effect size, alpha. Convention 0.80.
• Effect size: Cohen d = (xbar - mu_0)/s. Small 0.2, medium 0.5, large 0.8.

Confidence interval

CI for mean: xbar +/- t_{alpha/2, df} * s/sqrt(n). Example xbar=102, s=10, n=25, alpha=0.05: CI = [97.87, 106.13].

Alternatives

• Wilcoxon signed-rank: alternative to one-sample/paired t.
• Mann-Whitney U: alternative to two-sample t.
• Bootstrap: empirical distribution.
• Permutation test: shuffle group labels.

Multiple testing

• Bonferroni: alpha/k.
• Holm-Bonferroni: sequential.
• FDR (Benjamini-Hochberg): controls false discovery rate.

History

William Sealy Gosset (1876-1937), chemist at Guinness Brewery. Developed for small hop samples. Published 1908 in Biometrika under "Student" pseudonym. Fisher (1925) formalized in likelihood framework. Welch (1947) for unequal variances. Satterthwaite (1946) df approximation.

Calculator modes

Five modes: one-sample, two-sample Welch (unequal variances), paired, critical t for given alpha, confidence interval. p-value via regularized incomplete beta function. Critical t via Beasley-Springer approximation.

Effect size, confidence intervals and statistical power

A t-test answers whether a difference is distinguishable from noise, but not how large or how important that difference is. For the latter you need the effect size, most commonly Cohen's d, which expresses the gap between means in units of the pooled standard deviation. A d of 0.2 is conventionally small, 0.5 medium and 0.8 large, and reporting it prevents the common error of mistaking a tiny but statistically significant effect for a meaningful one. A confidence interval carries the same information in the original units: the calculator's interval gives the plausible range for the true mean, and its width conveys precision far better than a bare p-value.

The mirror image of significance is statistical power, the probability that a test detects a real effect of a given size. Power grows with the sample size, the effect size and the chosen significance level, and falls as variability rises. Before collecting data, researchers run a power analysis to find the sample size that gives an acceptable chance, usually 80 percent, of detecting the effect they care about. An underpowered study risks missing a genuine effect and, paradoxically, tends to exaggerate the effects it does find. Together effect size, confidence intervals and power turn a bare accept-or-reject verdict into a quantitative statement about magnitude and reliability.

One-tailed versus two-tailed, and reading the p-value

Every t-test rests on a choice of alternative hypothesis. A two-tailed test asks whether the means differ in either direction and splits the significance level between both tails; this is the default, and the p-value the calculator reports. A one-tailed test asks a directional question, whether one mean is specifically greater or specifically smaller, and places the whole significance level in a single tail. Because that concentrates the rejection region, a one-tailed test is more powerful when the direction is genuinely known in advance, but choosing it after seeing the data inflates the false-positive rate and is considered misconduct. For a symmetric t-distribution the one-tailed p-value is simply half the two-tailed one.

Interpreting the p-value correctly is just as important as computing it. The p-value is the probability of observing a test statistic at least as extreme as the one obtained, assuming the null hypothesis is true. It is not the probability that the null hypothesis is true, nor the probability that the result arose by chance, and a threshold such as 0.05 is a convention, not a law of nature. A result just under the threshold is not qualitatively different from one just above it, which is why modern practice reports the exact p-value alongside an effect size and a confidence interval rather than a bare significant-or-not label.

Updated July 2026