Which Test?

Planning

How many patients do you need?

Work out the sample size before you collect data. A study that is too small cannot answer its question, and a study that is too large wastes patients and time.

What do you want to know?
In the unit of your measurement. The smallest difference that matters clinically.
From a similar study or a pilot. In the unit of your measurement.
Usually 0.05.
Usually 0.8 or 0.9.
Patients you expect to lose. 0 if none.

You need

64 patients per group

128 patients in total.

A sentence for your protocol

With a two-sided significance level of 0.05 and a power of 80%, 64 patients per group are needed to detect a difference of 5 with a standard deviation of 10.

What you have to decide

  • The difference that matters clinically. Not the difference you hope to find, but the smallest one that would change practice.
  • The spread of your measurement (standard deviation) for a measurement, or the expected proportions for a yes/no outcome. Take them from a similar study or a pilot.
  • The significance level (usually 0.05) and the power (usually 0.8 or 0.9).
  • The dropout you expect, so that enough patients remain for the analysis.

Common mistakes

  • Calculating the power after the study from the result you found ("observed power"). It adds no information beyond the p-value. Use the confidence interval to show what the study can exclude.
  • Taking the standard deviation from a very small pilot. It is itself uncertain; consider a safety margin.
  • Choosing the effect size to fit the number of patients you have, instead of the other way around.
  • Forgetting that a model with many covariates, or several outcomes, needs more patients than these simple calculators give.

Check it in R

The numbers here use standard formulas. You can verify them with these commands. They give the same result to within a patient or two.

R
# Two means: 5 units difference, SD 10, 80% power, two-sided 5%
power.t.test(delta = 5, sd = 10, sig.level = 0.05, power = 0.8)

# Two proportions: 50% against 65%
power.prop.test(p1 = 0.5, p2 = 0.65, sig.level = 0.05, power = 0.8)

# Paired means (or one group against a reference value)
power.t.test(delta = 5, sd = 10, power = 0.8, type = "paired")

When to ask a statistician

Clustered patients, several primary outcomes, interim analyses, non-inferiority designs, repeated measurements and prediction models need a different calculation. See advanced topics. Terms such as power and effect size are explained in the glossary.