What not to do
The classic mistakes, and what to do instead
Each result in the test chooser lists what to avoid for its method. These cut across methods.
Treating repeated measurements as independent
Two knees, several visits or matched pairs from one patient are not separate observations. Ignoring this shrinks standard errors and inflates false positives.
Many t-tests instead of one ANOVA
Every extra pairwise test raises the chance of a false positive. Test all groups at once, then correct post hoc comparisons.
Letting a normality test decide
In large samples a trivial deviation is 'significant'; in small samples real skew goes undetected. Look at the plot and the sample size, and check Shapiro–Wilk's W rather than its p-value. With about 25+ per group, mild skew is fine for a t-test.
Comparing subgroup p-values
'Significant in men, not in women' is not evidence of a difference. Test the interaction.
Reporting only p-values
Give the effect size with a 95% confidence interval. It shows direction, size and precision.
Calling an odds ratio a risk ratio
When the outcome affects more than about 10% of patients, the OR overstates the relative risk.
Model building by stepwise selection alone
Choose predictors on clinical grounds, and keep about 10 events per predictor.
Correlation as proof of agreement
Two devices can correlate at r = 0.95 and still differ by a clinically important amount. Use Bland–Altman.
A test at every time point
Results at adjacent visits are linked, and the number of tests piles up. Use a summary measure or a mixed model.
Ignoring censoring
Dropping patients who haven't had the event yet, or using logistic regression with unequal follow-up, biases the result.
This is a starting point. Studies with clustered patients, missing data or several outcomes need a statistician. See advanced topics.