Which Test?

Draft: This page has not been reviewed by a statistician yet. Use it as a starting point and check important choices with a statistician.

Prediction models (not covered yet)

A prediction model estimates the risk of an outcome for an individual patient; building one is a separate task that this guide does not cover yet.

Clinical example

Estimating a patient's risk of a cardiovascular event over the next few years from age, blood pressure and cholesterol.

When to use it

  • You want to predict the risk of an outcome for an individual patient from several characteristics.
  • The aim is prediction, not estimating the effect of one factor.

What to report

  • The model, with all its coefficients or a way to compute the risk
  • How the model was validated
  • Reporting that follows the TRIPOD guideline

Common mistakes

Avoid selecting variables by stepwise or by p-value

It tends to overfit, biases the estimates and makes confidence intervals too narrow. Choose variables in advance, for example with a causal diagram (DAG).

Mind the number of events

For yes/no and survival outcomes, the number of events limits how many variables the model can hold reliably, not the number of patients.

What a prediction model is

A prediction model combines several patient characteristics into an estimate of risk for one patient. A diagnostic model estimates the probability that a condition is present now. A prognostic model estimates the probability of a future outcome.

This is a different task from comparing groups or estimating the effect of a treatment. The question is how well the model predicts for new patients, not whether one factor is associated with the outcome.

Why it needs a statistician

Developing and validating a prediction model involves many choices that are easy to get wrong: which variables to consider, how to handle continuous variables and missing data, how to avoid overfitting, and how to test the model on patients that were not used to build it.

Plan the study together with a statistician before you collect or analyze data. See advanced topics for when to bring one in.

Reporting

Reporting should follow the TRIPOD guideline (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis).