Sample size for human subjects
Many studies are too small to detect even large effects (Table 1)
| Expected difference (P1-P2) |
Total sample size required * |
| 5% |
1450-3200 |
| 10% |
440-820 |
| 20% |
140-210 |
| 30% |
80-100 |
| 40% |
50-60 |
5% significance level, 80% power. Small numbers may be justified (p1 < 0.1)
What should you look for?
- Clinical trials should always report sample size calculations
- Authors with 'negative' results (i.e.found no difference) should not report equivalence unless sufficiently proven -"absence of evidence is not evidence of absence"
Bias
Randomisation is the best way of avoiding bias but it is not always possible or appropriate.
Some biases affecting observational studies:
- Treatment-by-indication bias: different treatments are given to different groups of patients because of differences in their clinical condition.
- Historical controls: will tend to exaggerate treatment effect as recent patients benefit from improvements in health care over time and special attention as a study participant. Recent patients are also likely to be more restrictively selected.
- Retrospective data collection: availability and recording of events and patient characteristics may be related to the groups being compared.
- Ecological fallacy: an association observed between variables on an aggregate level does not necessarily represent the association that exists at the individual level.
Some biases affecting observational studies and clinical trials:
- Selection bias: low response rate or high refusal rate – were patients that participated different to those that did not?
- Informative dropout – was follow-up curtailed for reasons connected to the primary outcome? If so, imbalance in dropout rates between the groups being compared will introduce bias.
Bias in clinical trials:
No-one should know what the next random allocation is going to be as this may affect whether or when the patient is entered into the trial. Using date of birth, hospital number, or simply alternating between treatments is therefore inappropriate. Central randomisation is ideal. Unblinded assessment of outcomes may be influenced by knowledge of the treatment group.
Look for:
- Appreciation and measures taken to reduce bias through study design
- Selection of patients, collection of data, definition and assessment of outcome and, for clinical trials, method of randomisation should be clearly described
- Number and reasons for withdrawal should be reported by treatment group
- Appropriate analytic methods such as multiple regression should be used to adjust for differences between groups in observational studies
- Authors should discuss likely biases and potential impact on their results
Method comparison studies
If different methods are evaluated by different observers then the method differences are confounded with observer differences. The study must be repeated with each observer using all methods.
