Factors that lead to decreasing evidence quality in the GRADE framework
| FACTORS | EXPLANATION | EXAMPLES |
|---|---|---|
| Risk of bias | An investigator assigns patients to the intervention group based on day of the week or date of birth, many patients drop out of the study, and the analysis is conducted to consider only those who adhered to the intervention23 | |
| • Allocation concealment | Whether those enrolling patients are aware of the group to which the next patient will be allocated | |
| • Blinding | Whether those involved in the study are aware of the study arm to which patients are allocated | |
| • Loss to follow-up | Whether many patients were lost during the trial, and whether the analysis was performed to take these losses into account | |
| • Selective outcome reporting | When outcomes are reported based on the results and not based on a priori selection of important outcomes | |
| • Other limitations | When there are other factors that could decrease the quality of the study | |
| Inconsistency (variability in results) | If different studies provide widely different estimates of effect | There are 4 studies examining the effect of an intervention on an outcome; 2 studies show positive effects, 1 shows a negative effect, while the final study shows a null effect24 |
| Indirectness | The similarity between the question being addressed by the CTFPHC and the available evidence about the population, intervention, comparison groups, and outcomes | There are no data comparing drug A and B to each other, but there are data comparing drug A to a placebo and drug B to a placebo. There are no data on the effect of screening on cervical cancer mortality, but there are data on the surrogate outcome of invasive cervical cancer25 |
| Imprecision | Studies are imprecise if they have few patients or few events or very wide CIs around the estimates that lead to greater uncertainty about the findings | A group of studies has a wide CI and the clinical action that would be recommended changes depending on whether the upper boundary of the CI or the lower boundary of the CI represents the truth26 |
| Publication bias | An overestimate or underestimate in the effect that occurs when studies that find no effects are not published | Review authors fail to identify all relevant studies, negative studies are not published, or industry influence is suspected27 |
CTFPHC—Canadian Task Force on Preventive Health Care, GRADE—grading of recommendations, assessment, development, and evaluation.
Data from Guyatt et al.23–27