Confidence Assessment Criteria in Evidence-Based Medicine (EBM)

Confidence Assessment Criteria in Evidence-Based Medicine (EBM)

Analytical Report

• What is confidence assessment criteria?

Confidence assessment criteria are a set of criteria used for evaluating and rating the quality of evidence mainly randomized control trial and observational studies. It is also used for grading the strength of recommendations in guidelines. This assessment system is known as the GRADE (Grading of Recommendations Assessment, Development, and Evaluation). It is designed for examining alternative management strategies and interventions mentioned in systematic reviews and guidelines. These assessment criteria include the risk of bias, inconsistency, indirectness, imprecision, publication bias, the large magnitude of an effect, and dose-gradient response. (book) GRADE offers a straightforward and organized procedure for systematic health care reviews and recommendations to develop and present summaries of evidence, including its quality. Additionally, it provides a guide for using recommendations to clinicians and patients. (guidelines 1)

• How they are different?

The assessment criteria are used for grading down and grading up the evidence by systematic review authors and guidelines developers. Out of seven criteria, the risk of bias, inconsistency, indirectness, imprecision, and publication bias are used for grading down the evidence whereas, the large magnitude of an effect and dose-gradient response are used for grading up the evidence. (book)

Risk of bias/study limitation:

 Risk of bias or study limitation can occur at any stage of a research study. Randomized controlled trials (RCTs) and observational studies, both can present with a flaw in their study design and/or study conduct, and additional risk of misleading results. Study limitations for RCTs include lack of allocation concealment, lack of blinding, incomplete accounting of patients and outcome events, selective outcome report bias, use of unvalidated outcome measures, recruitment bias, and stopping early for desired benefits. Study limitations for observational studies include failure to develop and apply appropriate eligibility criteria, flawed measurement of exposure and outcome, failure to adequately control confounding factors, and incomplete follow-up. (guidelines 4)

Inconsistency:

 The evidence is not graded up in quality if they produce consistent results, but can be graded down if they produce inconsistent results. The judgment of the degree of heterogeneity depends on the likeness of point estimates, the degree of overlap of confidence intervals, and statistical criteria including tests of heterogeneity. (Guidelines 7)

Indirectness:

 Direct evidence originates from research that directly compares the interest of interventions in the interest of the population, and measures the outcomes important for the patient. Indirect evidence can be of four ways when patients differ from the interest of the population when tested intervention differ from the intervention of interest, when outcome vary from primary interest, when clinicians have to choose from interventions which are not tested in head-to-head comparison yet. (guidelines 8)

Imprecision:

 The main criterion for GRADE’s assessment of the precision of the study is to focus on the 95% confidence interval (CI) around the difference in effect between interventions and control for each result. If a recommendation or clinical course of action differs if the truth is represented by the upper limit versus the lower limit of the CI, consider the study for rating down for its imprecision. If the effects are larger and both the sample size and the number of events are discreet, consider the study for rating down for imprecision even if CIs appear satisfactorily narrow. (guidelines 6)

Publication bias:

 Withholding of “negative” results of a study by industry sponsors or researchers can result in publication bias. Evidence suggests that studies with statistically important results are generally more likely to be published than studies with statistically insignificant results. Systematic reviews carried early on, when very few initial studies are available, it is most likely to overestimate the effects when the publication of negative studies delayed as the studies published later may not be included. This phenomenon is called “lag bias”. It is a type of publication bias. It can occur for two reasons either authors do not submit their studies to prominent journal with what they perceived as unremarkable results or a study may end up being published in a local journal or non-indexed database because of repeated rejection from prominent journals. The observational studies show a higher risk of publication bias as compared to RCTs. (guidelines 5)

The large magnitude of effect:

 Grading up by one or two levels is possible when the outcome of the research is sufficiently large enough. Some clinical interventions like the hip joint replacement to reduce pain and increase functionality in patients with severe osteoarthritis and insulin to prevent mortality in diabetic ketoacidosis clinicians are extremely confident of their outcomes. In these situations, according to GRADE’s criteria and definition of quality of evidence, the quality of evidence of these clinical interventions can be considered high even though the evidence originates from observational studies. (guideline 9)

Dose-gradient response:

 The dose-gradient response is considered as one of the most important criteria for a cause-effect relationship for a long time. This gradient may increase the confidence in the results of the observational studies and hence enhances the quality of evidence. For example, the confidence in the outcomes of observational studies that show slow infant growth in breast milk fed infants whereas, accelerated growth in infants fed with breast milk and formula, and more accelerated in only formula-fed infants. (guideline 9)

Conclusion:

 Based on the above-mentioned information it is evident that all the assessment criteria used in the GRADE system are different from each other and plays a crucial role in assessing and grading of evidence.

References

• Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., et al. (2011). GRADE guidelines: 1.
• Introduction—GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology, 64(4), 383- 394. doi:10.1016/j.jclinepi.2010.04.026
• Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., . . . Schünemann,
• H. J. (2011). GRADE guidelines: 4. rating the quality of evidence—study limitations (risk of bias). Journal of Clinical Epidemiology, 64(4), 407- 415. doi:10.1016/j.jclinepi.2010.07.017
• Guyatt, G. H., Oxman, A. D., Montori, V., Vist, G., Kunz, R., Brozek, J., . . . Schünemann, H. J.
• (2011). GRADE guidelines: 5. rating the quality of evidence—publication bias. Journal of Clinical Epidemiology, 64(12), 1277-1282. doi:10.1016/j.jclinepi.2011.01.011
• Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., . . . Schünemann,
• H. J. (2011). GRADE guidelines 6. rating the quality of evidence—imprecision. Journal of Clinical Epidemiology, 64(12), 1283-1293. doi:10.1016/j.jclinepi.2011.01.012
• Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., . . . GRADE
• Working Group. (2011). GRADE guidelines: 7. rating the quality of evidence—inconsistency. Journal of Clinical Epidemiology, 64(12), 1294- 1302. doi:10.1016/j.jclinepi.2011.03.017
• Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., . . . GRADE
• Working Group. (2011). GRADE guidelines: 8. rating the quality of evidence—indirectness. Journal of Clinical Epidemiology, 64(12), 1303-1310. doi:10.1016/j.jclinepi.2011.04.014
• Guyatt, G. H., Oxman, A. D., Sultan, S., Glasziou, P., Akl, E. A., Alonso-Coello, P., . . . GRADE
• Working Group. (2011). GRADE guidelines: 9. rating up the quality of evidence. Journal of Clinical Epidemiology, 64(12), 1311-1316. doi:10.1016/j.jclinepi.2011.06.004
• Guyatt, G., Rennie, D., Meade, R., Cook, D. (2015). Users’ Guides to the Medical Literature:Essentials of Evidence-Based Clinical Practice, Third Edition. McGraw-Hill Education.