An evaluation of the Global trigger Tool method: teams' ability to reproduce adverse event findings within hospitals

Introduction Global Trigger Tool (GTT) is a retrospective review method where inpatient medical records are randomly sampled and reviewed by trained GTT teams to identify and monitor adverse event (AE) rates over time. Tracking AEs over time is a useful way to indicate if targeted changes result in improved patient safety. The method presupposes that the teams are consistent in their record reviews. The objective of our study was to examine several teams’ ability to reproduce their findings from a previous review. There is no consensus in the GTT literature regarding this reproducibility.
Methods Eight teams from six Norwegian hospitals applied the GTT method to a total of 958 randomly selected inpatient medical records. Each inpatient record was previously reviewed in 2016-2018 and then reexamined in 2023. We estimated the teams’ agreement between the previous and new ratings, with focus on the teams’ reproducibility for rating the presence of at least one AE or not. The teams’ ratings of the number of AEs and the AE severity level were also considered. We computed and compared several well-known agreement measures. The study was approved by Regional Committees for Medical and Health Research Ethics in Norway. The participating hospitals’ Data Protection Officers have given their approval.
Results Overall there was good to very good agreement between the teams’ previous and new ratings for the presence of at least one AE or not. The unadjusted agreement for all teams together was 84.9%, and on team level it ranged between 76.7% and 89.2%. The agreement for all teams together, adjusted for agreement by chance, ranged from Fleiss’ κ: 0.673 (0.625 – 0.722) to Gwet’s AC1: 0.718 (0.674 – 0.762). At team level, we found some differences in the reproducibility of the presence of at least one AE or not. The ability to reproduce previous findings on the number of AEs and AE severity level varied more across teams.
Conclusion Our findings support the use of the GTT method as a tool for identifying and monitoring AE rates over time, with the aim of reducing patient harm. It appears to be more challenging to obtain good consistency when rating the number of AEs and AE severity level than the presence of at least one AE or not. We believe there is a need for better training and auxiliary resources for the teams if the goal is to improve the reproducibility of the number of AEs and AE severity level.