TY - JOUR
T1 - Error rates of data processing methods in clinical research
T2 - A systematic review and meta-analysis of manuscripts identified through PubMed
AU - Garza, Maryam Y.
AU - Williams, Tremaine
AU - Ounpraseuth, Songthip
AU - Hu, Zhuopei
AU - Lee, Jeannette
AU - Snowden, Jessica
AU - Walden, Anita C.
AU - Simon, Alan E.
AU - Devlin, Lori A.
AU - Young, Leslie W.
AU - Zozus, Meredith N.
N1 - Publisher Copyright:
© 2024 Elsevier B.V.
PY - 2025/3
Y1 - 2025/3
N2 - Background: In clinical research, prevention of data errors is paramount to ensuring reproducibility of trial results and the safety and efficacy of the resulting interventions. Over the last 40 years, empirical assessments of data accuracy in clinical research have been reported, however, there has been little systematic synthesis of these results. Although notable exceptions exist, little evidence exists regarding the relative accuracy of different data processing methods. Methods: A systematic review of the literature identified through PubMed was performed to identify studies that evaluated the quality of data obtained through data processing methods typically used in clinical trials. Quantitative information on data accuracy was abstracted from the manuscripts and pooled. Meta-analysis of single proportions based on the Freeman-Tukey transformation method and the generalized linear mixed model approach were used to derive an overall estimate of error rates across data processing methods used in each study for comparison. Results: A total of 93 papers (published from 1978 to 2008) meeting our inclusion criteria were categorized according to their data processing methods. The accuracy associated with data processing methods varied widely, with error rates ranging from 2 errors per 10,000 fields to 2,784 errors per 10,000 fields. MRA was associated with both high and highly variable error rates, having a pooled error rate of 6.57% (95% CI: 5.51, 7.72). In comparison, the pooled error rates for optical scanning, single-data entry, and double-data entry methods were 0.74% (0.21, 1.60), 0.29% (0.24, 0.35) and 0.14% (0.08, 0.20), respectively. Conclusions: Data processing methods may explain a significant amount of the variability in data accuracy. MRA error rates, for example, were high enough to impact decisions made using the data and could necessitate increases in sample sizes to preserve statistical power. Thus, the choice of data processing methods can likely impact process capability and, ultimately, the validity of trial results.
AB - Background: In clinical research, prevention of data errors is paramount to ensuring reproducibility of trial results and the safety and efficacy of the resulting interventions. Over the last 40 years, empirical assessments of data accuracy in clinical research have been reported, however, there has been little systematic synthesis of these results. Although notable exceptions exist, little evidence exists regarding the relative accuracy of different data processing methods. Methods: A systematic review of the literature identified through PubMed was performed to identify studies that evaluated the quality of data obtained through data processing methods typically used in clinical trials. Quantitative information on data accuracy was abstracted from the manuscripts and pooled. Meta-analysis of single proportions based on the Freeman-Tukey transformation method and the generalized linear mixed model approach were used to derive an overall estimate of error rates across data processing methods used in each study for comparison. Results: A total of 93 papers (published from 1978 to 2008) meeting our inclusion criteria were categorized according to their data processing methods. The accuracy associated with data processing methods varied widely, with error rates ranging from 2 errors per 10,000 fields to 2,784 errors per 10,000 fields. MRA was associated with both high and highly variable error rates, having a pooled error rate of 6.57% (95% CI: 5.51, 7.72). In comparison, the pooled error rates for optical scanning, single-data entry, and double-data entry methods were 0.74% (0.21, 1.60), 0.29% (0.24, 0.35) and 0.14% (0.08, 0.20), respectively. Conclusions: Data processing methods may explain a significant amount of the variability in data accuracy. MRA error rates, for example, were high enough to impact decisions made using the data and could necessitate increases in sample sizes to preserve statistical power. Thus, the choice of data processing methods can likely impact process capability and, ultimately, the validity of trial results.
KW - Clinical data management
KW - Clinical research
KW - Data collection
KW - Data quality
KW - Medical record abstraction
UR - http://www.scopus.com/inward/record.url?scp=85211121354&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85211121354&partnerID=8YFLogxK
U2 - 10.1016/j.ijmedinf.2024.105749
DO - 10.1016/j.ijmedinf.2024.105749
M3 - Review article
C2 - 39647291
AN - SCOPUS:85211121354
SN - 1386-5056
VL - 195
JO - International Journal of Medical Informatics
JF - International Journal of Medical Informatics
M1 - 105749
ER -