TY - JOUR
T1 - Evaluating the evidence
T2 - The methodological and reporting quality of comparative observational studies of surgical interventions in urological publications
AU - Tseng, Timothy Y.
AU - Breau, Rodney H.
AU - Fesperman, Susan F.
AU - Vieweg, Johannes
AU - Dahm, Philipp
PY - 2009/4
Y1 - 2009/4
N2 - Objective To develop and apply a standardized evaluation form for assessing the methodological and reporting quality of observational studies of surgical interventions in urology. METHODS An evaluation standard was developed using the Consolidated Standards for Reporting Trials statement and previously reported surgical reporting quality instruments. Consensus scoring among three reviewers was developed using two distinct sets of studies. All comparative observational trials involving therapeutic surgical procedures published in four major urological journals in 1995 and 2005 were randomly assigned to each reviewer. Categories of reporting adequacy included background, intervention, statistical analysis, results and discussion. RESULTS Twenty-seven articles in 1995 and 62 in 2005 met the inclusion criteria; 90% of studies were retrospective. From 1995 to 2005, the overall reporting quality score increased by 3.9 points (95% confidence interval, CI, 2.7-5.9; P = 0.001), from a mean (sd) of 19.1 (3.9) to 23.0 (4.2) on a scale of 0-42. There were significant improvements in the reporting categories of study background (+0.7 points, 95% CI 0.1-1.3, P = 0.043, 0-8-point scale), intervention (+1.6 points, 0.8-2.3, P = 0.001, 0-9-point scale), and statistical analysis (+0.8 points, 0.2-1.4, P = 0.006, 0-9-point scale). There were smaller and statistically insignificant improvements for results (+0.5 points, -0.3 to 1.2, P = 0.217, 0-10-point scale) and discussion reporting (+0.4 points, -0.1 to 0.8, P = 0.106, 0-6-point scale). CONCLUSIONS There have been minor improvements in the reporting of observational studies of surgical intervention between 1995 and 2005. However, reporting quality remains suboptimal. Clinical investigators, reviewers and journal editors should continue to strive for transparent reporting of the observational studies representing the bulk of the clinical evidence for urological procedures.
AB - Objective To develop and apply a standardized evaluation form for assessing the methodological and reporting quality of observational studies of surgical interventions in urology. METHODS An evaluation standard was developed using the Consolidated Standards for Reporting Trials statement and previously reported surgical reporting quality instruments. Consensus scoring among three reviewers was developed using two distinct sets of studies. All comparative observational trials involving therapeutic surgical procedures published in four major urological journals in 1995 and 2005 were randomly assigned to each reviewer. Categories of reporting adequacy included background, intervention, statistical analysis, results and discussion. RESULTS Twenty-seven articles in 1995 and 62 in 2005 met the inclusion criteria; 90% of studies were retrospective. From 1995 to 2005, the overall reporting quality score increased by 3.9 points (95% confidence interval, CI, 2.7-5.9; P = 0.001), from a mean (sd) of 19.1 (3.9) to 23.0 (4.2) on a scale of 0-42. There were significant improvements in the reporting categories of study background (+0.7 points, 95% CI 0.1-1.3, P = 0.043, 0-8-point scale), intervention (+1.6 points, 0.8-2.3, P = 0.001, 0-9-point scale), and statistical analysis (+0.8 points, 0.2-1.4, P = 0.006, 0-9-point scale). There were smaller and statistically insignificant improvements for results (+0.5 points, -0.3 to 1.2, P = 0.217, 0-10-point scale) and discussion reporting (+0.4 points, -0.1 to 0.8, P = 0.106, 0-6-point scale). CONCLUSIONS There have been minor improvements in the reporting of observational studies of surgical intervention between 1995 and 2005. However, reporting quality remains suboptimal. Clinical investigators, reviewers and journal editors should continue to strive for transparent reporting of the observational studies representing the bulk of the clinical evidence for urological procedures.
KW - Epidemiological
KW - Evidence-based medicine
KW - Observational study methods
KW - Publishing standards
KW - Research design standards
UR - http://www.scopus.com/inward/record.url?scp=63449124481&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=63449124481&partnerID=8YFLogxK
U2 - 10.1111/j.1464-410X.2008.08155.x
DO - 10.1111/j.1464-410X.2008.08155.x
M3 - Article
C2 - 19021602
AN - SCOPUS:63449124481
SN - 1464-4096
VL - 103
SP - 1026
EP - 1031
JO - BJU International
JF - BJU International
IS - 8
ER -