Evaluating the evidence: The methodological and reporting quality of comparative observational studies of surgical interventions in urological publications

Timothy Y. Tseng, Rodney H. Breau, Susan F. Fesperman, Johannes Vieweg, Philipp Dahm

Research output: Contribution to journalArticlepeer-review

24 Scopus citations


Objective To develop and apply a standardized evaluation form for assessing the methodological and reporting quality of observational studies of surgical interventions in urology. METHODS An evaluation standard was developed using the Consolidated Standards for Reporting Trials statement and previously reported surgical reporting quality instruments. Consensus scoring among three reviewers was developed using two distinct sets of studies. All comparative observational trials involving therapeutic surgical procedures published in four major urological journals in 1995 and 2005 were randomly assigned to each reviewer. Categories of reporting adequacy included background, intervention, statistical analysis, results and discussion. RESULTS Twenty-seven articles in 1995 and 62 in 2005 met the inclusion criteria; 90% of studies were retrospective. From 1995 to 2005, the overall reporting quality score increased by 3.9 points (95% confidence interval, CI, 2.7-5.9; P = 0.001), from a mean (sd) of 19.1 (3.9) to 23.0 (4.2) on a scale of 0-42. There were significant improvements in the reporting categories of study background (+0.7 points, 95% CI 0.1-1.3, P = 0.043, 0-8-point scale), intervention (+1.6 points, 0.8-2.3, P = 0.001, 0-9-point scale), and statistical analysis (+0.8 points, 0.2-1.4, P = 0.006, 0-9-point scale). There were smaller and statistically insignificant improvements for results (+0.5 points, -0.3 to 1.2, P = 0.217, 0-10-point scale) and discussion reporting (+0.4 points, -0.1 to 0.8, P = 0.106, 0-6-point scale). CONCLUSIONS There have been minor improvements in the reporting of observational studies of surgical intervention between 1995 and 2005. However, reporting quality remains suboptimal. Clinical investigators, reviewers and journal editors should continue to strive for transparent reporting of the observational studies representing the bulk of the clinical evidence for urological procedures.

Original languageEnglish (US)
Pages (from-to)1026-1031
Number of pages6
JournalBJU International
Issue number8
StatePublished - Apr 2009
Externally publishedYes


  • Epidemiological
  • Evidence-based medicine
  • Observational study methods
  • Publishing standards
  • Research design standards

ASJC Scopus subject areas

  • Urology


Dive into the research topics of 'Evaluating the evidence: The methodological and reporting quality of comparative observational studies of surgical interventions in urological publications'. Together they form a unique fingerprint.

Cite this