Introducing a novel applicant ranking tool to predict future resident performance: A pilot study

Sarah N. Bowe, Erik K. Weitzel, William N. Hannah, Brian M. Fitzgerald, Gregory P. Kraus, Christopher J. Nagy, Stephen A. Harrison

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


The purposes of this study are to (1) introduce our novel Applicant Ranking Tool that aligns with the Accreditation Council for Graduate Medical Education competencies and (2) share our preliminary results comparing applicant rank to current performance. After a thorough literature review and multiple roundtable discussions, an Applicant Ranking Tool was created. Feasibility, satisfaction, and critiques were discussed via open feedback session. Inter-rater reliability was assessed using weighted kappa statistic (κ) and Kendall coefficient of concordance (W). Fisher’s exact tests evaluated the ability of the tool to stratify performance into the top or bottom half of their class. Internal medicine and anesthesiology residents served as the pilot cohorts. The tool was considered user-friendly for both data input and analysis. Inter-rater reliability was strongest with intradisciplinary evaluation (W = 0.8–0.975). Resident performance was successfully stratified into those functioning in the upper vs. lower half of their class within the Clinical Anesthesia-3 grouping (p = 0.008). This novel Applicant Ranking Tool lends support for the use of both cognitive and noncognitive traits in predicting resident performance. While the ability of this instrument to accurately predict future resident performance will take years to answer, this pilot study suggests the instrument is worthy of ongoing investigation.

Original languageEnglish (US)
Pages (from-to)e1514-e1520
JournalMilitary medicine
Issue number1
StatePublished - Jan 2017
Externally publishedYes

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health


Dive into the research topics of 'Introducing a novel applicant ranking tool to predict future resident performance: A pilot study'. Together they form a unique fingerprint.

Cite this