Assessment of pediatric residents' otoscopic interpretive skills by videotaped examinations

Background. - Accurate diagnosis of otitis media is essential to facilitate appropriate management. Few residency programs assess formally their residents' competency in diagnosing middle ear disease. Objective. - To evaluate the performance of pediatric residents' otoscopic interpretive skills by level of training, with a videotaped otoendoscopic examination (VOE). Methods. - A VOE was used to assess and compare the performance of a cross-sectional sample of 141 residents with one another by level of training as well as with novice and expert groups. Total score, sensitivity, specificity, and kappa coefficients were calculated for each subject by comparing the subject's answers on the 50-ear test with the VOE's expert panel-derived answers, and averages were determined for each training level. Results. - Each pediatric resident training group had moderate agreement (mean κ coefficient range: .45-.56) with the VOE answers, compared with the novice group (mean κ: .31, fair) and expert group (mean κ: .80, substantial). Twenty-eight residents (20%) had fair or less agreement (κ < .41) with the VOE answers. The mean total scores of all pediatric resident training levels were significantly (P < .05) lower than the expert group and significantly (P < .05) higher than the novice group (with exception of the early postgraduate year-1 group). Subjects with more training had higher kappa levels (r = .33, <.001, Spearman) when results were compared among novice, residents, and experts. Conclusions. - We found the VOE to be a feasible and reliable instrument to accurately distinguish novice, resident, and expert level skills in the determination of middle ear effusion status.