Speech tasks and interrater reliability in perceptual voice evaluation

Objective/Hypothesis The optimal selection of speech task is essential for more reliable perceptual ratings and a better understanding of the perceptual qualities of pathologic voices. Nevertheless, researchers have rarely explored this issue using the GRBAS scale. This study investigates the effect of speech task selection on interrater reliability during perceptual voice assessment. Study Design Experimental study. Methods Sixty subjects, 39 dysphonic subjects and 21 normal controls, performed 13 speech tasks including three 5-second sustained vowel sounds (/α/, /i/, and /u/) each at three pitch levels (high, habitual, and low), maximum phonation of the vowel /α/, pitch glide, counting from 1 to 10, and oral reading of the Rainbow Passage. A group of 18 graduate students in speech-language pathology served as perceptual judges and rated the dysphonic severity for the speech samples based on three parameters in the GRBAS scale - Grade, Roughness, and Breathiness. The formalism of the AC₁ statistic proposed by Gwet was applied to determine relative reliability between the speech tasks and the raters. Results The counting task and sustained vowel /α/ in high, habitual, and low registers exhibited the most reproducibility and consequently the highest reliability statistic. Conclusions The counting task and sustained /α/ phonation are the optimal tasks for perceptual voice judgment in regard to interrater reliability. Future perceptional studies may benefit from this finding to determine the relationship between speech task selection and the validity of any given perceptual rating system in terms of sensitivity and specificity.