Intraclass correlation as a reliability check on nominal data

The procedure described yields estimates of the reliability with which judges employ nominal categories, when such categorical assignment has been sequentially dichotomized into a series of orthogonal contrasts. The program is written in Fortran IV for the CDC-6600. The user must specify k - 1 sets of weights for contrasts among the k categories. The program then assigns these weights as scores to each subject, and computes a one-way analysis of variance for each of the k - 1 sets of scores. Output from the program includes the scores assigned to each subject for each of the contrasts, the analysis of variance for each contrast, the intraclass correlation, and the weights for each contrast. Current limits are 20 categories, 100 subjects, and 10 judges, but these can be enlarged by the user. Program and sample data are available from the authors on request. Comment cards in the program indicate required input.