Collinearity, tetrachoric correlation, and criterion group assignment of examinees failing only one performance validity test (PVT): Testing Leonhard’s (2023) PVT critique

Objective: Recent research has challenged use of multiple performance validity tests (PVTs) over concerns of multicollinearity among them and regarding uncertainty about how to handle indeterminate cases that fail only one PVT. This study sought to address these concerns. Method: This study involved secondary analysis of an existing dataset of 313 cases evaluated in a civil forensic context. We examined the relationship of PVT failure and cognitive performance, examined evidence of collinearity among PVTs using various correlation coefficients (i.e. Pearson, tetrachoric, and phi), and compared different operationalizations of valid and invalid performance with logistic regression and receiver operating characteristic analysis. Results: Our findings demonstrate that PVTs do not show pronounced collinearity, as previously claimed. Correcting errors in the calculation of tetrachoric correlations resulted in lower correlations than previously reported (mean r_tet =.57) contradicting claims of redundancy among PVTs. PVT failure was associated with a linear decline in performance on cognitive tests. Logistic regression indicated that use of multiple PVTs improved classification accuracy in contrast to the argument that a single PVT is sufficient. Including Fail = 1 cases in a valid performance group results in minimal changes in diagnostic accuracy compared to when these cases are excluded from analysis as “indeterminate.” Conclusions: These results refute the claim of marked collinearity among PVTs and concerns about their redundancy and rather, show evidence of incremental validity. Findings support continued use of multiple PVTs to improve classification accuracy.