Performance validity in older adults: Observed versus predicted false positive rates in relation to number of tests administered

Introduction: This study examined false positive rates on embedded performance validity tests (PVTs) in older adults grouped by cognitive status. Method: The research design involved secondary analysis of data from the National Alzheimer’s Coordinating Center database. Participants (N = 22,688) were grouped by cognitive status: normal (n = 10,319), impaired (n = 1,194), amnestic or nonamnestic mild cognitive impairment (MCI; n = 5,414), and dementia (n = 5,761). Neuropsychological data were used to derive 5 PVTs. Results: False positive rates on individual PVTs ranged from 3.3 to 26.3% with several embedded PVTs showing acceptable specificity across groups. The proportion of participants failing two or more PVTs varied by cognitive status: normal (1.9%), impaired (6.6%), MCI (13.2%), and dementia (52.8%). Comparison of observed and predicted false positive rates at different specificity levels (.85 or.90) demonstrated significant differences in all comparisons. In normal and impaired groups, predicted rates were higher than observed rates. In the MCI group, predicted and observed comparisons varied: Predicted rates were higher with specificity at.85 and lower with specificity at.90. In the dementia group, predicted rates underestimated observed rates. Conclusions: Despite elevated false positives in conditions involving severe cognitive compromise, several measures retain acceptable specificity regardless of cognitive status. Predicted false positive rates based on the number of PVTs administered were not observed empirically. These findings do not support the utility of simulated data in predicting false positive rates in older adults.