TY - JOUR
T1 - Performance validity in older adults
T2 - Observed versus predicted false positive rates in relation to number of tests administered
AU - Davis, Jeremy J.
N1 - Funding Information:
The NACC database is funded by the National Institute on Aging/National Institutes of Health (NIA/NIH) [grant number U01 AG016976]. NACC data are contributed by the NIA-funded ADCs: [grant number P30 AG019610] (principal investigator, PI, Eric Reiman, MD), [grant number P30 AG013846] (PI Neil Kowall, MD), [grant number P50 AG008702] (PI Scott Small, MD), [grant number P50 AG025688] (PI Allan Levey, MD, PhD), [grant number P50 AG047266] (PI Todd Golde, MD, PhD), [grant number P30 AG010133] (PI Andrew Saykin, PsyD), [grant number P50 AG005146] (PI Marilyn Albert, PhD), [grant number P50 AG005134] (PI Bradley Hyman, MD, PhD), [grant number P50 AG016574] (PI Ronald Petersen, MD, PhD), [grant number P50 AG005138] (PI Mary Sano, PhD), [grant number P30 AG008051] (PI Thomas Wisniewski, MD), [grant number P30 AG013854] (PI M. Marsel Mesulam, MD), [grant number P30 AG008017] (PI Jeffrey Kaye, MD), [grant number P30 AG010161] (PI David Bennett, MD), [grant number P50 AG047366] (PI Victor Henderson, MD, MS), [grant number P30 AG010129] (PI Charles DeCarli, MD), [grant number P50 AG016573] (PI Frank LaFerla, PhD), [grant number P50 AG005131] (PI James Brewer, MD, PhD), [grant number P50 AG023501] (PI Bruce Miller, MD), [grant number P30 AG035982] (PI Russell Swerdlow, MD), [grant number P30 AG028383] (PI Linda Van Eldik, PhD), [grant number P30 AG053760] (PI Henry Paulson, MD, PhD), [grant number P30 AG010124] (PI John Trojanowski, MD, PhD), [grant number P50 AG005133] (PI Oscar Lopez, MD), [grant number P50 AG005142] (PI Helena Chui, MD), [grant number P30 AG012300] (PI Roger Rosenberg, MD), [grant number P30 AG049638] (PI Suzanne Craft, PhD), [grant number P50 AG005136] (PI Thomas Grabowski, MD), [grant number P50 AG033514] (PI Sanjay Asthana, MD, FRCP), [grant number P50 AG005681] (PI John Morris, MD), [grant number P50 AG047270] (PI Stephen Strittmatter, MD, PhD).
Publisher Copyright:
© 2018, © 2018 Informa UK Limited, trading as Taylor & Francis Group.
PY - 2018/11/26
Y1 - 2018/11/26
N2 - Introduction: This study examined false positive rates on embedded performance validity tests (PVTs) in older adults grouped by cognitive status. Method: The research design involved secondary analysis of data from the National Alzheimer’s Coordinating Center database. Participants (N = 22,688) were grouped by cognitive status: normal (n = 10,319), impaired (n = 1,194), amnestic or nonamnestic mild cognitive impairment (MCI; n = 5,414), and dementia (n = 5,761). Neuropsychological data were used to derive 5 PVTs. Results: False positive rates on individual PVTs ranged from 3.3 to 26.3% with several embedded PVTs showing acceptable specificity across groups. The proportion of participants failing two or more PVTs varied by cognitive status: normal (1.9%), impaired (6.6%), MCI (13.2%), and dementia (52.8%). Comparison of observed and predicted false positive rates at different specificity levels (.85 or.90) demonstrated significant differences in all comparisons. In normal and impaired groups, predicted rates were higher than observed rates. In the MCI group, predicted and observed comparisons varied: Predicted rates were higher with specificity at.85 and lower with specificity at.90. In the dementia group, predicted rates underestimated observed rates. Conclusions: Despite elevated false positives in conditions involving severe cognitive compromise, several measures retain acceptable specificity regardless of cognitive status. Predicted false positive rates based on the number of PVTs administered were not observed empirically. These findings do not support the utility of simulated data in predicting false positive rates in older adults.
AB - Introduction: This study examined false positive rates on embedded performance validity tests (PVTs) in older adults grouped by cognitive status. Method: The research design involved secondary analysis of data from the National Alzheimer’s Coordinating Center database. Participants (N = 22,688) were grouped by cognitive status: normal (n = 10,319), impaired (n = 1,194), amnestic or nonamnestic mild cognitive impairment (MCI; n = 5,414), and dementia (n = 5,761). Neuropsychological data were used to derive 5 PVTs. Results: False positive rates on individual PVTs ranged from 3.3 to 26.3% with several embedded PVTs showing acceptable specificity across groups. The proportion of participants failing two or more PVTs varied by cognitive status: normal (1.9%), impaired (6.6%), MCI (13.2%), and dementia (52.8%). Comparison of observed and predicted false positive rates at different specificity levels (.85 or.90) demonstrated significant differences in all comparisons. In normal and impaired groups, predicted rates were higher than observed rates. In the MCI group, predicted and observed comparisons varied: Predicted rates were higher with specificity at.85 and lower with specificity at.90. In the dementia group, predicted rates underestimated observed rates. Conclusions: Despite elevated false positives in conditions involving severe cognitive compromise, several measures retain acceptable specificity regardless of cognitive status. Predicted false positive rates based on the number of PVTs administered were not observed empirically. These findings do not support the utility of simulated data in predicting false positive rates in older adults.
KW - Dementia
KW - false positive rate
KW - older adults
KW - performance validity
UR - http://www.scopus.com/inward/record.url?scp=85047157072&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85047157072&partnerID=8YFLogxK
U2 - 10.1080/13803395.2018.1472221
DO - 10.1080/13803395.2018.1472221
M3 - Article
C2 - 29779432
AN - SCOPUS:85047157072
VL - 40
SP - 1013
EP - 1021
JO - Journal of Clinical and Experimental Neuropsychology
JF - Journal of Clinical and Experimental Neuropsychology
SN - 0168-8634
IS - 10
ER -