Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation

Simon B. Eickhoff, Thomas E. Nichols, Angela R. Laird, Felix Hoffstaedter, Katrin Amunts, Peter T Fox, Danilo Bzdok, Claudia R. Eickhoff

Research output: Contribution to journalArticle

121 Citations (Scopus)

Abstract

Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis.

Original languageEnglish (US)
Pages (from-to)70-85
Number of pages16
JournalNeuroImage
Volume137
DOIs
StatePublished - Aug 15 2016

Fingerprint

Meta-Analysis
Neuroimaging
Sample Size
Publications
Research Personnel
Databases
Brain
Research

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology

Cite this

Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation. / Eickhoff, Simon B.; Nichols, Thomas E.; Laird, Angela R.; Hoffstaedter, Felix; Amunts, Katrin; Fox, Peter T; Bzdok, Danilo; Eickhoff, Claudia R.

In: NeuroImage, Vol. 137, 15.08.2016, p. 70-85.

Research output: Contribution to journalArticle

Eickhoff, SB, Nichols, TE, Laird, AR, Hoffstaedter, F, Amunts, K, Fox, PT, Bzdok, D & Eickhoff, CR 2016, 'Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation', NeuroImage, vol. 137, pp. 70-85. https://doi.org/10.1016/j.neuroimage.2016.04.072
Eickhoff, Simon B. ; Nichols, Thomas E. ; Laird, Angela R. ; Hoffstaedter, Felix ; Amunts, Katrin ; Fox, Peter T ; Bzdok, Danilo ; Eickhoff, Claudia R. / Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation. In: NeuroImage. 2016 ; Vol. 137. pp. 70-85.
@article{b3efa9d69c39401991d7eb339e2c8dfc,
title = "Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation",
abstract = "Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis.",
author = "Eickhoff, {Simon B.} and Nichols, {Thomas E.} and Laird, {Angela R.} and Felix Hoffstaedter and Katrin Amunts and Fox, {Peter T} and Danilo Bzdok and Eickhoff, {Claudia R.}",
year = "2016",
month = "8",
day = "15",
doi = "10.1016/j.neuroimage.2016.04.072",
language = "English (US)",
volume = "137",
pages = "70--85",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation

AU - Eickhoff, Simon B.

AU - Nichols, Thomas E.

AU - Laird, Angela R.

AU - Hoffstaedter, Felix

AU - Amunts, Katrin

AU - Fox, Peter T

AU - Bzdok, Danilo

AU - Eickhoff, Claudia R.

PY - 2016/8/15

Y1 - 2016/8/15

N2 - Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis.

AB - Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis.

UR - http://www.scopus.com/inward/record.url?scp=84969560247&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84969560247&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2016.04.072

DO - 10.1016/j.neuroimage.2016.04.072

M3 - Article

C2 - 27179606

AN - SCOPUS:84969560247

VL - 137

SP - 70

EP - 85

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

ER -