Biologically relevant simulations for validating risk models under small-sample conditions

Alex F. Bokov, Laura S. Manuel, Alfredo Tirado-Ramos, Jonathan A Gelfond, Scott D. Pletcher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In designing scientific experiments, power analysis is too often given a superficial treatment - choice of sample size is often made based on idealized distributions and simplistic tests that do not reflect the real-world constraints under which the actual data will be collected. We have developed a general Monte Carlo framework for two-group comparisons which samples points from a two-dimensional parameter space and at each point generates simulated datasets which are compared to simulated datasets for a 'control group' at a fixed point in the parameter space. Rather than uniformly sampling this parameter space, our algorithm rapidly converges on a contour corresponding to the smallest detectable difference for the sample size of interest. We apply this framework, implemented as an R library called PowerTrip, to directly comparing the performance and sensitivity to sample size of the Gompertz survival model to several other commonly used survival models. We find that the Gompertz mortality model performs approximately as well as the Weibull and the Cox models throughout most of the parameter space, but outperforms the competing models in cases where initial mortality rate (IMR) and rate of acceleration (RoA) change in opposite directions.

Original languageEnglish (US)
Title of host publication2017 IEEE Symposium on Computers and Communications, ISCC 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages290-295
Number of pages6
ISBN (Electronic)9781538616291
DOIs
StatePublished - Sep 1 2017
Event2017 IEEE Symposium on Computers and Communications, ISCC 2017 - Heraklion, Greece
Duration: Jul 3 2017Jul 7 2017

Other

Other2017 IEEE Symposium on Computers and Communications, ISCC 2017
CountryGreece
CityHeraklion
Period7/3/177/7/17

Fingerprint

Small Sample
Parameter Space
Sample Size
Survival Model
Simulation
Cox Model
Power Analysis
Sample point
Mortality Rate
Weibull
Mortality
Model
Fixed point
Converge
Sampling
Experiment
Experiments
Framework

Keywords

  • Maximum likelihood estimation
  • Monte Carlo methods
  • Statistics

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Mathematics(all)
  • Computer Science Applications
  • Computer Networks and Communications

Cite this

Bokov, A. F., Manuel, L. S., Tirado-Ramos, A., Gelfond, J. A., & Pletcher, S. D. (2017). Biologically relevant simulations for validating risk models under small-sample conditions. In 2017 IEEE Symposium on Computers and Communications, ISCC 2017 (pp. 290-295). [8024544] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISCC.2017.8024544

Biologically relevant simulations for validating risk models under small-sample conditions. / Bokov, Alex F.; Manuel, Laura S.; Tirado-Ramos, Alfredo; Gelfond, Jonathan A; Pletcher, Scott D.

2017 IEEE Symposium on Computers and Communications, ISCC 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 290-295 8024544.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Bokov, AF, Manuel, LS, Tirado-Ramos, A, Gelfond, JA & Pletcher, SD 2017, Biologically relevant simulations for validating risk models under small-sample conditions. in 2017 IEEE Symposium on Computers and Communications, ISCC 2017., 8024544, Institute of Electrical and Electronics Engineers Inc., pp. 290-295, 2017 IEEE Symposium on Computers and Communications, ISCC 2017, Heraklion, Greece, 7/3/17. https://doi.org/10.1109/ISCC.2017.8024544
Bokov AF, Manuel LS, Tirado-Ramos A, Gelfond JA, Pletcher SD. Biologically relevant simulations for validating risk models under small-sample conditions. In 2017 IEEE Symposium on Computers and Communications, ISCC 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 290-295. 8024544 https://doi.org/10.1109/ISCC.2017.8024544
Bokov, Alex F. ; Manuel, Laura S. ; Tirado-Ramos, Alfredo ; Gelfond, Jonathan A ; Pletcher, Scott D. / Biologically relevant simulations for validating risk models under small-sample conditions. 2017 IEEE Symposium on Computers and Communications, ISCC 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 290-295
@inproceedings{1d43615df14f42a99fde1b8890616391,
title = "Biologically relevant simulations for validating risk models under small-sample conditions",
abstract = "In designing scientific experiments, power analysis is too often given a superficial treatment - choice of sample size is often made based on idealized distributions and simplistic tests that do not reflect the real-world constraints under which the actual data will be collected. We have developed a general Monte Carlo framework for two-group comparisons which samples points from a two-dimensional parameter space and at each point generates simulated datasets which are compared to simulated datasets for a 'control group' at a fixed point in the parameter space. Rather than uniformly sampling this parameter space, our algorithm rapidly converges on a contour corresponding to the smallest detectable difference for the sample size of interest. We apply this framework, implemented as an R library called PowerTrip, to directly comparing the performance and sensitivity to sample size of the Gompertz survival model to several other commonly used survival models. We find that the Gompertz mortality model performs approximately as well as the Weibull and the Cox models throughout most of the parameter space, but outperforms the competing models in cases where initial mortality rate (IMR) and rate of acceleration (RoA) change in opposite directions.",
keywords = "Maximum likelihood estimation, Monte Carlo methods, Statistics",
author = "Bokov, {Alex F.} and Manuel, {Laura S.} and Alfredo Tirado-Ramos and Gelfond, {Jonathan A} and Pletcher, {Scott D.}",
year = "2017",
month = "9",
day = "1",
doi = "10.1109/ISCC.2017.8024544",
language = "English (US)",
pages = "290--295",
booktitle = "2017 IEEE Symposium on Computers and Communications, ISCC 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Biologically relevant simulations for validating risk models under small-sample conditions

AU - Bokov, Alex F.

AU - Manuel, Laura S.

AU - Tirado-Ramos, Alfredo

AU - Gelfond, Jonathan A

AU - Pletcher, Scott D.

PY - 2017/9/1

Y1 - 2017/9/1

N2 - In designing scientific experiments, power analysis is too often given a superficial treatment - choice of sample size is often made based on idealized distributions and simplistic tests that do not reflect the real-world constraints under which the actual data will be collected. We have developed a general Monte Carlo framework for two-group comparisons which samples points from a two-dimensional parameter space and at each point generates simulated datasets which are compared to simulated datasets for a 'control group' at a fixed point in the parameter space. Rather than uniformly sampling this parameter space, our algorithm rapidly converges on a contour corresponding to the smallest detectable difference for the sample size of interest. We apply this framework, implemented as an R library called PowerTrip, to directly comparing the performance and sensitivity to sample size of the Gompertz survival model to several other commonly used survival models. We find that the Gompertz mortality model performs approximately as well as the Weibull and the Cox models throughout most of the parameter space, but outperforms the competing models in cases where initial mortality rate (IMR) and rate of acceleration (RoA) change in opposite directions.

AB - In designing scientific experiments, power analysis is too often given a superficial treatment - choice of sample size is often made based on idealized distributions and simplistic tests that do not reflect the real-world constraints under which the actual data will be collected. We have developed a general Monte Carlo framework for two-group comparisons which samples points from a two-dimensional parameter space and at each point generates simulated datasets which are compared to simulated datasets for a 'control group' at a fixed point in the parameter space. Rather than uniformly sampling this parameter space, our algorithm rapidly converges on a contour corresponding to the smallest detectable difference for the sample size of interest. We apply this framework, implemented as an R library called PowerTrip, to directly comparing the performance and sensitivity to sample size of the Gompertz survival model to several other commonly used survival models. We find that the Gompertz mortality model performs approximately as well as the Weibull and the Cox models throughout most of the parameter space, but outperforms the competing models in cases where initial mortality rate (IMR) and rate of acceleration (RoA) change in opposite directions.

KW - Maximum likelihood estimation

KW - Monte Carlo methods

KW - Statistics

UR - http://www.scopus.com/inward/record.url?scp=85030531050&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030531050&partnerID=8YFLogxK

U2 - 10.1109/ISCC.2017.8024544

DO - 10.1109/ISCC.2017.8024544

M3 - Conference contribution

AN - SCOPUS:85030531050

SP - 290

EP - 295

BT - 2017 IEEE Symposium on Computers and Communications, ISCC 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -