Alternative to Resampling Methods in Maximum Likelihood Estimation for NLMEMs by Borrowing from Bayesian Methodology
Sebastian Ueckert (1,2), Marie-Karelle Riviere (1), France Mentré (1)
(1) IAME, UMR 1137, INSERM and University Paris Diderot, Paris, France; (2) Pharmacometrics Research Group, Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden
Objectives: Asymptotic theory-based statistics such as confidence intervals (CI) from the covariance matrix (COV) and p-values (PVAL) from the Wald test (WALD) are the basis for most model-driven decisions in drug development. For small sample sizes these approximations do not hold and resampling methods like bootstrap (BOOT) or permutation tests (PERM) are employed. Sampling from the Bayesian posterior distribution represents an alternative, if uniform priors are used, as posterior and likelihood become proportional to each other [1]. With the development of Hamiltonian Monte-Carlo (HMC) methods, this approach becomes computationally attractive for hierarchical models [2]. The objective of this work was therefore to compare HMC-based calculation of CI and PVAL with existing approaches.
Methods: A simulation study using a one-compartment model and different study sizes was used to evaluate the performance of HMC-based sampling from the normalized likelihood to calculate CI and PVAL.
CI: evaluation was based on runtime, median CI and coverage, and in comparison to CI obtained via covariance matrix (COV), log-likelihood profiling (LLP) and non-parametric bootstrap (BOOT).
PVAL: evaluation was based on runtime, type-I error and power, and in comparison to PVAL obtained via Wald test (WALD), log-likelihood ratio test (LRT) and permutation test (PERM).
The HMC methods were implemented in R [3] using STAN [4] with improper priors for sampling. Asymptotic theory and resampling-based results were obtained in NONMEM 7.3 [5] using PsN 4.3.17 [6].
Results: The simulations showed good agreement between COV, BOOT and HMC based CIs for large sample sizes. For small sample sizes, COV CIs deviated considerably from CIs obtained with BOOT or HMC. Results for PVAL were similar, with type-I error rates close to the nominal ones for all three methods at large sample sizes, but deviations for WALD at small sample sizes. In terms of computation time the HMC-based methods were >30 times faster than resampling methods.
Conclusions: In this comparison the HMC based methods appear as a very promising approach, showing good agreement with asymptotic results for large and equal or better performance than resampling methods for small sample sizes as well as drastically shorter run times.
This work was supported by the DDMoRe project.
References:
[1] Bolstad WM (2013) Introduction to bayesian statistics. John Wiley & Sons.
[2] Betancourt MJ, Girolami M (2013) Hamiltonian Monte Carlo for hierarchical models. ArXiv13120906 Stat
[3] R Core Team (2014) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria
[4] Stan Development Team (2015) RStan: the r interface to stan, version 2.6.0.
[5] Beal SL, Sheiner LB, Boeckman A, Bauer RJ (2013) NONMEM user’s guides (1989-2013). Icon Development Solutions, Ellicott City, MD, USA
[6] Lindbom L, Pihlgren P, Jonsson NE (2005) PsN-toolkit–a collection of computer intensive statistical methods for non-linear mixed effect modeling using NONMEM. Comput Methods Programs Biomed 79:241–257.
