Bootstrap methods for estimating uncertainty of parameters in mixed-effects models
Hoai-Thu Thai (1), France Mentré (1), Nicholas H.G. Holford (3), Christine Veyrat-Follet (2), Emmanuelle Comets (1)
(1) INSERM, UMR 738, F-75018 Paris, France; Univ Paris Diderot, Sorbonne Paris Cité, UMR 738, F-75018 Paris, France; (2) Drug Disposition Department, Sanofi, Paris, France; (3) Department of Pharmacology and Clinical Pharmacology, University of Auckland, Auckland, New Zealand
Objectives: Nonparametric case bootstrap is frequently used in PK/PD for estimating standard error (SE) and confidence interval (CI) of parameters [1-2]. Residual bootstraps resampling both random effects and residuals are an alternative approach to case bootstrap which resamples entire individuals [3-4]. These methods have not been well studied in mixed-effects models (MEM). We aimed to study and propose appropriate bootstrap methods in MEM and to evaluate their performance by simulation using examples of disease progression model in Parkinson's disease [5] (for linear MEM) and PK model of aflibercept (Zaltrap®) [6], a novel anti-VEGF drug (for nonlinear MEM).
Methods: Different bootstraps accounting for between-subject and residual variabilities were implemented in R 2.14. Corrections of random effects and residuals for variance underestimation were investigated [7]. The bootstrap performances were first assessed in LMEM with homoscedastic error by a simulation (k=1000 replicates and B=1000 bootstrap samples/replicate) with 3 balanced designs (rich, sparse, large error). The best bootstraps in LMEM were then evaluated in the NLMEM with heteroscedastic error by a simulation (k=100/B=1000) with 2 balanced (frequent/sparse) and 1 unbalanced designs. Bootstraps were compared in terms of bias of parameters, SE and coverage rate of 95% CI. R 2.14 and MONOLIX 4.1 were used to fit the data in LMEM and NLMEM, respectively.
Results: Our simulations showed a good performance of the case bootstrap and the nonparametric/parametric residual bootstraps with a correction for variance underestimation in LMEM [8]. In NLMEM, these methods performed well in the balanced designs, except for the sparse design where they greatly overestimated SE of a parameter estimate having a skewed distribution. In the unbalanced design, the case bootstrap overestimated the SE of this parameter and the nonparametric residual bootstrap overestimated the SE of variances even with stratification on frequent/sparse sampling. The asymptotic method performed well in most cases, except for low coverage rates of highly nonlinear parameters.
Conclusion: The bootstraps only provide better estimates of uncertainty in NLMEM with high nonlinearity compared to the asymptotic method. The nonparametric residual bootstrap works as well as the case bootstrap. However, they may face practical problems, e.g skewed distributions in parameter estimates and unbalanced designs where stratification may be insufficient.
References:
[1] Ette EI. Stability and performance of a population pharmacokinetic model. J Clin Pharmacol 1997; 37(6):486-495.
[2] Parke J, Holford NHG, Charles BG. A procedure for generating bootstrap samples for the validation of nonlinear mixed-effects population models. Comput Methods Programs Biomed. 1999; 59:19-29.
[3] Das S, Krishen A. Some bootstrap methods in nonlinear mixed-effects models. J Stat Plan Inference 1999; 75: 237-245.
[4] Ocana J, El Halimi R, Ruiz de Villa MC, Sanchez JA. Bootstrapping repeated measures data in a nonlinear mixed-models context. Mathematics Preprint Series 2005.
[5] Holford NHG, Chan PLS, Nutt JG, Kieburtz K, Shoulson I and Parkinson Study Group. Disease progression and pharmacodynamics in Parkinson disease-evidence for functional protection with Levodopa and other treatments. J Pharmacokinet Pharmacodyn. 2006; 33 (3): 281-311.
[6] Thai H-T, Veyrat-Follet C, Vivier N, Dubruc C, Sanderink G, Mentré F, Comets E. A mechanism-based model for the population pharmacokinetics of free and bound aflibercept in healthy subjects. Br J Clin Pharmacol 2011, 72(3): 402-414.
[7] Carpenter JR, Goldstein H, Rasbash J. A novel bootstrap procedure for assessing the relationship between class size and achievement. Appl Statist 2003; 52:431-443. 26.
[8] Thai H-T, Veyrat-Follet C, Mentré F and Comets E. A comparison of bootstrap approaches for estimating standard errors of parameters in linear mixed effects models. Pharm Stat 2013, doi :10.1002/pst.1561.
