Evaluating Bootstrap Methods in Nonlinear Mixed Effect Models (NMEM) Using a PK Model
Yingwen Dong (1), Kuenhi Tsai (2)
(1) Biogen Idec (2) Merck & Co., Inc.
Objectives: The bootstrap methods in PK/PD studies have not investigated the bias and reliability of various bootstrap confidence interval (CI) methods or the application of using the parametric (residual) bootstrap method performed [1]. In addition, the bootstrap distribution and CI of PK/PD parameters are often compared to the parameter estimation and its derived CI of the original data as a tool of model validation. The legitimacy of this approach is explored here. The objectives are (1) to utilize statistical criteria to investigate the bias and reliability of popular bootstrap CI methods, (2) to compare nonparametric and parametric bootstrap (residual) methods, and (3) to assess whether bootstrap distribution and CI can be used for model validation.
Methods: Nonparametric (sample with replacement) and parametric (residual) bootstrap methods are investigated. Bootstrap confidence intervals were constructed using percentile, t interval, bias-corrected, bias-corrected and accelerated, and hybrid approaches. The simulated PK model was a one compartment model with first-order absorption. Two sampling schemes with small and moderate number of subjects were investigated. 100 replicates of the dataset were generated for each scheme with 100 times of bootstrapped samples for each replication. The parameters were assessed using bias, standard deviation, root mean squared error, and the coverage probability of 95% CIs. The performance of bootstrapping was also evaluated in the event that an inter-subject variability on the absorption rate was incorrectly specified.
Results: The nonparametric method is superior to the parametric method in bias and CI coverage of the clearance and its inter-subject variability. The standard normal method has a better coverage than the rest of the methods. All bootstrap CI methods perform equivalently well in the nonparametric method. When the model is incorrectly specified in the smaller random error case, all parameters except the intra-subject variability term have a good coverage with all the methods. The coverage of the CIs for the intra-subject variability is low for the standard normal method and all bootstrap CIs.
Conclusions: The nonparametric method is concluded to be better than the parametric method may be due to the limitation of the current parametric bootstrap method that only resamples the intra-subject random error. However, the coverage is no better than the standard normal method in both rich and sparse sampling. Since most PK/PD models assume normal or log-normal distribution random errors, application of the bootstrap CI methods to get better estimation of CI is questionable. The similar parameter estimation and CI coverage of the original data set and bootstrap data in misspecification results show bootstrap CIs cannot serve as a tool for model validation if the model is incorrectly specified in the original data.
Reference:
[1] William, P., and Y. Kim. 2007 Resampling techniques and their application to pharmacometrics, in Pharmacometrics: The Science of Quantitative Pharmacology, E. Ette and P. Williams (Eds.) Wiley, Hoboken, NJ, 2007, Chapter 15
