Precision of Parameter Estimates: Covariance Step ($COV) versus Bootstrap Procedure
Leonid Gibiansky
Metrum Research Group LLC, Tariffville, CT, USA
Objectives: To compare parameter estimates (PE), standard errors (SE) and 95% confidence intervals (CI) on PE obtained by bootstrap procedure and NONMEM $COV step.
Methods: Investigation included 13 pharmacokinetic and pharmacodynamic models identified on real datasets: 5 linear compartmental and 5 Emax models, 1 proportional odds, 1 time-to-event, and 1 Poisson model for count data. Overall, the models included 126 THETA parameters, 31 OMEGA parameters and 9 SIGMA parameters. First, NONMEM V was used to fit the models using either First Order Conditional Estimation method with eta-epsilon Interaction (FOCEI) or LAPLACIAN method with LIKELIHOOD option. All models converged and provided PE, SE, and eigenvalues of the correlation matrix (EV). Separate runs with MATRIX=S and R options were used to compute SE and EV and compare them with SE and EV obtained by the default option. Then, 1000 bootstrap datasets stratified by covariates of interest were created for each model, and PE were obtained for each of these datasets. Results (irrespectively of convergence) were used to obtain 95%CI on PE. Medians and standard deviations of bootstrap parameter distributions were treated as bootstrap PE and SE. Correlation matrix of the bootstrap PE was estimated, and it's eigenvalues computed for each model. Finally, results of $COV step and bootstrap procedures (PE, SE, CI and EV) for all models together were compared visually and by computing correlation coefficients.
Results: Results indicate surprisingly good agreement of PE, SE, CI and EV obtained by the two procedures: correlation coefficients exceeded 0.98. Correlation coefficients for CI normalized by PE exceeded 0.83 for all but two models. Agreement was even better for the well-estimated parameters (those with relative standard error RSE=100*SE/PE not exceeding 20%). Contrary to the expectations, agreement of the results for variance parameters was similar to those for THETA parameters. For SIGMA parameters, nearly perfect agreement was observed.
Conclusions: In the variety of examples with different PK and PD models, NONMEM FOCEI and LAPLACE estimation methods in conjunction with the default option of the $COV step provided excellent approximation of the estimation precision. This applies not only to the model parameters but also to variances of the intersubject variability and the residual error. Nearly perfect agreement with bootstrap results was observed for the well-estimated parameters with RSE < 20%. MATRIX=S or R options also provided good approximation of the estimation precision.
References: Similar problems were discussed in Dartois C, et al. Evaluation of Uncertainty Parameters Estimated by Different Population PK Software and Methods. J Pharmacokinet Pharmacodyn. Jan 2007.
