NONMEM Termination Status is Not an Important Indicator of the Quality of Bootstrap Parameter Estimates
Holford, Nick, Carl Kirkpatrick, Steve Duffull
Dept of Pharmacology & Clinical Pharmacology, University of Auckland, Auckland, New Zealand, School of Pharmacy, University of Queensland, Brisbane, Australia
Background: NONMEM’s ability to compute the covariance matrix of the estimates has been recommended as a requirement to accept the adequacy of the parameter estimates. We have explored empirically the relationship between NONMEM’s termination status and the distribution of bootstrap parameter estimates.
Methods: A non-parametric bootstrap procedure was used to generate at least 1000 sets of parameter estimates for 13 NONMEM problems. NONMEM termination status was assigned types: 1. convergence successful with $covariance, 2. successful with failed $covariance, 3. failed with rounding errors, 4. failed with infinite objective function, 5. failed for other reasons, 6. all successful types, 7. all failed types, 8. all types. Empirical 25, 50 and 75% quartiles were used to describe the parameter estimates for each termination type. Bias for each termination type was calculated relative to each type 1 quartile. The correlation matrix of the bootstrap parameter estimates was used to compute a condition number (ratio of highest to lowest eigenvalue). A minimum of 20 bootstrap replicates was required to compute bias and 100 replicates to compute the condition number.
Results: All problems had a bootstrap condition number less than 500. There was no correlation between the % of Type 1 runs and the condition number. The percentage of problems with absolute bias less than 10% was 77,92 and 85 (25,50,75,% quartiles) when all successful runs (Type 6) were used. Corresponding % were 69,77,69 for all runs (Type 8).
Conclusions: 1. Selecting all successful bootstrap replicates (Type 6) provides accurate and precise description of NONMEM parameter estimates. 2. Using all replicates irrespective of termination status (Type 8) provides reasonably accurate descriptions without the requirement for successful $covariance computation.