Evaluation Of The Nonparametric Estimation Method In NONMEM VI: Application To Real Data
Paul G Baverel, Radojka M Savic, Justin J Wilkins, Mats O Karlsson,
Division of Pharmacokinetics and Drug Therapy, Department of Pharmaceutical Biosciences, Faculty of Pharmacy, Uppsala University, Sweden
Background: A nonparametric estimation method is available in NONMEM VI. A previous study indicated that this new feature showed promising properties when analyzing simulated data in that the nonparametric distribution of the parameter estimates matched the true distribution used in simulation.[1] However, experience with real data sets is so far limited.Objectives: The aim of this study was to evaluate the predictive performance of the nonparametric estimation method in comparison with standard parametric methods when applied to real data sets.
Methods: Four methods for estimating model parameters and parameter distributions (FO, FOCE, nonparametric preceded by FO (NONP-FO) and nonparametric preceded by FOCE (NONP-FOCE)) were compared for 30 models previously developed using real data and the FO or FOCE methods in NONMEM. These 30 models included 21 PK (one-, two-, and three-compartment models) and 9 PD models (e.g. direct inhibitory Emax models, indirect effect models). A brief description of the data sets is given as follow: range of number of subjects from 8 to 637 and average observations per subject were from 2 to 45. Numerical predictive checks (NPCs) were used to test the appropriateness of each model.[2] Up to 1000 new datasets were simulated from each model and with each estimation method and used to construct 95% and 50% prediction intervals (PIs). The percentages of outliers (expected values being 5% and 50% for the 95% and 50% PIs, respectively) were obtained, as well as the ratio of points above to below the median. In order to appreciate the predictive performance of each method, the mean absolute error (MAE, %) and the mean error (ME, %) were computed as indicators of imprecision and bias and compared using statistical t-tests.
Results: With both FO and FOCE, there was a tendency to simulate more variability than observed which was not the case with NONP-FO and NONP-FOCE. Overall, less imprecision and less bias were observed with nonparametric than parametric methods. Furthermore, the t-tests performed revealed that the imprecision related to the ratio of points above / below the medians was significantly lower (p<0.05) with nonparametric methods than with parametric ones. Regarding the PIs percentages of outliers, NONP-FOCE displayed significantly less imprecision than FOCE at 50%PI but not at 95%PI. For FO, no significant difference was found in this aspect but NONP-FO showed significantly less bias than FO at 95%PI.
Conclusions: When applied to real data sets and evaluated using a predictive check, the nonparametric estimation methods in NONMEM VI performed better than the corresponding parametric methods (FO or FOCE) with less imprecision and less bias for the majority of the outcomes investigated in this study.
References:
[1] Savic RM, Kjellsson M, Karlsson MO. Evaluation of the nonparametric estimation method in NONMEM VI beta PAGE 15 (2006) Abstr 937 [www.page-meeting.org/?abstract=937]
[2] Wilkins JJ, Karlsson MO, Jonsson EN. Patterns and power for the visual predictive check. PAGE 15 (2006) Abstr 1029 [www.page-meeting.org/?abstract=1029]
