The OFVPPC: A simulation objective function based diagnostic
A. Largajolli (1), S. Jönsson (1), M. O. Karlsson (1)
(1) Department of Pharmaceutical Biosciences, Uppsala University, Sweden
Objectives: Ability to assess the model adequacy through appropriate diagnostics is crucial in pharmacometrics. Here we suggest a simulation-based model diagnostic, the OFVPPC, which relies on the information content present in the objective function (OFV) to detect model misspecifications, outliers and estimation method limits.
Methods: From final model estimates 1000 stochastic simulations and estimations (SSE) implemented in PsN (1) were performed in NONMEM (2). The population (obsPOFV) and individual OFVs (obsIOFV) based on observed data were compared to the corresponding distributions of OFVs based on simulated data (simPOFV and simIOFV). In the SSE, a full estimation or an evaluation (MAXEVAL=0) of the model was performed. Both real data sets and simulations were used to explore these diagnostics.
Results: Model misspecification, both in the structural and stochastic model components, were often, but not always, clearly identified with POFVPPC. Indication of misspecification was evident from the obsPOFV being higher than the distribution given by the simPOFV. When the first-order method was used to estimate parameters, the short-coming of the method, increasingly for highly nonlinear models, was typically evident as a lower obsPOFV compared to the simPOFV distribution. Individual subjects with outlying data could often be identified from their obsIOFV being different from the corresponding distribution of simIOFV. In general, the same conclusion of misspecification regarding POFV and IOFV could be made regardless of whether full estimation was made of simulated data, or if only an evaluation (MAXEVAL=0) was performed. For IOFV, however, the use of MCETA, a new NONMEM feature that allows trying different set of initial estimates for the individual MAP estimation, could sometimes provide more reliable results when only evaluation was performed.
Conclusions: The OFV information, a sensitive measure that sums up the model fit, was exploited to build a diagnostic tool to be used during model building and for detection of outliers. The results of the OFVPPC are easy to visualize but similar to other simulation based diagnostic relies on the final critical judgement of the user.
References:
[1] L. Lindbom, P. Pihlgren, and E. N. Jonsson. Psntoolkit: a collection of computer intensive statistical methods for non-linear mixed effect modeling using nonmem. Computer Methods and Programs in Biomedicine, 79(3):241, Sept. 2005.
[2] S. Beal, L. Sheiner, A. Boeckmann, and R. Bauer. Nonmem user's guides (1989-2014). Technical report, Icon Development Solutions, Ellicott City, MD, USA, 2014.
