A new exact test to globally assess a population PK and/or PD model
Laffont C. M., Concordet D.
UMR181 Physiopathologie et Toxicologie Expérimentales, INRA, ENVT, Toulouse, France.
Objectives: Standardised predictions errors are provided by most software (WRES in NONMEM; PWRES in MONOLIX) and are widely used as a diagnostic tool in routine [1]. They can be calculated in several ways which are more or less accurate [2,3], but their main limitation is that they come from the theory of linear models and do not apply to non-linear models [2]. In that context, new metrics on observations have been proposed [2,4]. Normalised prediction distribution errors (NPDE) represent a major improvement as they do not rely on any approximation of the model and are uncorrelated within an individual [2]. However, absence of correlation does not imply data independence (unless individual vectors of observations are Gaussian, which is barely the case for non-linear models). It results that, while NPDE actually follow a N(0,1) distribution at each observation time, their joint distribution is not standard Gaussian. Our objective is thus to develop an exact test that overcomes this issue of data dependence and allows to globally assess a population PK/PD model.
Methods: As for NPDE [2], we calculate for each individual i the vector of standardised predictions errors (Ui) using the expectations and full variance matrix estimated empirically over K simulations. We use a random projection method (see [5] for an application) that allows an easy analysis of dependent data. Briefly, we project Ui on random directions drawn from a uniform distribution on the unit sphere. We then use these projections to perform a global test and propose an easy diagnostic graph that does not require a subjective interpretation: the QQ "ring" plot. Our test compares, using the sup-norm, the empirical distribution of projected Ui with their distribution under the null hypothesis (H0). Simulation studies were performed with different PK or PK/PD models, under H0 and several alternative hypotheses (H1), to evaluate the level and power of the test. The performances of our test were compared with those of Kolmogorov-Smirnov test applied to NPDE, WRES and PWRES (population mean predictions) for a N(0,1) distribution, under H0 and H1 (NPDE only).
Results: Our test showed very good performances both in terms of type I error and power. The performances of NPDE were also good but revealed insufficiencies for highly non-Gaussian models. In agreement with previous work [2], the performances of both WRES and PWRES under H0 were very poor.
Conclusions: We have developed an exact test for evaluation of population models. Its good theoretical properties were confirmed by several simulation studies using different PK and/or PD models. We also propose a very innovative graph as a global diagnostic tool.
References:
[1] Brendel K et al. Are population pharmacokinetic and/or pharmacodynamic models adequately evaluated? A survey of the literature from 2002 to 2004. Clin Pharmacokinet 2007; 46:221-234.
[2] Brendel K. et al. Metrics for external model evaluation with an application to the population pharmacokinetics of gliclazide. Pharm Res 2006; 23:2036-2049.
[3] Hooker AC et al. Conditional weighted residuals (CWRES): a model diagnostic for the FOCE method. Pharm Res 2007; 24:2187-2197.
[4] Mentré F and Escolano S. Prediction discrepancies for the evaluation of nonlinear mixed-effects models. J Pharmacokinet Pharmacodyn 2006; 33:345-367.
[5] Cuesta-Albertos JA et al. The random projection method in goodness of fit for functional data. Computational Statistics & Data Analysis 2007; 51:4814-4831.
