You have problems to interpret VPC? Try VIPER!
Dalia Khachman, Celine M. Laffont and Didier Concordet
UMR181 Physiopathologie et Toxicologie Expérimentales, INRA, ENVT, Toulouse, France.
Objectives: Model evaluation has become a key component of the modelling process. In this respect, Visual Predictive Checks (VPC) are very popular as they allow direct comparison of observations (concentration or effects) with their predictive distribution under the model, diagnosing both structural and random effects’ models [1]. Despite these advantages, VPC present several limitations [2,3]. First, their interpretation is quite subjective since it is not always possible to know the number of observations that should be outside prediction intervals due to correlations within individuals. Second, stratification of the data is often necessary in case of different dosage regimens and whenever covariates are included in the model. Such stratification may lead to uninformative VPC as several VPC plots are performed with fewer data per plot. In that context, we propose a new graphical tool called VIPER (VIsual Predictive Extended Residuals). This new tool was designed to perform an accurate and easier evaluation of the model in a VPC-like manner without VPC drawbacks.
Methods: For each individual i, we calculate from the observations the vector of standardised predictions errors (Ui) using the expectation and diagonal variance matrix estimated empirically over k simulations. We then calculate the sup-norm of Ui, keeping information on the sign, and compare this sup-norm with the corresponding predictive distribution under the model (taking into account the subject’s characteristics). Since individuals are independent, so are their sup-norms. Therefore, it was possible to represent all sup-norms of all individuals on a single graph (provided some rescaling) and define prediction intervals so that the overall probability of observing more than a given amount of data points out of the prediction intervals was less than 0.001 under the null hypothesis (H0). The performance characteristics of VIPER were tested using various population PK models under H0 and several alternative hypotheses (H1).
Results: VIPER showed good performances for global model evaluation and allowed to overcome VPC-related issues in all tested models. Advantages towards other visual tools (NPC [1], PC-VPC [3]) are discussed.
Conclusions: Based on the present evaluation, VIPER appear to be an easy and powerful visual tool for global model evaluation.
References:
[1] Karlsson MO and Holford N. A Tutorial on Visual Predictive Checks. PAGE 17 (2008) Abstr 1434 [www.page-meeting.org/?abstract=1434].
[2] Karlsson MO and Savic RM. Diagnosing model diagnostics. Clin Pharmacol Ther 2007; 82:17-20.
[3] Bergstrand M et al. Prediction Corrected Visual Predictive Checks. ACoP (2009) Abstr F7. [http://www.go-acop.org/sites/all/assets/webform/Poster_ACoP_VPC_091002_two_page.pdf].
Acknowledgment: Dalia Khachman was supported by a fellowship from the Lebanese National Council for Scientific Research (Beirut, Lebanon).
