Standardized Visual Predictive Check – How and When to used it in Model Evaluation
Diane D. Wang, Shuzhong Zhang
Pfizer Inc
Objectives: PK/PD modeling is increasingly used in drug development, and model evaluation has become an important component in the modeling process to confirm model adequacy. The objectives of the present study were to examine the performance of Visual Predictive Check (VPC), a commonly-used model evaluation method, in evaluating PK/PD models with different properties, to illustrate the situations where VPC may not be applicable, and to propose Standardized Visual Predictive Check (SVPC) for evaluating PK/PD models for which VPC is inadequate.
Methods: The PK/PD datasets ("observed" data) were simulated with pre-defined models under two common scenarios. One scenario is a nonlinear PK/PD model with subjects receiving different doses of drug (e.g. body weight-based dosing). The other is a PK/PD model with covariate effects (e.g. age as a covariate for clearance). These datasets were then fitted with the corresponding true models (pre-defined models) and the alternative (false) models. VPC and SVPC were conducted for both true and false models, respectively, and their performances were compared. SVPC was performed by plotting against time the percentile of each observation in the dataset in relation to its 1000 simulated observations derived from the true and false models.
Results: In both of the aforementioned situations, VPC failed to distinguish between the true and the alternative models. Specifically, VPC suggested model inadequacy for the true model in case of nonlinear PK/PD with different doses, and it indicated model adequacy for the false model in case of PK/PD with covariate effects. However, SVPC distinguished correctly the true and false models in both scenarios.
Conclusions: Although being used more and more frequently, VPC may be inadequate for model evaluation in many cases, such as nonlinear PK/PD models with different doses and PK/PD models with covariate effects (unless the covariates can be conveniently stratified). VPC is only applicable in situations where dose normalized PK/PD profiles for all subjects are expected to be the same when Ωs and σs are all set to 0. However, SVPC can be used in all situations.
