The Visual Predictive Check – Superiority to Standard Diagnostic (Rorschach) Plots
Nick Holford
Dept of Pharmacology & Clinical Pharmacology, University of Auckland, Private Bag 92019, Auckland, New Zealand
Objective: NONMEM users typically demonstrate the adequacy of a model by displaying plots of PRED vs DV or IPRED vs DV along with weighted and unweighted residuals. These are often called the standard diagnostic plots. An alternative way of evaluating a model is to simulate from the final estimates and compare the distribution of the observations with the simulated distribution. A plot of the time course of the observations and prediction interval for the simulated values provides a visual predictive check. The objective of this report is to compare the standard diagnostic plots with the visual predictive check in terms of their ability to suggest improvements to the model structure and confirm the suitability of the final model.
Methods: Plasma warfarin concentrations and effects on prothrombin complex activity have been reported by O’Reilly et al. (1, 2). These data were fitted with a one compartment disposition model with first-order input and elimination plus either an immediate effect model, an effect compartment model or a turnover model. The turnover model is known a priori to be the most appropriate mechanistic model.
Results: The standard diagnostic plots did not give a clear indication of the best model. There was some indication in the residual plot that the immediate effect model was not performing well at the time of the earliest post treatment prothrombin complex observation. The visual predictive check demonstrated the lack of fit of the direct and effect compartment models both for structural and stochastic components.
Conclusion: The standard diagnostic plots should be called Rorschach plots because their interpretation is dependent on the mind of the observer. The visual predictive check is diagnostic of both the fixed and random effects parts of a PKPD model.
References:
[1] O'Reilly RA, Aggeler PM. Studies on coumarin anticoagulant drugs. Initiation of warfarin therapy without a loading dose. Circulation 1968;38:169-177.
[2] O'Reilly RA, Aggeler PM, Leong LS. Studies of the coumarin anticoagulant drugs: The pharmacodynamics of warfarin in man. Journal of Clinical Investigation 1963;42(10):1542-1551.