Are population PK/PD models adequately evaluated? An exhaustive survey of the literature between 2002 and 2004
Brendel, Karl(1,3), Céline Dartois(2,3), Emmanuelle Comets(1), Christian Laveille(4), Annabelle Lemmenuel-Diot(3), Brigitte Tranchand(2,6), Pascal Girard(2,5), Céline Laffont(3), France Mentré(1)
(1)INSERM U738, Université Paris 7, AP-HP, CHU Bichat-Claude Bernard, Paris, France ; (2)EA 3738; Medecine Faculty of Lyon Sud, Oullins, France; (3)Institut de Recherches Internationales Servier, Courbevoie, France; (4 )Exprimo NV, Lummen, Belgium; (5)INSERM, Lyon, France; (6)Centre Léon Bérard, Lyon, France
Purpose: Model evaluation is an important issue in population PK and/or PD analyses. The objective of the present study was to perform a systematic review of all population PK/PD analyses published between 2002 and 2004 in order to have an overview of the current methods used to evaluate a model and to assess whether those models were adequately evaluated.
Methods: We selected 324 papers in Pubmed using defined keywords and built a data abstraction form (DAF) composed of a checklist of items to extract the relevant information from these articles with respect to model evaluation. In the DAF, evaluation methods were divided into three subsections: basic internal methods (goodness-of-fit plots (GOF), uncertainty on parameter estimates and model sensitivity), advanced internal methods (data splitting, resampling techniques and Monte Carlo simulations) and external model evaluation.
Results: Basic internal evaluation was the most frequently used method: 65 % of the models were evaluated using GOF. Standard errors or confidence intervals were reported in 50 % of models for fixed effects but were only reported in 22 % of models for random effects. Advanced internal methods were used in approximately 25 % of models: data splitting was more often used than bootstrap and cross-validation; simulations were used in 6 % of models to evaluate models by visual predictive check or by posterior predictive check. External evaluation was performed in only 7 % of models.
Conclusions: Using the subjective synthesis on model evaluation quoted for each paper, models were judged to be adequately evaluated in 28 % of cases for PK models and 26 % of cases for PD models. From this review, it appears that basic internal evaluation was preferred to more advanced methods probably because they are often available easily with most software. However, if the aim of modelling is predictive, advanced internal methods or more stringent external methods are highly recommended.