Handling data below the quantification limit in viral kinetic modeling for model evaluation and prediction of treatment outcome
Thi Huyen Tram Nguyen (1,2), Jérémie Guedj (1,2), Jing Yu (3), Micha Levi (4), Emmanuelle Comets (1,2), France Mentré (1,2)
(1) IAME, UMR 1137, INSERM, F-75018 Paris, France (2) IAME, UMR 1137, Univ Paris Diderot, Sorbonne Paris Cité, F-75018 Paris, France (3) Novartis Institutes for BioMedical Research, Inc, Cambridge MA 02139, USA (4) Novartis Pharmaceutical Corp., East Hanover NJ 07936, USA
Introduction
Viral kinetic (VK) models have provided many insights into viral infection with various viruses, such as hepatitis C virus (HCV) and human immunodeficiency virus (HIV). They allow evaluating the mechanism of action and effectiveness of antiviral agents [1–3]. The understanding of virologic response determinants can be improved by including pharmacokinetic (PK) information [4]. VK model can also be used to predict individual treatment outcome basing on early responses in order to optimize the duration of anti-HCV treatment [5]. One common problem in VK modeling is data below the quantification limit (BQL), which become more and more frequent with new effective treatments. These data can be handled with several estimation methods [6–8], but how they impact model evaluation and treatment response prediction and how to properly handle them in these steps is still a question. Most of the current diagnostic tools, including prediction discrepancies (pd) and normalized prediction distribution errors (npde), do not take into account BQL data.
Objectives
1) Extend two evaluation metrics, pd and npde, to handle BQL data.
2) Build a VK model using PK data to characterize virologic response to alisporivir (ALV) and pegylated interferon (pegIFN) given alone or in combination then evaluate this PKVK model with the new npde.
3) Evaluate the impact of BQL data, design and a priori information of population parameters on individual parameter estimates and treatment outcome prediction.
Methods
1) Pd/npde belong to the family of simulation-based evaluation metrics. They were developed in our group and implemented in an R package by Comets et al. to evaluate nonlinear mixed effect models [9,10]. Pd are computed as the quantile of observations in their predictive distribution obtained by simulation [10]. Npde are pd calculated from decorrelated observed and simulated data to handle within subjects correlations [11]. We developed a method to handle BQL data. First we evaluated the probability for an observation to be BQL (pBQL) from its predictive distribution. Its pd is then drawn randomly from a uniform distribution [0;pBQL]. To compute npde, BQL data are imputed from their pd drawn previously and the predictive distribution. The same imputation is performed for simulated data. Npde are then computed as usual from imputed datasets. The extended pd/npde were compared with naïve methods that omit BQL data. These methods were evaluated on a bi-exponential model for HIV infection, using graphic evaluation, type I errors and powers of 4 statistical tests, calculated from 1000 simulations under different scenarios [12].
2) ALV is a cyclophilin inhibitor with potent anti-HCV activity [13]. We estimated the effectiveness of ALV and pegIFN in 88 patients infected with HCV genotypes (G) 1-4 treated for 4 weeks with 3 doses of ALV ± pegIFN (phase 2 study DEB-025-203). First, we analyzed the PK of both drugs then used PK predictions as driving functions for the VK model. The PKVK model was evaluated with the new npde. We used this model to predict treatment response (%BQL data and cure rate (SVR)) for three ALV-containing treatment arms of the phase 2 study VITAL-1 [14]. Virus eradication (SVR) was assumed if the infected cells become lower than 10-4.2IU/mL under treatment [15]. BQL data were handled with SAEM in MONOLIX 4.2.2.
3) Individual treatment outcome can be predicted from early VK data, which may contain many BQL data. To study the use of VK data to predict individual response and factors influencing the prediction, we simulated VK data and SVR of 1000 HCV G2/3 patients treated with pegIFN/Ribavirin for 24 weeks, considering 4 nested designs with different sampling time and duration. We estimated individual parameters (infection rate β, infected cells’ loss rate δ, virion clearance c and treatment efficacy ε) with Bayesian method by fixing population parameters at 3 sets of a priori information (true model with simulation parameters; false model Mδε with δ and ε of G1 patients; false model Mβ with a modified β). Data simulated below 45 IU/mL were considered to be BQL [16].
Results
1) In model evaluation, ignoring BQL data resulted in biased and uninformative diagnostic plots, which were much improved with our proposed method. In presence of BQL data, type I errors obtained with naïve approach were much higher than 5%. With new metrics, type I errors were slightly higher than 5% only in scenarios with high interindividual variability but remained much lower than those obtained with naïve approach. Powers to detect model misspecifications with new npde were satisfactory in most cases but decreased when BQL data proportion increased [12]. These extensions as well as many other graphic options were implemented in npde package (version 2.0) for R, released on October 2012 [www.npde.biostat.fr]
2) In the analysis of PKVK data of patients treated by ALV ± pegIFN, a model assuming additive effect of both ALV and pegIFN in blocking viral production characterized viral loads with satisfactory plots of new npde. HCV genotype was found to significantly affect pegIFN effectiveness and infected cells’ loss rate. ALV’s effectiveness was not significantly different across GT and was higher (~90%) at doses ≥600 mg QD. Most of the observed responses in VITAL-1, where ALV was given at doses different from those observed in our study, were included in the 95% prediction intervals provided by the model.
3) In the simulation for Bayesian estimation, precise estimation of individual parameters and treatment outcome was obtained with only 6 data points in the 1st month of treatment. This remained valid even though wrong a priori population parameters were set as long as the parameters were identifiable (δ, ε) and BQL data were properly handled. False a priori information on estimable parameters (δ, ε) could lead to severe estimation/prediction errors if BQL data were omitted. However, taking into account BQL data did not improved estimation/prediction errors if false a priori information for poorly identifiable parameter (β) was used [16].
Conclusions
We proposed a method to extend pd/npde to take into account BQL data. The new metrics show better behaviors than naive methods that omit BQL data in evaluation and were implemented in the package npde 2.0. They were used to evaluate the PKVK model developed to analyze a phase 2 study of ALV where BQL data represented 27.6% of viral loads. The PKVK model was able to describe the observed data and provide reasonable predictions for virologic responses. Bayesian estimation of individual VK parameters can give precise prediction of treatment outcome from only few early responses, provided that BQL data are correctly handled and correct a priori information is available. These results highlighted the possibility of using VK models to evaluate treatment effect, to predict treatment response, to support future clinical trials and to individualize therapy when BQL data are correctly handled.
References
[1] Chatterjee A, Smith PF, Perelson AS. Hepatitis C viral kinetics: the past, present, and future. Clin Liver Dis. 2013 Feb;17(1):13–26.
[2] Perelson AS, Ribeiro RM. Estimating drug efficacy and viral dynamic parameters: HIV and HCV. Stat Med. 2008 Oct 15;27(23):4647–57.
[3] Ding AA, Wu H. Assessing antiviral potency of anti-HIV therapies in vivo by comparing viral decay rates in viral dynamic models. Biostat Oxf Engl. 2001 Mar;2(1):13–29.
[4] Shudo E, Ribeiro RM, Perelson AS. Modeling HCV kinetics under therapy using PK and PD information. Expert Opin Drug Metab Toxicol. 2009 Mar;5(3):321–32.
[5] Brunetto MR, Colombatto P, Bonino F. Bio-mathematical models of viral dynamics to tailor antiviral therapy in chronic viral hepatitis. World J Gastroenterol WJG. 2009 Feb 7;15(5):531–7.
[6] Samson A, Lavielle M, Mentré F. Extension of the SAEM algorithm to left-censored data in nonlinear mixed-effects model: Application to HIV dynamics model. Comput Stat Data Anal. 2006 Dec 1;51(3):1562–74.
[7] Beal SL. Ways to fit a PK model with some data below the quantification limit. J Pharmacokinet Pharmacodyn. 2001 Oct;28(5):481–504.
[8] Bergstrand M, Karlsson MO. Handling data below the limit of quantification in mixed effect models. AAPS J. 2009 Jun;11(2):371–80.
[9] Comets E, Brendel K, Mentré F. Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: the npde add-on package for R. Comput Methods Programs Biomed. 2008 May;90(2):154–66.
[10] Mentré F, Escolano S. Prediction discrepancies for the evaluation of nonlinear mixed-effects models. J Pharmacokinet Pharmacodyn. 2006 Jun;33(3):345–67.
[11] Brendel K, Comets E, Laffont C, Laveille C, Mentré F. Metrics for external model evaluation with an application to the population pharmacokinetics of gliclazide. Pharm Res. 2006 Sep;23(9):2036–49.
[12] Nguyen THT, Comets E, Mentré F. Extension of NPDE for evaluation of nonlinear mixed effect models in presence of data below the quantification limit with applications to HIV dynamic model. J Pharmacokinet Pharmacodyn. 2012 Oct;39(5):499–518.
[13] Gallay P, Lin. Profile of alisporivir and its potential in the treatment of hepatitis C. Drug Des Devel Ther. 2013 Feb;105.
[14] Guedj J, Yu J, Levi M, Li B, Kern S, Naoumov NV, et al. Modeling viral kinetics and treatment outcome during alisporivir interferon-free treatment in HCV genotype 2/3 patients. Hepatol Baltim Md. 2013 Dec 23;
[15] Dixit NM, Layden-Almer JE, Layden TJ, Perelson AS. Modelling how ribavirin improves interferon response rates in hepatitis C virus infection. Nature. 2004 décembre;432(7019):922–4.
[16] Nguyen THT, Guedj J, Yu J, Levi M, Mentré F. Influence of a priori Information, Designs, and Undetectable Data on Individual Parameters Estimation and Prediction of Hepatitis C Treatment Outcome. CPT Pharmacomet Syst Pharmacol. 2013;2:e56.
