Model Averaging in viral dynamic models
A. Gonçalves (1), France Mentré (1), Annabelle Lemenuel-Diot (2), J. Guedj (1)
(1) IAME, UMR 1137, INSERM, Université Paris Diderot, Sorbonne Paris Cité Paris, France (2) Roche Pharmaceutical Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel
Objectives: Nonlinear mixed effect models (NLMEM) are now becoming a central tool in viral dynamic models to estimate parameters of viral pathogenesis and identify relevant factors limiting viral replication [1–3]. However data fitting and model building remain challenging due to the fact that i) models often involves poorly identifiable parameters and/or ii) several structural models with different biological assumptions may provide nearly similar fits to the data. To overcome these issues, the most standard way is to fit the data using a set of candidate models and then to retain the model providing the best fit to the data using standard tools of model selection (such as AIC). However, this approach of Model Selection (MS) ignores the uncertainty due to multiple tested models and thus is subject to overoptimistic conclusions [4]. Here we assess the benefit of using of model averaging (MA) to provide better parameter estimates and more robust predictions, an approach that weighs predictions of different candidate models [4][5].
Methods: We evaluated MA by simulations in two different settings, both in the context of an acute viral infection, using parameters estimated during Ebola virus infection [6]. In the first setting, we focused on estimation step and provided confidence intervals of estimated parameters when some model parameters are fixed to arbitrary values. Data were simulated according to a target cell limited model [7] where both the eclipse phase rate and the initial viral load inoculum cannot be identified and were fixed to different plausible values [8]. Parameters and confidence intervals were then estimated and we compared the coverage rate of the estimated parameters, in particular the reproductive ratio number, R0, under MA, MS and the true model used for simulation.
In the second setting we focused on the predictions derived from MS and MA and considered, in addition to the target cell limited model, 4 models describing the putative role of the innate and adaptive immune system in clearing infection. The magnitude of immune response was comparable between models and they provided similar fits to the data. For each trial replicate, we predicted the median AUC under increasing treatment effects (from 10% to 99.9%) for MA, MS and the true model used for simulations. Relative root mean square errors (RRMSE), relative bias (RB) were calculated to compare prediction precision. Finally, the Kullback-Liebler divergence of the median AUC (KLDAUC) were computed to evaluate MA and MS predictive performances. KLDAUC represents the divergence between the true and estimated probability distributions.
Each simulated trial included 30 individuals drawn every 3 days from day 3 to day 18. Under each setting and for each trial replicate, parameters were estimated by maximum likelihood using the SAEM algorithm implemented in Monolix2018R2 and standard errors were obtained by stochastic approximation.
Results: Regarding the first simulation setting, the true model was selected in the different scenario in less than 63% of the cases. This, therefore, led to a poor coverage of R0 was comprised between 0.37 and 0.62. This was corrected using MA, where the coverage rates increased above 0.90 in all cases. In the second setting, MA was associated with a better prediction of the median AUC compared to MS. When simulating with efficacy of 99%, both RRMSE and RB lowered from 52.9% to 36.7% and 12.3% to 6.7% respectively. Using MA, the mean KLDAUC was reduced by 50% compared to MS. Finally, the true model used to simulate the data was not selected up to 49% of the cases leading then to wrong conclusion about the mechanism of the immune response.
Conclusions: This work shows how model selection, by ignoring the model uncertainty, can lead to biased estimates and/or predictions of the median AUC under treatment. Furthermore, this work illustrates that model averaging may be useful in the context of viral dynamic models to take into account the fact that several candidate models can provide equally good fits to the data.
References:
[1] Murray J, Ribeiro R. Special Issue “Mathematical Modeling of Viral Infections.” Viruses. 2018;10:303.
[2] Guedj J, Dahari H, Rong L, Sansone ND, Nettles RE, Cotler SJ, et al. Modeling shows that the NS5A inhibitor daclatasvir has two modes of action and yields a shorter estimate of the hepatitis C virus half-life. Proc Natl Acad Sci. 2013;110:3991–6.
[3] Nowak MA, Bonhoeffer S, Hill AM, Boehme R, Thomas HC, McDade H. Viral dynamics in hepatitis B virus infection. Proc Natl Acad Sci. 1996;93:4398–4402.
[4] Buckland ST, Burnham KP, Augustin NH. Model Selection: An Integral Part of Inference. Biometrics. 1997;53:603.
[5] Buatois S, Ueckert S, Frey N, Retout S, Mentré F. Comparison of Model Averaging and Model Selection in Dose Finding Trials Analyzed by Nonlinear Mixed Effect Models. AAPS J. 2018;20:56.
[6] Madelain V, Baize S, Jacquot F, Reynard S, Fizet A, Barron S, et al. Ebola viral dynamics in nonhuman primates provides insights into virus immuno-pathogenesis and antiviral strategies. Nat Commun. 2018;9.
[7] Phillips AN. Reduction of HIV concentration during acute infection: independence from a specific immune response. Science. 1996;271:497–499.
[8] Best K, Guedj J, Madelain V, de Lamballerie X, Lim S-Y, Osuna CE, et al. Zika plasma viral dynamics in nonhuman primates provides insights into early infection and antiviral strategies. Proc Natl Acad Sci. 2017;114:8847–52.
