Modelling approaches in dose finding clinical trial: Simulation-based study comparing predictive performances of model averaging and model selection.
Simon Buatois(1,2), Sebastian Ueckert(3), Nicolas Frey(1),Sylvie Retout(1), France Mentré(2)
(1) Roche Pharma Research and Early Development, Clinical Pharmacology, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Grenzacherstrasse 124, 4070 Basel, Switzerland (2) IAME, UMR 1137, INSERM, F-75018 Paris, France (3) Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden
Objectives: In dose finding clinical trials, modeling based approaches require selection of the model (MS) that best describes the data. However, MS ignores model uncertainty which could impair predictive performance [1,2]. To overcome this limit, model averaging (MA) might be used and has recently been applied to nonlinear mixed effect models (NLMEM) [3]. MA allows taking into account the uncertainty across all candidate models by weighting them in function of an information criterion (IC) [1].
The objective of this work is to compare predictive performances of MA and MS based on a predefine set of NLMEMs with a same disease progression model and different dose-effect relationships
Methods: Clinical trial simulations were based on a simplified version of a disease model which characterizes the time course of visual acuity (VA) of age-related macular degeneration patients [4]. The study design was set to 300 patients who were equally randomized in four different arms receiving receptively a placebo or one of the doses of a hypothetical drug. The dose-effect relationship was assumed to follow an emax function. Three scenarios were investigated assuming doses across ED50, doses lower than ED50 or no dose effect. Under each scenario, 500 trial replicates were simulated.
For each trial, parameters of four candidate models (emax, sigmoid emax, log-linear and linear) were estimated using importance sampling in NONMEM7.3 and several IC were investigated to select a model (MS) or compute weights (MA).
The estimation of the minimal effective dose (MED) and the Kullback–Leibler divergence (DKL) between the true and the predicted distributions of the VA change from baseline were used as performance criteria to compare MS and MA.
Results: The overall predictive performance of the MED was better for MA than MS (up to 10% reduction of the root mean squared error). When looking at the entire dose response profile, the mean DKL was reduced (up to 50%) when using MA compared to MS. Finally, regardless of the modelling approaches, AIC outperformed the others IC.
Conclusions: By estimating weights on a predefine set of NLMEMs, MA adequately described the data and showed better predictive performance than MS increasing the likelihood to accurately characterize the optimal dose.
References:
[1] S. T. Buckland, K. P. Burnham, and N. H. Augustin, “Model Selection: An Integral Part of Inference,” Biometrics, vol. 53, no. 2, pp. 603–618, 1997.
[2] K. Schorning, B. Bornkamp, F. Bretz, and H. Dette, “Model selection versus model averaging in dose finding studies,” Stat. Med., vol. 35, no. 22, pp. 4021–4040, Sep. 2016.
[3] Y. A. Aoki, B. Hamrén, D. Röshammar, and A. C. Hooker, “Averaged Model Based Decision Making for Dose Selection Studies.” Available: http://www.page-meeting.org/?abstract=3121.
[4] C. Diack, D. Schwab, and N. Frey, “An empirical drug-disease model to characterize the effect of Ranibizumab on disease progression in wet AMD patients.” Available: http://www.page-meeting.org/?abstract=3569.
