A pharmacometric extension of MCP-MOD in dose finding studies
Simon Buatois (1,2,3), Sebastian Ueckert (4), Nicolas Frey (1),Sylvie Retout (1,2), France Mentré (3)
(1) Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Grenzacherstrasse 124, 4070 Basel, Switzerland (2) INSTITUT ROCHE, 30, cours de l'île Seguin, 92650 Boulogne-Billancourt, France (3) IAME, UMR 1137, INSERM, University Paris Diderot, Sorbonne Paris Cité, Paris, France (4) Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden
Objectives:
Finding the right dose remains a crucial milestone in clinical drug development [1]. According to the ICH-E4 guidance [2]: “dose finding should rely on model-based estimation rather than hypothesis testing via pairwise comparisons”. In this context, the multiple comparison procedures and modeling (MCP-MOD) technique [3,4] received an EMA and FDA qualification opinion as an efficient statistical methodology for model-based design and analysis of Phase II dose finding studies under model uncertainty. Starting from a predefined set of candidate models, MCP-MOD is a two steps approach which, first, establishes the evidence of a drug effect using multiple contrast tests (MCP step) and, then estimates the dose to be brought into the confirmatory phase using a model based approach (MOD step).
In its current implementation, however, the MCP step is limited to the exploration of several mean dose response functions measured at end of trial and, hence, ignores longitudinal information. The consequence is a reduction in power. Pharmacometric analyses (PMX), on the other hand, utilize all the data but generally involve the process of model selection which ignores model structure uncertainty [5–7] and may lead to type I error inflation.
The objective of this work was, therefore, to extend the MCP-MOD methodology and allow for longitudinal nonlinear mixed effects models (NLMEM) for both MCP and MOD step. An additional objective is to evaluate the merits of model selection (MS) and model averaging (MA) for the MOD step through clinical trial simulations (CTS) under various designs and various dose effect models.
Methods:
MCP-MOD extension:
The proposed extension utilizes a predefined set of NLMEM for both MCP and MOD steps. This set is referred to as candidate models.
In the MCP step, the presence of a drug effect is tested using a likelihood ratio test (LRT) between the reference model (i.e.: no dose response relationship) and the best candidate model (MS) according to the Akaike information criterion (AIC) obtained after fitting each model to the data. The critical value for the test is derived through 500 simulations of the same selection procedure under the null-hypothesis with parameter values obtained from the placebo arm of the study (corrected LRT).
In the MOD step, either the candidate model that best describe the data is selected (MS) or a weighted mixture of the candidate models is used (MA).
Evaluation:
CTS were used to evaluate the proposed extension in terms of the following metrics:
- Maintenance of the nominal (5%) type I error in the MCP part per se and in comparison to a non-corrected LRT. For the latter, the distribution of the changes in objective function is summarized for each candidate model and followed by a classical MS. Type one error rate was assumed to be adequate if it was within the 2.5th (3.2%) and 97.5th (7%) percentiles of a binomial distribution with a probability of success of 5% on 500 trial replicates.
- Coverage probability in the MOD step, defined as percentage of trial where the 95% confidence interval contained the true value of interest which could either be a percentiles of the response distribution between the placebo and treatment arms at end-of-treatment, or the minimum effective dose. Coverage was assumed to be adequate if it was within the 2.5th (93%) and 97.5th (96.8%) percentiles of a binomial distribution with a probability of success of 95% on 500 trial replicates.
The evaluation was performed under various simulation scenarios of a hypothetical phase II clinical trial for a monoclonal antibody indicated in the treatment of wet age related macular degeneration (wet-AMD). The duration of the study was set to 12 months with 5 parallel arms (placebo or one of following doses 100, 200, 400 and 1000). Observation times are at baseline, day 7 & every month during 12 months. Finally, two sample size were investigated assuming either 300 (60 per arm) or 50 (10 per arm) patients. Clinical trial simulations were based on a simplified version of a disease model which characterizes the time course of visual acuity (VA) of wet-AMD patients [8] plus one of 5 symptomatic drug effects (no drug-effect, Linear, Log-linear, Emax, and Sigmoid-emax). The resulting 10 scenarios were simulated 500 times.
Implementation:
All simulation and estimation were performed using NONMEM 7.4 with importance sampling estimation algorithm. For each simulated dataset and each candidate model 1000 population parameters were drawn from a multivariate normal distribution where the mean were set to the maximum likelihood estimates and the variance to the variance covariance matrix of the estimates. Using MS, the population parameters were drawn from the best candidate model and using MA, a probability was associated to each one of the candidate models [9]. Finally, 50000 Monte Carlo simulations were used to compute the different values of interest.
Results:
Regarding the MCP part, the type I error rate of the corrected LRT was adequate and equal to 6.2% and 4.6% for the design with 300 and 50 patients, respectively. Using an uncorrected LRT, a substantial increase of the type I error rate was found for both sample size (up to 9.2%).
Regarding the MOD part, the design with 300 patients was associated to adequate coverages for MA and MS. Under the Log-linear and Emax simulation scenarios, the MA method was tending to over-estimate model uncertainty (coverage up to 98.8%) when the MS method was under-estimating the confidence intervals (coverage down to 91.2%). Under the design with 50 patients, MA was leading to better coverage performances than the MS. Coverage performances were mostly adequate under the Linear and Log-linear scenarios and below the lower boundary for the remaining simulation scenarios.
Conclusions:
This work extends the MCP-MOD methodology to use NLMEM in both MCP and MOD step. By deriving the reference distribution of the LRT under the null-hypothesis the method maintains the nominal type-I error while using the full longitudinal information. The work, furthermore, shows how model averaging provides substantially better coverage in the MOD step, and how the ignorance of model uncertainty leads to an under-estimation of the confidence intervals.
References:
[1] J. Cross, H. Lee, A. Westelinck, J. Nelson, C. Grudzinskas, and C. Peck, “Postmarketing drug dosage changes of 499 FDA-approved new molecular entities, 1980–1999,” Pharmacoepidemiol. Drug Saf., vol. 11, no. 6, pp. 439–446, Sep. 2002.
[2] “Dose-response information to support drug registration (ICH Harmonized Tripartite Guideline),” 1994. [Online]. Available: http://www.ich.org/products/guidelines/efficacy/efficacy-single/article/dose-response-information-to-support-drug-registration.html. [Accessed: 14-Jun-2017].
[3] F. Bretz, J. C. Pinheiro, and M. Branson, “Combining multiple comparisons and modeling techniques in dose-response studies,” Biometrics, vol. 61, no. 3, pp. 738–748, Sep. 2005.
[4] J. Pinheiro, B. Bornkamp, E. Glimm, and F. Bretz, “Model-based dose finding under model uncertainty using general parametric models,” Stat. Med., vol. 33, no. 10, pp. 1646–1661, May 2014.
[5] Y. Aoki, D. Röshammar, B. Hamrén, and A. C. Hooker, “Model selection and averaging of nonlinear mixed-effect models for robust phase III dose selection,” J. Pharmacokinet. Pharmacodyn., pp. 1–17, Nov. 2017.
[6] S. Buatois, S. Ueckert, N. Frey, S. Retout, and F. Mentré, “Comparison of model averaging & model selection in dose finding trials analyzed by nonlinear mixed effect models.,” Am. Assoc. Pharm. Sci. J., vol. In press.
[7] Y. Aoki, B. Hamrén, D. Röshammar, and A. C. Hooker, “Averaged Model Based Decision Making for Dose Selection Studies.” [Online]. Available: https://www.page-meeting.org/?abstract=3121. [Accessed: 21-Feb-2018].
[8] C. Diack, D. Schwab, and N. Frey, “An empirical drug-disease model to characterize the effect of Ranibizumab on disease progression in wet AMD patients.” [Online]. Available: http://www.page-meeting.org/?abstract=3569.
[9] S. T. Buckland, K. P. Burnham, and N. H. Augustin, “Model Selection: An Integral Part of Inference,” Biometrics, vol. 53, no. 2, pp. 603–618, 1997.
