A Semi Parametric Method for the Estimation of End of Treatment Effect
Mohamed Gewily, Yersultan Mirasbekov, Gustaf J. Wellhagen, Mats O. Karlsson
Department of pharmacy, Uppsala University, Uppsala, Sweden
Objectives: End of treatment effect is a commonly used endpoint in Randomized Clinical Trials (RCTs). Several methods are used for end of treatment analysis; non-parametric Mixed Models for Repeated Measures (MMRM) are considered reliable, because of their ability to handle patient dropouts and their unbiased estimates (1). MMRM can have a spectrum of residual error correlation structures depending on the initial assumptions, with the unconstrained being the most flexible. On the other hand, parametric Non-Linear Mixed Effect Models (NLMEM) use less parameters compared to MMRM and can estimate treatment effects more accurately if adequately specified, potentially increasing power in efficacy analysis (2). A method that uses MMRM and NLMEM can markedly decrease the number of subjects required for RCTs if it is unbiased and more precise compared to MMRM alone. In this abstract, we propose a Semi-Parametric-Approach (SPA) that uses an estimator (π) published by Yuan and Yin (3) to assign weights to both model predictions to accurately estimate the treatment effect. We use real placebo data to demonstrate the properties of SPA in terms of prediction accuracy and type1 error, compared to MMRM, NLMEM and NLMEM with Individual Model Averaging, IMA (4).
Methods: We used placebo data from 3 clinical trials: The Movement Disorder Society's Unified Parkinson's Disease Rating Scale (MDS-UPDRS) disease progression (5), the Alzheimer's Disease Assessment Scale-Cognitive subscale (ADAS-Cog) disease progression (6), and the LIKERT pain score from painful distal diabetic neuropathy disease progression (7). To standardize the research frame work, 6 time points were chosen with no dropouts, except for LIKERT, 7. 100 datasets were generated per trial by randomly allocating patients to treatment- placebo arms. Afterwards, different models were fitted to the resulting datasets (model, nb parameters for UPDRS, ADAS-Cog, LIKERT): i) NLMEM (NLMEM 8,8,8), ii) misspecified NLMEM (NLMEM_mis, 7,7,7), iii) mixture model NLMEM (IMA,9,9,9), iv) mixture model NLMEM_mis (IMA_mis,8,8,8), v) MMRM with an unconstrained residual error correlation (MMRM,54,54,70). The trial endpoint was the placebo-adjusted end-of-treatment effect size. The semi parametric estimator (π) was calculated after bootstrapping 600 times per fit (i.e., 600 bootstraps*100 treatment allocations per model per dataset). SPA prediction was computed as: π* model prediction + (1-π) * MMRM prediction. Bias was calculated relative to zero (no effect) and type 1 error was computed for NLMEM, MMRM and SPA with a binomial confidence interval of ± 4.
Results: For all data, treatment effect predictions were largely unbiased for SPA, MMRM, NLMEM, IMA, IMA_mis, but biased for NLMEM_mis. Precision advantage was observed for SPA, NLMEM, IMA_mis and IMA as seen in mean bias ± standard deviation of effect estimates. Type 1 error was largely controlled for all models, except for NLMEM_mis and occasionally for SPA_NLMEM_mis. Model (bias ± SD, MSE, type1 error UPDRS | ADAS-Cog | LIKERT): MMRM (-0.68±3.05,9.7,7%|-0.16±1.51,2.3,4%|-0.04±0.43,0.43,4%), NLMEM (-0.01±0.56,0.31,0%|-0.05±0.68,0.46,12%|-0.22±0.082,0.23,52%) NLMEM_mis (4.2±0.54,18.1,100%|0.12±0.78,0.63,5%|-0.49±0.09,0.5,100%), IMA (-0.04±0.56,0.31,0%|-0.04±0.62,0.39,3%|-0.01±0.12,0.12,4%), IMA_NLMEM_mis (-0.22±1.14,1.35,2%|-0.07±0.74,0.55,3%|-0.01±0.14,0.14,4%), SPA_NLMEM (-0.48±2.27,5.34,2%|-0.09±1.16,1.36,4%|-0.1±0.313,0.32,8%), SPA_NLMEM_mis (0.34±3.07,9.5,14%|-0.014±1.201,1.43,3%|-0.17±0.38,0.419,23%), SPA_IMA (-0.5±2.26,5.33,2%|-0.09±1.16,1.35,2%|-0.03±0.3,0.3,1%), SPA_IMA_mis (-0.55±2.37,5.9,2%|-0.1±1.17,1.37,3%|-0.03±0.3,0.53,0%).
Conclusions: SPA showed more accurate effect size predictions compared to MMRM alone. When NLMEM was highly misspecified, SPA accuracy was comparable to that of MMRM. Indeed, for SPA, MMRM acts as a buffer where the further the NLMEM bootstrapped predictions from the initial MMRM estimate, the less weight assigned by SPA to NLMEM. This is advantageous because MMRM has proven to be robust in the estimation of end of treatment effect (1,8). SPA benefits from NLMEM if a precision advantage is observed, by assigning more weight to NLMEM. Type 1 error for SPA was controlled except when the underlying NLMEM was severely misspecified. A pre-selection method for NLMEM can be beneficial in this context to maximize the benefit from SPA.
References:
- Mallinckrodt CH, Lane PW, Schnell D, Peng Y, Mancuso JP. Recommendations for the Primary Analysis of Continuous Endpoints in Longitudinal Clinical Trials. Drug Inf J. 2008 Jul;42(4):303–19.
- Karlsson KE, Vong C, Bergstrand M, Jonsson EN, Karlsson MO. Comparisons of Analysis Methods for Proof-of-Concept Trials. CPT Pharmacomet Syst Pharmacol. 2013 Jan 16;2:e23.
- Yuan Y, Yin G. Dose-response curve estimation: a semiparametric mixture approach. Biometrics. 2011 Dec;67(4):1543–54.
- Chasseloup E, Tessier A, Karlsson MO. Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments. AAPS J. 2021 May 3;23(3):63.
- Access Data | Parkinson’s Progression Markers Initiative [Internet]. [cited 2022 Apr 5]. Available from: https://www.ppmi-info.org/access-data-specimens/download-data
- Ito K, Corrigan B, Zhao Q, French J, Miller R, Soares H, et al. Disease progression model for cognitive deterioration from Alzheimer’s Disease Neuroimaging Initiative database. Alzheimers Dement J Alzheimers Assoc. 2011 Mar;7(2):151–60.
- Plan EL, Elshoff JP, Stockis A, Sargentini-Maier ML, Karlsson MO. Likert Pain Score Modeling: A Markov Integer Model and an Autoregressive Continuous Model. Clin Pharmacol Ther. 2012;91(5):820–8.
- Wellhagen GJ, Karlsson MO, Kjellsson MC. Comparison of Precision and Accuracy of Five Methods to Analyse Total Score Data. AAPS J. 2020 Dec 17;23(1):9.
