A comparative benchmark study of empirical tumor size models in NSCLC clinical data
Anna Mishina (1-2), Kirill Zhudenkov (1-3), Kirill Peskov (1-3)
(1) M&S Decisions FZ-LLC, Dubai, UAE, (2) Research Center of Model-Informed Drug Development, I.M. Sechenov First Moscow State Medical University, Moscow, Russia, (3) Marchuk Institute of Numerical Mathematics of the Russian Academy of Sciences (RAS)
Introduction: Tumor size dynamics models are the key data analytics tools for analysis of the surrogate biomarker data in oncology trials, e.g. these models are key elements in joint1,2 and sequential modeling approaches3 applied for the predictive simulation of time-to-event clinical endpoints such as overall (OS) and progression-free (PFS) survival. In fact the particular model parameters can be as stand-alone surrogate markers4. Despite the widespread utility, the choice of specific model structure often lacks clear justification.
Systematic comparative analysis of the most frequently used empirical SLD models qualified and validated against a representative dataset is essential for understanding the descriptive and predictive power of these models and for identifying the most appropriate framework for future modeling studies.
Objectives: The objectives of this work are: (i) to develop a methodological framework to assess the descriptive and predictive power of longitudinal biomarker models; (ii) to apply this framework to the most commonly used SLD dynamics empirical models and (iii) to conduct a comparative analysis via internal and external validation.
Methods: This study analyzed longitudinal tumor size data from 381 advanced non-small cell lung cancer (NSCLC) patients treated with Erlotinib (NCT00364351), sourced from the Project DataSphere5. Tumor size was measured as the sum of the longest diameters of target lesions (SLD) according to RECIST 1.1 criteria6. Model development followed standard population model methodology8, exploring combinations of residual error and mixed effects models, and applying selection criteria based on parameter identifiability and Akaike Information Criterion (AIC) values. The search for the final model utilized the SCM algorithm, prioritizing models by AIC and identifiability (relative standard error of the parameters < 50%). The repeated cross-validation was performed by random dividing the dataset into training and validation subsets in a 2:1 ratio across 50 scenarios. The comparative analysis of empirical models across 50 scenarios was conducted using nonparametric Wilcoxon signed-rank test with Holm-Bonferroni adjustment, quantifying descriptive power through AIC and mean squared error (MSE) calculated for training subsets. Validation implied simulating SLD profiles based on early treatment data (truncated by 3 months) and comparing predicted to observed outcomes, using MSE as a measure of predictive accuracy and graphical representation by means of visual predictive check.
Results: The following five empirical models for SLD dynamics7 were chosen for the analysis: bi-exponential model (BXM), BXM with additional parameter characterizing sensitive part of tumor, linear-exponential (LXM), exponential-quadratic, and simplified tumor growth inhibition (TGI) models. Out of the five models analyzed, three (BXM, LXM, and simplified TGI) were able to successfully represent a final population model based on the dataset, that is, for these models there was a high reproducibility of the optimal population model (in ≥50% of scenarios) and this model coincided with the full data optimal model. Among these, the TGI model demonstrated superior descriptive ability, showing statistically significant differences in both AIC and MSE values compared to the BXM and LXM (p.adj=7.8e-10 and p.adj=1.5e-09, respectively for AIC, and p.adj=9.3e-10 and p.adj=1.9e-09 for MSE). The LXM ranked second, indicating a clear hierarchy in model performance based on descriptive statistics. This trend persisted in the predictivity/validation analysis, with the TGI model outperforming others in predictive accuracy, as evidenced by MSE calculated on the validation dataset (p.adj=5.8e-08 for both comparisons with the BXM and LXM).
Conclusions: Current analysis revealed significant differences in predictive power between commonly used empirical models of tumor size dynamics. A simplified TGI model with an additional parameter characterizing the emergence of resistance shows better performance than BXM and LXM models.
Interestingly, for the investigated data, the analysis favors a LXM over its BXM counterpart, traditionally preferred in literature, underscoring the criticality of model selection in accurately depicting tumor dynamics.
This research contributes a methodological framework for empirical model optimization, offering a basis for more accurate predictions of tumor dynamics and model component choice for joint and sequential modeling framework.
References:
[1] Zhudenkov K, Gavrilov S, Sofronova A, et al. A workflow for the joint modeling of longitudinal and event data in the development of therapeutics: Tools, statistical methods, and diagnostics. CPT Pharmacomet Syst Pharmacol. 2022;11(4):425-437. doi:10.1002/psp4.12763
[2] Baaz M, Cardilin T, Jirstrand M. Model-based prediction of progression-free survival for combination therapies in oncology. CPT Pharmacomet Syst Pharmacol. 2023;12(9):1227-1237. doi:10.1002/psp4.13003
[3] Gonçalves A, Marchand M, Chan P, Jin JY, Guedj J, Bruno R. Comparison of two-stage and joint TGI-OS modeling using data from six atezolizumab clinical studies in patients with metastatic non-small cell lung cancer. CPT Pharmacomet Syst Pharmacol. 2024;13(1):68-78. doi:10.1002/psp4.13057
[4] Bruno R, Marchand M, Yoshida K, et al. Tumor Dynamic Model-Based Decision Support for Phase Ib/II Combination Studies: A Retrospective Assessment Based on Resampling of the Phase III Study IMpower150. Clin Cancer Res. 2023;29(6):1047-1055. doi:10.1158/1078-0432.CCR-22-2323
[5] https://data.projectdatasphere.org/projectdatasphere/html/home
[6] Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). Eur J Cancer. 2009;45(2):228-247. doi:10.1016/j.ejca.2008.10.026
[7] Yin A, Moes DJAR, van Hasselt JGC, Swen JJ, Guchelaar HJ. A Review of Mathematical Models for Tumor Dynamics and Treatment Resistance Evolution of Solid Tumors. CPT Pharmacomet Syst Pharmacol. 2019;8(10):720-737. doi:10.1002/psp4.12450
[8] Brum L. Guidance for Industry. Clin Pharmacol. Published online 2022.
