Explainable machine learning for time-to-event analysis: exploring time-dependent covariate effects on models of small cell lung cancer
Luca Marzano(1), Adam Darwich(1), Asaf Dan(2), Salomon Tendler(2), Jayanth Raghotama(1), Rolf Lewhenson(2), Luigi De Petris(2), Sebastiaan Meijer(1)
(1) Division of Health Informatics and Logistics, School of Engineering Sciences in Chemistry, Biotechnology and Health (CBH), KTH Royal Institute of Technology, Huddinge, Sweden, (2) Dept. of Oncology-Pathology, Karolinska Institutet and the lung oncology center, Karolinska University hospital, Stockholm, Sweden.
Objectives: Traditional time-to-event analysis relies on parametric or semi-parametric models, such as the Cox proportional hazards model, to estimate the effects of covariates on survival outcomes [1]. These models have limitations in handling complex relationships and interactions between covariates.
In recent years, machine learning algorithms for survival analysis have been developed as an alternative to these traditional models [2]. However, these algorithms, although powerful, are often viewed as black box models that lack interpretability, which may limit their practical applicability.
There has been a growing interest in explaining and interpreting machine learning models, which has led to the emergence of the field of explainable artificial intelligence (XAI) [3]. However, despite the potential benefits of XAI for verifying survival models, few studies have focused on how these methods could be of benefit to black box models based on real-world data. Furthermore, the proposed post-hoc explanation methods lack time-dimension in the final explanation [4].
In this study, we explored the use of novel time-dependent XAI to analyze the effects of time-dependent covariates on small cell lung cancer data. The entire data-based analysis process was dissected by observing the effects of the time dimension at each step, from model selection to the impact of covaries on the individual patient.
Methods: The analyzed data included real-world patients treated at Karolinska University Hospital [5] (RWD) and three Phase III clinical trial studies [6] (RCT) treated with platinum-doublet chemotherapy. The real-world data covariates included were age, sex, TNM staging, performance status, lab values, brain metastasis, and concomitant radiotherapy. The aggregated dataset formed by RWD and RCT patients included age, sex, performance status, brain metastasis, protocol violations, and data typology.
Several machine learning models were trained for time-to-event analysis: Cox regression, boosted generalized additive cox, boosted tree, bag tree, gradient boosting with regression trees, extremely randomized trees, and random survival forest (traditional, accelerated oblique, and conditional) [2]. The performance of the models was evaluated using C-index[7], and the time variation of Brier Score [8] and Cumulative/Dynamic AUC [9].
Global XAI techniques (temporal feature importance and partial dependence [10]) were used to explore the models overall covariate impact on the model predictions. Local XAI techniques (ceteris-paribus [10] and SurvShap(t) [4]) were used to investigate individual predictions of the patients.
The models were firstly trained only on the RWD. Then, the algorithms were tested on an aggregated cohort of the RWD and RCTs data.
Results: Extreme randomized trees, random survival forest, accelerated oblique random survival forest provided the best performances. The time-based profile of the performances pointed out the efficiency of the accelerated oblique random forest predictions for long out-of-distribution observations of the real-world data.
Global XAI showed the time range of the reliability of the model, and when trend inversions regarding treatment decision variables and how covariates interact over the time.
Local XAI allowed to assess covariate impact at individual patient level, thus testing impact of covariates between long survivals and the comparison between real-world and clinical trials.
Conclusions: Our results demonstrate the potential of this approach for informing model verification, interpretability of time-to-event predictions, thus providing insights into the mechanisms driving model predictions and demonstrate the utility of this approach in clinical research.
References: [1] de Neve, J., & Gerds, T. A. (2020). On the interpretation of the hazard ratio in Cox regression. Biometrical Journal, 62(3), 742–750. https://doi.org/10.1002/bimj.201800255 [2] Wang, P., Li, Y., & Reddy, C. K. (2019). Machine Learning for Survival Analysis. ACM Computing Surveys, 51(6), 1–36. https://doi.org/10.1145/3214306 [3] Barredo Arrieta, A., Díaz-Rodríguez, N., del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/10.1016/J.INFFUS.2019.12.012 [4] Krzyzinski, M., Spytek, M., Baniecki, H., & Biecek, P. (2023). SurvSHAP(t): Time-dependent explanations of machine learning survival models. Knowledge-Based Systems, 262, 110234. https://doi.org/10.1016/J.KNOSYS.2022.110234 [5] Marzano, L., Darwich, A. S., Tendler, S., Dan, A., Lewensohn, R., de Petris, L., Raghothama, J., & Meijer, S. (2022). A novel analytical framework for risk stratification of real-world data using machine learning: A small cell lung cancer study. Clinical and Translational Science, 15(10), 2437–2447. https://doi.org/10.1111/cts.13371 [6] Green, A. K., Reeder-Hayes, K. E., Corty, R. W., Basch, E., Milowsky, M. I., Dusetzina, S. B., ... & Wood, W. A. (2015). The project data sphere initiative: accelerating cancer research by sharing data. The oncologist, 20(5), 464-e20. [7] Harrell, F. E. (2001). Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis (Vol. 608). New York: springer. [8] Graf, E., Schmoor, C., Sauerbrei, W., & Schumacher, M. (1999). Assessment and comparison of prognostic classification schemes for survival data. Statistics in medicine, 18(17-18), 2529-2545. [9] Hung, H., & Chiang, C. T. (2010). Optimal composite markers for time-dependent receiver operating characteristic curves with censored survival data. Scandinavian journal of statistics, 37(4), 664-679. [10] Goldstein, Alex, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2015. “Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation.” Journal of Computational and Graphical Statistics 24 (1): 44–65. https://doi.org/10.1080/10618600.2014.907095