Supporting decision making and early prediction of survival for oncology drug development using a pharmacometrics-machine learning based model.
Sébastien Benzekry (1), Mélanie Karlsen (1), Abdessamad El Kaoutari (1), René Bruno (2), Ales Neubert (3), François Mercier (4), Martin Stern (5), Bruno Gomes (6), Suresh Vatakuti (7), Peter Curle (8) and Candice Jamois (9)
(1) COMPO (COMPutational pharmacology and clinical Oncology), Inria Sophia Antipolis – Méditerranée and Center for Research on Cancer of Marseille, Inserm U1068, CNRS UMR7258, Institut Paoli-Calmettes, Aix-Marseille University, France; (2) Clinical Pharmacology, Genentech-Roche, Marseille, France; (3) Roche Pharma Research and Early Development Data & Analytics, Roche Innovation Center Basel, Switzerland; (4) Pharma Development Data Sciences, Roche Innovation Center Basel, Switzerland; (5) Early Clinical Development Oncology, Roche Glycart, Schlieren, Switzerland; (6) Early Biomarker Development Oncology, Roche Pharma Research and Early Development, Roche Innovation Center Basel, Switzerland; (7) Predictive Modeling and Data Analytics, Pharmaceutical Sciences, Roche Pharma Research and Early Development, Roche Innovation Center Basel, Switzerland; (8) Safety and Early Development Informatics, Roche Pharma Research and Early Development, Roche Innovation Center Basel, Switzerland; (9) Translational PKPD and Clinical Pharmacology, Pharmaceutical Sciences, Roche Pharma Research and Early Development, Roche Innovation Center Basel, Switzerland
Introduction: Despite some recent advances in cancer treatment, the attrition rate in late phase studies remains high. Predicting response to therapy using readouts from early studies where data are often limited and patients' diseases are very advanced remains a challenge. However, host fitness and tumor biology biomarkers could be good prognostic indicators of survival [1, 2]. Early on-treatment changes could improve survival prediction compared to baseline factors only. Nonlinear mixed effect (NLME) models are useful to leverage longitudinal data and can help identify trends in observed data. Nevertheless, these models can be limited for data mining purposes and machine learning (ML) could help.
Objectives: (1) Integration of a large database composed of baseline patients’ and disease characteristics, longitudinal lab parameters and tumor size data, genomic and transcriptomic data from patients (pts) with advanced non-small cell lung cancer treated with atezolizumab (ATZ); (2) Development of a NLME-ML based model for early prediction of survival; (3) Study risk prediction
Methods: The model was trained on data from 3 ATZ phase 2 trials (862 pts), and validated on OAK phase 3 trial (553 pts) [3-6].
Model development: Training data consisted of baseline clinical variables (p=73), transcriptomic and mutational data (p=58,311 transcripts and 395 genes), longitudinal data for tumor size (TK, 5,570 data points) and four pharmacodynamic (PD) markers: neutrophils (ANC), C-reactive protein (CRP), lactate dehydrogenase (LDH) and albumin (61,296 data points). NLME models were used to describe TK and PD parameters time courses [7-10]. Individual empirical bayes estimates (EBEs) from each model parameters, clinical factors, transcriptomic and mutational data were used as features for the survival ML models. Dimensionality reduction and features selection methods were applied to identify a “minimal signature model”. Several algorithms of survival prediction and classification (survival at 12-Months landmark) were tested using nested cross-validation [11-12]. Various performance metrics were evaluated (e.g., C-index, area under the curve (AUC), sensitivity (SE), specificity (SP), positive and negative predictive values (PPV, NPV), survival and calibration curves).
Model validation: The model was applied to pts' data from the ATZ (N=396) and docetaxel (DTX) (N=354) control arm in OAK. Individual EBEs from each model parameters identified using Bayesian estimation were used as features. Data were truncated using the time from randomization of the first patient (5 - 100 weeks) to mimic real on-study conditions. Predicted survival hazard ratio (HR) was compared to observed survival: for each patient and both arms (ATZ and DTX), the ML model was used to generate 100 replicates of predicted survival curves for each pt and subsequently 100 study-level survival curves. Mean (95% prediction interval (PI)) was compared to observed HR (and 95% confidence interval (CI)).
Results: The best NLME and ML models were double-exponential (TK, ANC, CRP, LDH), hyperbolic (albumin) models and Random Survival Forest (RSF) algorithm. The minimal signature model was composed of 26 features: 11 routine clinical variables, 3 TK and 12 PD model derived parameters. It exhibits good predictive power (C-index = 0.818 ± 0.029, AUC = 0.905 ± 0.0414) that was significantly improved when PD model metrics were added to baseline variables or TK parameters. RNAseq data had low predictive power. Model simulations were able to reproduce retrospectively the survival curves of ATZ and DTX arms from OAK, despite the different mode of action of each drug (anti PD-L1 checkpoint inhibitor and taxoid antineoplastic agent) and accurately predicted the study outcome with a predicted HR (95% PI) of 0.765 (0.692 - 0.829) versus observed HR (95% CI) of 0.765 (0.64 - 0.913). Applied to partial data, the model was able to predict ATZ survival benefit over DTX already at 8 months (30 weeks) post randomization and predicted HR (95% PI): 0.802 (0.655 - 0.907) versus observed HR (95% CI): 1.04 (0.386 - 2.79). ATZ’s survival benefit over DTX was seen in the data from 21 months onwards.
Conclusion: This NLME-ML based model could inform the decision to move an asset to a later phase of development through the analysis of early data. Its ability to predict survival in both treatment arms from OAK holds the premise for a potential extrapolation to other drugs within the same disease setting.
References:
[1] Shukuya, T. & Carbone, D. P. Predictive Markers for the Efficacy of anti–PD-1/PD-L1 Antibodies in Lung Cancer. Journal of Thoracic Oncology 11, 976–988 (2016).
[2] Becker, T. et al. An enhanced prognostic score for overall survival of patients with cancer derived from a large real-world cohort. Ann Oncol 31, 1561–1568 (2020).
[3] Fehrenbacher L et al. Atezolizumab versus docetaxel for patients with previously treated non-small-cell lung cancer (POPLAR): a multicentre, open-label, phase 2 randomised controlled trial. Lancet (2016)
[4] Solange Peters et al. Phase II Trial of Atezolizumab As First-Line or Subsequent Therapy for Patients With Programmed Death-Ligand 1–Selected Advanced Non–Small-Cell Lung Cancer (BIRCH). JCO (2017)
[5] Spigel D.R et al. FIR: Efficacy, Safety, and Biomarker Analysis of a Phase II Open-Label Study of Atezolizumab in PD-L1–Selected Patients With NSCLC. Journal of Thoracic Oncology (2018)
[6] Fehrenbacher L et al. Updated Efficacy Analysis Including Secondary Population Results for OAK: A Randomized Phase III Study of Atezolizumab versus Docetaxel in Patients with Previously Treated Advanced Non–Small Cell Lung Cancer. Journal of Thoracic Oncology (2018)
[7] Stein et al. Tumor Growth Rates Derived from Data for Patients in a Clinical Trial Correlate Strongly with Patient Survival: A Novel Strategy for Evaluation of Clinical Trial Data. Oncologist, 2008 Oct;13(10):1046-54.
[8] Claret, L. et al. A Model of Overall Survival Predicts Treatment Outcomes with Atezolizumab versus Chemotherapy in Non-Small Cell Lung Cancer Based on Early Tumor Kinetics. Clin Cancer Res 24, 3292–3298 (2018).
[9] Gavrilov S. et al. Longitudinal Tumor Size and Neutrophil-to-Lymphocyte Ratio Are Prognostic Biomarkers for Overall Survival in Patients With Advanced Non-Small Cell Lung Cancer Treated With Durvalumab. CPT PSP. 2021 Jan; 10 (1): 67–74
[10] Claret L. et al. Model-based prediction of outcome of the atezolizumab Phase 3 study OAK in non-small cell lung cancer based on early tumor kinetic data. JCO 2017, 35(15_suppl):e14517-e14517
[11] Commenges D, Jacqmin-Gadda H. - Survival analysis. Dynamical Biostatistical Models. Chapman and Hall/CRC; 2015
[12] Heagerty, P. J., Lumley, T. & Pepe, M. S. Time-Dependent ROC Curves for Censored Survival Data and a Diagnostic Marker. Biometrics 56, 337–344 (2000)
