Data-Driven Discovery of Interpretable Feedback Mechanisms in Acute Myeloid Leukaemia using DeepPumas
Carl Julius Martensen (1,2) , Niklas Korsbo (2) , Sebastian Sager (1), Vijay Ivaturi (2)
(1) Otto von Guericke University Magdeburg, (2) Pumas-AI Inc.
In pharmacometrics, model derivation and selection are critical for quantitatively analyzing drug-biological interactions, elucidating pharmacokinetic/-dynamic (PKPD) relationships, and optimizing dosing regimens. Through systematic evaluation and validation, these models become essential tools for predictive analytics and informed decision-making in drug development and clinical practice. It draws on decades of expertise and is arduous and time-consuming. Scientific machine learning offers an alternative approach to classical methods, combining first principles and data-driven components [1, 2]. As an extension to classical system identification, optimization-based methods allow the distillation of equations from data, transforming data into interpretable models in a semi-automated fashion [3, 4, 5, 6]. Nonlinear mixed-effect modeling has not embraced the methodology that has been successful in other domains. We aim to bridge this research gap by employing neural networks to efficiently learn unknown submodels from heterogeneous real-world data sources. We develop and apply a method to recover a symbolic expression of the extracted signal. To our knowledge, this is the first attempt to fully recover a symbolic expression within this domain of application.
Objectives: The objective is to employ neural networks for inferring latent submodels within nonlinear mixed-effect (DeepNLME) frameworks using actual data. Concurrently, the project seeks to devise a method for translating the derived signal into a symbolic representation. The focus is on replacing the feedback mechanism in the established Friberg [7] model with empirical data from a study on acute myeloid leukemia treatment using intermediate to high-dose cytarabine [8].
Methods: We are examining the performance of models for acute myeloid leukemia using the publicly accessible dataset presented in [8]. The dataset includes 23 patients who received induction therapy and achieved complete remission. The treatment plans involved one to three consecutive cycles with Ara-C doses ranging from 1 to 3 g/m2. Following model selection to obtain a valid PK model (from data initially reported in [9] using naive pooled), different versions of the Friberg model with one, two, and three transition compartments were successively applied in Pumas [10] utilizing first-order conditional estimation (FOCE). In all models, the feedback term was replaced with neural networks that use two normally distributed random effects and white blood cell count as inputs. The hybrid model was trained similarly to the baseline using DeepPumas TM, (PumasAI, Delaware US). We performed symbolic regression [11] on the typical value-based outputs of the network, assuming a consistent structural form of the unknown equation with parameter variations.
Results: Our research indicates that the DeepNLME model performed similarly to the baseline model in terms of accuracy, both on the training and test sets. We were able to extract both the original feedback term and a data-driven alternative comparable to the baseline but preferable in terms of numerical fitting. Specifically, we propose using a saturating function similar to, but not identical to, a classical Hill equation instead of exponential inverse feedback to describe the effect of mature white blood cells on proliferation. The final log-likelihood of the best models in all transition compartments is (−696.4, −252.1) for the baseline, (−693.3, −219.4) for the DeepNLME, and (−664.8, −213.9) for the symbolic identification (train and test set, respectively).
Conclusions: In conclusion, we demonstrate that scientific machine learning techniques can extend traditional statistical modeling approaches and automate model discovery, with DeepPumas being the first product to successfully implement this. We successfully trained DeepNLME models that perform as well as their mechanistic counterparts. Even in the absence of a baseline model, the neural network surrogate can provide an initial data-driven hypothesis for modeling. Furthermore, we symbolically recovered not only the original baseline model, thereby underlining the validity of our approach, but also an alternative with equally good performance. This study is a promising start for data-driven modeling applicable to PKPD systems and general statistical models, enabling the algorithmic generation of candidate models for unknown mechanisms.
References:
[1] M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D. Spergel, and S. Ho, “La-
grangian Neural Networks,” July 2020. arXiv:2003.04630 [physics, stat].
[2] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural net-
works: A deep learning framework for solving forward and inverse problems
involving nonlinear partial differential equations,” Journal of Computational
Physics, vol. 378, pp. 686–707, Feb. 2019.
[3] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations
from data by sparse identification of nonlinear dynamical systems,” Proceedings of
the National Academy of Sciences, vol. 113, pp. 3932–3937, Apr. 2016. Publisher:
Proceedings of the National Academy of Sciences.
[4] N. M. Mangan, S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Inferring Biological
Networks by Sparse Identification of Nonlinear Dynamics,” IEEE Transactions
on Molecular, Biological and Multi-Scale Communications, vol. 2, pp. 52–63, June
2016. Conference Name: IEEE Transactions on Molecular, Biological and Multi-
Scale Communications.
[5] M. Cranmer, A. Sanchez Gonzalez, P. Battaglia, R. Xu, K. Cranmer, D. Spergel,
and S. Ho, “Discovering Symbolic Models from Deep Learning with Inductive Bi-
ases,” in Advances in Neural Information Processing Systems, vol. 33, pp. 17429–
17442, Curran Associates, Inc., 2020.
[6] C. Rackauckas, Y. Ma, J. Martensen, C. Warner, K. Zubov, R. Supekar, D. Skin-
ner, A. Ramadhan, and A. Edelman, “Universal Differential Equations for Scien-
tific Machine Learning,” Nov. 2021. arXiv:2001.04385 [cs, math, q-bio, stat].
[7] L. E. Friberg, A. Henningsson, H. Maas, L. Nguyen, and M. O. Karlsson, “Model
of Chemotherapy-Induced Myelosuppression With Parameter Consistency Across
Drugs,” Journal of Clinical Oncology, vol. 20, pp. 4713–4721, Dec. 2002. Pub-
lisher: Wolters Kluwer
[8] F. Jost, E. Schalk, K. Rinke, T. Fischer, and S. Sager, “Mathematical models for
cytarabine-derived myelosuppression in acute myeloid leukaemia,” PLOS ONE,
vol. 14, p. e0204540, July 2019. Publisher: Public Library of Science.
[9] W. Kern, E. Schleyer, M. Unterhalt, B. W ̈ormann, T. B ̈uchner, and W. Hidde-
mann, “High antileukemic activity of sequential high dose cytosine arabinoside
and mitoxantrone in patients with refractory acute leukemias,” Cancer, vol. 79,
no. 1, pp. 59–68, 1997.
[10] C. Rackauckas, Y. Ma, A. Noack, V. Dixit, P. K. Mogensen, S. Byrne, S. Mad-
dhashiya, J. B. S. Calder ́on, J. Nyberg, J. V. S. Gobburu, and V. Iva-
turi, “Accelerated Predictive Healthcare Analytics with Pumas, a High Perfor-
mance Pharmaceutical Modeling and Simulation Platform,” Nov. 2020. Pages:
2020.11.28.402297 Section: New Results.
[11] M. Cranmer, “Interpretable Machine Learning for Science with PySR and Sym-
bolicRegression.jl,” May 2023.
