DeepPumas for automatic discovery of individualizable functions governing longitudinal patient outcomes
Niklas Korsbo(1), Mohamed Tarek(1), Chris Elrod(1), Antoine Soubret(2), Francesco Brizzi(2), Christopher Rackauckas(1,3), Jogarao Gobburu(1,4), Vijay Ivaturi(1,4)
(1) Pumas-AI, (2) Roche, (3) Massachusetts Institute of Technology, (4) University of Maryland
Objectives:
The recent advent of Scientific Machine Learning (SciML) that enables us to combine the wealth of knowledge that modelers, biologists, pharmacologists, and clinicians already possess with the pattern recognition ability of machine learning hints at an imminent paradigm shift transforming the way we approach drug development and healthcare delivery. Such hybrid models of mechanistic insight, and data-driven function and pattern identification, have proven both data efficient and good at extrapolating longitudinal predictions beyond the time points used to train the model (1,2,3). However, for this putative paradigm shift to affect the pharmaceutical sciences, the SciML toolbox needs to be extended to handle data from heterogeneous sources such as different patients.
To address this, we have developed 'deep nonlinear mixed effects models' (DeepNLMEs) within the DeepPumas suite of capabilities. DeepNLME aims to enable data-driven model augmentation that can automatically discover individualizable biological interactions that govern patient longitudinal outcomes. Here we highlight the effect of using random effects as inputs to neural networks embedded in NLME models and that this enables data-driven identification of functions that can take state variables, time, or covariates but which parameterizes these functions in a way that maximizes our ability to account for patient outcome heterogeneity.
Methods:
We use a simple class of DeepNLME models, with embedded neural networks that take time and different numbers of random effects, η, ranging from 1 to 5, as inputs to explore how the random effects enable the model to account for various aspects of longitudinal patient outcome heterogeneity. We fit these models towards conditional and marginal likelihood estimates and compare the likelihood of individual predictions based on either full or early observation data from patients excluded from the fitting step.
Results:
We found that the inclusion of random effects as neural network inputs reduced the mean absolute individual prediction error by 81%, 91%, 94%, and 95%, respectively for 1 to 4 random effect inputs, compared to the otherwise identical neural network based model without random effects.
Significantly, though, this monotonic increase in accuracy with the number of random effects does not hold when we make individual predictions based on partial data. We find that simpler models (few random effects) outperform more expressive models (more random effects) when making long-term predictions based on early patient data but that the relative model performance shifts as data becomes more plentiful.
We also show that while the likelihood resulting from models trained with conditional or marginal likelihood maximization are similar when making individual predictions from full time courses, the marginal version has a much better likelihood when used with only early patient data.
Conclusions:
DeepPumas is highly capable of automatically discovering functional relationships between model variables, and it does so in a way that enables the individualization of patient outcome predictions. The level of detail with which the fitted model can predict individual patient outcomes is easily tuned by the number of random effects used in the fitting. However, model expressiveness comes with the potential cost of the model requiring more informative patient-specific data before the model's predictive performance exceeds that of a model with fewer random effects. The ease with which we can tune the model granularity is hugely impactful because it enables us to rapidly tailor model complexity to the level of signal we have available in our data.
Data-driven discovery of individually parameterized functions within NLME models opens up for faster and better identification of, for example, biomarker dynamics where complex combinations of other variables, drug, time, and individual parameters affect their trajectories. It could also help uncover unclear and individual relationships, such as the one between receptor occupancy and reported pain scores where patients might have different pain thresholds.
This ability to discover individualizable functions is a powerful addition to DeepPumas, which also has capabilities for prognostic factor identification and scientific machine learning.
References:
[1] Rackauckas et al. 2020, Universal Differential Equations for Scientific Machine Learning, arXiv
[2] Keith et al. 2021 Learning orbital dynamics of binary black hole systems from gravitational wave measurements, Physical Review Research 3, no. 4:043101
[3] Lemos et al. 2022, Rediscovering orbital mechanics with machine learning, arXiv