Machine learning-driven flattening of model priors: A comparative simulation study across multiple compounds
Marian Klose (1,2), Franziska Thoma (2,3), Lukas Kovar (4), Wilhelm Huisinga (3), Robin Michelet (1), Charlotte Kloft (1)
(1) Dept. of Clinical Pharmacy and Biochemistry, Institute of Pharmacy, Freie Universitaet Berlin, Germany, (2) Graduate Research Training program PharMetrX, (3) University of Potsdam, Institute of Mathematics/Institute of Biochemistry and Biology, Potsdam, Germany, (4) TMCP Data Sciences, Boehringer Ingelheim Pharma GmbH & Co. KG
Introduction
Model-informed precision dosing (MIPD) supports clinical decision-making by leveraging mathematical models and individual drug or biomarker measurements [1]. A typical MIPD workflow involves estimating individual maximum a posteriori (MAP) values for parameters which include interindividual variability (IIV). MAP values represent the mode of the posterior distribution and are obtained by maximising the product of the likelihood of the data and the parameter's prior distribution [2]. The selected prior distribution needs to properly reflect the uncertainty about the parameter and is often defined by a distribution around the covariate-adjusted typical parameter value with a variance given by the model’s IIV estimate. However, this representation of uncertainty is only accurate when the model closely represents the clinical population (CP). In reality, deviations between model and CP are expected [3]. This deviation gave rise to the method of machine learning (ML)-assisted flattening of model priors [4], where the variance of the prior is selectively increased to account for these deviations. While this method has shown considerable improvement in predictive performance for vancomycin [4], its potential benefit across other drugs has not been studied.
Objectives
To assess the impact of ML-driven flattening of priors on the predictive performance for five compounds which are commonly applied in Therapeutic Drug Monitoring (TDM) programs by conducting a simulation study.
Methods
Five compounds were investigated: vancomycin (VAN), meropenem (MEM), methotrexate (MTX), infliximab (IFX), and tacrolimus (TAC). To have a plausible representation of the differences between model and CP to which the model is applied, two published models per drug [5-14] were selected and split into a data-generating model (DM) and an applied model (AM) used for Bayesian forecasting.
A plausible TDM dataset was simulated with the DM (n=5000) using mrgsolve [15]. The PK-Sim® [16] population builder was used to generate demographically realistic virtual patients. All other covariates were drawn from lognormal distributions based on reported point estimates. Two administrations, each followed by a single drug concentration measurement, were simulated for each individual. RUV was included to account for the expected noise in clinical readouts.
MAP estimation using the AM was performed with prior weights λ=1 and λ=1/8 for standard priors (SP) and flattened priors (FP), respectively. While the first concentration was included for MAP estimation, the second was used as a reference to evaluate the predictive performance. For each patient and λ, the residual between prediction and withheld datapoint was calculated.
The labeled dataset was then split into a training/test cohort (75%/25%) to build a ML model which predicts whether FP or SP should be applied for a given individual. All ML steps were conducted using the tidymodels [17] framework in R. The XGBoost (XGB) algorithm [18] was employed, using a 5-fold cross-validation with 5 repeats and a grid search for hyperparameter tuning. Optimal hyperparameters were selected based on precision.
To assess the impact of the method on the predictive performance of the model, the test dataset was bootstrapped (n=1000). For each sample, RMSE and MPE along with their relative improvement compared to SP were calculated across patients.
Results
Results from the simulation study, presented as the median relative improvement (-) or deterioration (+) compared to SP, were:
- VAN
- o RMSE: -16.0% (-11 to -22% [4])
- o MPE: -40.8% (-42 to -74% [4])
- MEM
- o RMSE: -0.12%
- o MPE: -2.89%
- MTX
- o RMSE: +1.06%
- o MPE: -4.15%
- IFX
- o RMSE: -11.8%
- o MPE: +1.69%
- TAC
- o RMSE: -32.6%
- o MPE: +18.0%
Overall, the simulated impact ranged from -40.8% to +1.06% (RMSE) and -16.0% to +18.0% (MPE). Precision for the test dataset was 72.1% (VAN), 25.8% (MEM), 32.8% (MTX), 37.9% (IFX), and 71.3% (TAC).
Conclusion
The predicted improvements in RMSE and MPE for VAN are in agreement with the reported values. However, the impact of ML driven-flattening of priors on the predictive performance for other drugs varied substantially: The largest reduction in RMSE/MPE was indeed found for VAN with heterogenous improvements and even deteriorations for the other compounds. Missing impact of modified priors was also previously reported [19]. Moving forward, we plan to extend the investigation to identify variables which potentially explain the observed differences.
References
[1] F. Kluwe, R. Michelet, A. Mueller‐Schoell et al. Perspectives on Model‐Informed Precision Dosing in the Digital Health Era: Challenges, Opportunities, and Recommendations. Clin Pharmacol Ther 109: 29–36 (2021).
[2] L.B. Sheiner, B. Rosenberg, K.L. Melmon. Modelling of individual pharmacokinetics for computer-aided drug dosage. Computers and Biomedical Research 5: 441–459 (1972).
[3] R.J. Keizer, R. ter Heine, A. Frymoyer et al. Model‐Informed Precision Dosing at the Bedside: Scientific Challenges and Opportunities. CPT Pharmacometrics Syst Pharmacol 7: 785–787 (2018).
[4] J.H. Hughes, R.J. Keizer. A hybrid machine learning/pharmacokinetic approach outperforms maximum a posteriori Bayesian estimation by selectively flattening model priors. CPT Pharmacometrics Syst Pharmacol 10: 1150–1160 (2021).
[5] E. Dreesen, S. Berends, D. Laharie et al. Modelling of the relationship between infliximab exposure, faecal calprotectin and endoscopic remission in patients with Crohn’s disease. Br J Clin Pharmacol 87: 106–118 (2021).
[6] A.A. Fasanmade, O.J. Adedokun, M. Blank et al. Pharmacokinetic Properties of Infliximab in Children and Adults with Crohn’s Disease: A Retrospective Analysis of Data from 2 Phase III Clinical Trials. Clin Ther 33: 946–964 (2011).
[7] A. Boonpeng, S. Jaruratanasirikul, M. Jullangkoon et al. Population Pharmacokinetics/Pharmacodynamics and Clinical Outcomes of Meropenem in Critically Ill Patients. Antimicrob Agents Chemother 66: (2022).
[8] J. Lan, Z. Wu, X. Wang et al. Population Pharmacokinetics Analysis and Dosing Simulations Of Meropenem in Critically Ill Patients with Pulmonary Infection. J Pharm Sci 111: 1833–1842 (2022).
[9] F. Gallais, L. Oberic, S. Faguer et al. Body Surface Area Dosing of High-Dose Methotrexate Should Be Reconsidered, Particularly in Overweight, Adult Patients. Ther Drug Monit 43: 408–415 (2021).
[10] D.W. Faltaos, J.S. Hulot, S. Urien et al. Population pharmacokinetic study of methotrexate in patients with lymphoid malignancy. Cancer Chemother Pharmacol 58: 626–633 (2006).
[11] K. Ogasawara, S.D. Chitnis, R.Y. Gohh et al. Multidrug Resistance-Associated Protein 2 (MRP2/ABCC2) Haplotypes Significantly Affect the Pharmacokinetics of Tacrolimus in Kidney Transplant Recipients. Clin Pharmacokinet 52: 751–762 (2013).
[12] J.-B. Woillard, B.C.M. de Winter, N. Kamar et al. Population pharmacokinetic model and Bayesian estimator for two tacrolimus formulations - twice daily Prograf® and once daily Advagraf®. Br J Clin Pharmacol 71: 391–402 (2011).
[13] V. Goti, A. Chaturvedula, M.J. Fossler et al. Hospitalized Patients With and Without Hemodialysis Have Markedly Different Vancomycin Pharmacokinetics: A Population Pharmacokinetic Model-Based Analysis. Ther Drug Monit 40: 212–221 (2018).
[14] A.H. Thomson, C.E. Staatz, C.M. Tobin et al. Development and evaluation of vancomycin dosage guidelines designed to achieve new target concentrations. Journal of Antimicrobial Chemotherapy 63: 1050–1057 (2009).
[15] Kyle T Baron. mrgsolve: Simulate from ODE-Based Models. R package version 1.4.1, https://github.com/metrumresearchgroup/mrgsolve. (2024).
[16] S. Willmann, J. Lippert, M. Sevestre et al. PK-Sim®: a physiologically based pharmacokinetic ‘whole-body’ model. BIOSILICO 1: 121–124 (2003).
[17] M. Kuhn, H. Wickham. Tidymodels: a collection of packages for modeling and machine learning using tidyverse principles. (2020).
[18] T. Chen, C. Guestrin. XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining doi: 10.1145/2939672.2939785 (2016).
[19] F. Le Louedec, PAGE 31 (2023) Abstr 10296 [www.page- eeting.org/?abstract=10296]. (2023).
