Data-driven model selection for model-informed precision dosing: a case study with vancomycin
Wisse van Os (1), Amaury O’Jeanson (2), Carla Troisi (3), Chun Liu (3), Jasmine Hughes (4), Dominic Tong (4), Jordan Brooks (4), Ron Keizer (4)
(1) Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria; (2) Department of Pharmacy, Uppsala University, Uppsala, Sweden; (3) Department of Medical and Surgical Sciences, Alma Mater Studiorum-University of Bologna, Bologna, Italy; (4) InsightRX, San Francisco, California, USA
Introduction: Vancomycin is a widely-used antibiotic with a narrow therapeutic range. Elevated levels may lead to nephrotoxicity, while insufficient levels can result in treatment failure and development of antibiotic resistance. Achieving an optimal area under the concentration-time curve (AUC) between 400-600 mg·h/L is crucial for therapeutic success [1]. Model-informed precision dosing (MIPD) combines therapeutic drug monitoring (TDM) with population pharmacokinetic (PopPK) models to guide dosing decisions. There are multiple PopPK models available for vancomycin, and Bayesian approaches to model selection and model averaging have demonstrated accuracy for vancomycin predictions with one or more TDM samples available [2]. However, model selection before the first TDM sample becomes available remains challenging, and may be based on results from external validation studies, prior clinical experience and intuition.
Objectives: Train a machine learning (ML) model to perform PopPK model selection/model averaging for vancomycin based on patient characteristics available before the first TDM sample, to optimize personalized vancomycin dosing.
Methods: De-identified data entered by users of the InsightRX Nova MIPD platform (InsightRX, San Francisco, CA, USA), between 01/01/2020 and 19/09/2023 were retrospectively analyzed. Adult patients with at least one recorded TDM sample and corresponding a priori predictions from six PopPK models were included [3–8]. Data rows with missing or improbable patient characteristics or TDM values were removed. The dataset was partitioned into training, validation, and test sets (70-15-15% split). Various ML models were trained to predict the residual for each of the included PopPK models. Using the tidymodels framework in R, linear regression, penalized linear regression, random forests and xgboost models were investigated. The outcome variable was the absolute value of the residual of the predicted TDM value, normalized by the observed TDM value. Patient characteristics such as age, sex, weight, height, plasma creatinine, and derived values were used as model features (i.e., predictors). For each TDM observation in the validation dataset, the PopPK model with the lowest predicted absolute residual was designated the ML-selected PopPK model. In addition, a model averaging/ensembling approach was performed, with weights of each PopPK model proportional to the inverse of the squared predicted normalized residual. Predictive performance was assessed using various metrics, including relative root mean square errors (rRMSE) for precision, median relative prediction error (rPE) for bias, and percentage of large prediction errors (>5 mg/L or >30%).
Results: The final analysis dataset contained 298,211 TDM observations and corresponding PopPK model a priori predictions. The xgboost regression model emerged as the best-performing ML model for model selection. The most predictive features were plasma creatine, creatinine clearance (Cockcroft-Gault), age, sex, weight, height, body mass index (BMI) and body surface area (BSA). The ML-based model selection approach outperformed single PopPK models in terms of precision (rRMSE of 33.2% vs 33.4 to 43.9%) and demonstrated low bias (rPE of -4.62% vs -24.1 to 17.3%). ML-based model averaging slightly improved overall performance (rRMSE of 33.0% and rPE of -3.70%). ML-based model selection and model averaging also resulted in lower percentages of predicted residuals >5 mg/L (25.5% and 24.7%, respectively, vs 27.6 to 38.3% for single PopPK models) and residuals >30% (34.1% and 34.0%, respectively, vs 36.7 to 49.7% for single PopPK models). Retrospective identification of the true best model for each TDM sample resulted in 21.2% rRMSE, -0.75% rPE, and 9.7% and 10.9% of predicted residuals exceeding 5 mg/L and 30%, respectively.
Conclusions: ML-based model selection and model averaging approaches outperformed individual PopPK models and other approaches such as naive averaging to predict vancomycin concentrations, and appears to be promising in an MIPD setting before TDM samples are available. However, the observed performance gap between the prospectively predicted and retrospectively identified best models across our data set suggests that currently considered individual patient characteristics can only explain a fraction of the variability in vancomycin exposure. This emphasizes the importance of TDM and Bayesian approaches to model selection and averaging.
References:
[1] Rybak MJ, Le J, Lodise TP, Levine DP, Bradley JS, Liu C, et al. Therapeutic monitoring of vancomycin for serious methicillin-resistant Staphylococcus aureus infections: A revised consensus guideline and review by the American Society of Health-System Pharmacists, the Infectious Diseases Society of America, the Pediatric Infectious Diseases Society, and the Society of Infectious Diseases Pharmacists. Am J Health Syst Pharm 2020;77:835–64. https://doi.org/10.1093/ajhp/zxaa036.
[2] Uster DW, Stocker SL, Carland JE, Brett J, Marriott DJE, Day RO, et al. A Model Averaging/Selection Approach Improves the Predictive Performance of Model-Informed Precision Dosing: Vancomycin as a Case Study. Clin Pharmacol Ther 2021;109:175–83. https://doi.org/10.1002/cpt.2065.
[3] Buelga DS, del Mar Fernandez de Gatta M, Herrera EV, Dominguez-Gil A, García MJ. Population pharmacokinetic analysis of vancomycin in patients with hematological malignancies. Antimicrob Agents Chemother 2005;49:4934–41. https://doi.org/10.1128/AAC.49.12.4934-4941.2005.
[4] Carreno JJ, Lomaestro B, Tietjan J, Lodise TP. Pilot Study of a Bayesian Approach To Estimate Vancomycin Exposure in Obese Patients with Limited Pharmacokinetic Sampling. Antimicrob Agents Chemother 2017;61:e02478-16. https://doi.org/10.1128/AAC.02478-16.
[5] Colin PJ, Allegaert K, Thomson AH, Touw DJ, Dolton M, de Hoog M, et al. Vancomycin Pharmacokinetics Throughout Life: Results from a Pooled Population Analysis and Evaluation of Current Dosing Recommendations. Clin Pharmacokinet 2019;58:767–80. https://doi.org/10.1007/s40262-018-0727-5.
[6] Goti V, Chaturvedula A, Fossler MJ, Mok S, Jacob JT. Hospitalized Patients With and Without Hemodialysis Have Markedly Different Vancomycin Pharmacokinetics: A Population Pharmacokinetic Model-Based Analysis. Ther Drug Monit 2018;40:212–21. https://doi.org/10.1097/FTD.0000000000000490.
[7] Tong DMH, Hughes JH, Keizer RJ. Use of Age-Adjusted Serum Creatinine in a Vancomycin Pharmacokinetic Model Decreases Predictive Performance in Elderly Patients. Ther Drug Monit 2021;43:139–40. https://doi.org/10.1097/FTD.0000000000000819.
[8] Thomson AH, Staatz CE, Tobin CM, Gall M, Lovering AM. Development and evaluation of vancomycin dosage guidelines designed to achieve new target concentrations. J Antimicrob Chemother 2009;63:1050–7. https://doi.org/10.1093/jac/dkp085.
