Continuous learning in model-informed precision dosing: a case study
Ron Keizer, Jasmine Hughes, Sirj Goswami
InsightRX
Background: Model-informed precision dosing (MIPD) has the potential to optimize drug dosing for many narrow-therapeutic window and biomarker-guided drugs.[1-3] However, choosing the optimal model from literature for a new target population is often difficult, especially if no retrospective data is available to evaluate existing models. Naively applying existing models into a new population often introduces significant bias and/or imprecision.[4,5] Developing new models for each new patient population requires the collection of a sufficiently-sized dataset, collected either retrospectively or prospectively, as well as considerable time and effort to develop the model, delaying potential optimal treatment of patients using MIPD by months to years. An alternative approach we have proposed previously is to implement a "continuous learning" (CL) strategy[4,6]. Specifically, this approach entails:
1. Implementing an initial model in the point-of-care (POC) MIPD tool.
2. Using data collected in the tool to update the underlying model parameters, and then implementing the updated model in the POC tool on a (semi-)continuous basis.
The initial model can be taken either from literature or pre-specified and trained on a small initial test dataset representative of the target population.
Objectives: Evaluate the potential improvement in predictive performance of CL applied to MIPD vancomycin dosing
Methods: De-identified patient data (dosing history, time-varying covariate data, TDM sampling for vancomycin in adults) from two large US hospitals (site A and B) collected on the InsightRX platform during routine care were used. Three parametric population PK (popPK) models were selected that previously showed good performance and were built on data from a general adult population[7], a hemato-oncological population[8] and an obese population[9]. Additionally, a new popPK model was defined with a pre-specified model structure based on prior knowledge of vancomycin PK (2 compartment iv linear PK model, eGFR estimated using Cockcroft-Gault affecting clearance using a power function, and allometric scaling of clearance and volume parameters). This pre-specified CL model was then trained on test datasets with varying sizes (n = 50, 100, 250, 500 patients) for each site. The predictive performance of the literature and CL models was evaluated in a holdout dataset of n=346 and n=120 patients for site A and B respectively. Predictive precision was defined as the ability of the tool to predict the next vancomycin trough level for the patient given all data available prior to the collected level. This metric, quantified as the root mean squared error (RMSE), was calculated iteratively over the individual patient data for the second level and onward. Computation was automated using NONMEM FOCEI and the PsN proseval tool.
Results: In site A, CL improved predictive precision by 16–39% compared to the literature models: RMSE for the CL models was 4.4–4.7 mg/L while RMSE for literature models was 5.6–7.2 mg/L. In site B, except for the CL model trained at n=50 patients, CL improved predictive precision by 19–40% (RMSE = 4.2–4.3 mg/L vs 5.4–7.0 mg/L). In site B, the CL model trained on the lowest number of patients (n=50) showed much poorer predictive performance (RMSE 13.0 mg/L) than the literature and other CL models.
Conclusion: Continuous learning can allow for a considerable improvement in the predictive capacity compared to existing models from literature. While application of CL models trained on small datasets may lead to increased error, the sample size necessary to build new models that surpass existing models is attainable in clinical practice. The benefit of training the model on increasingly larger datasets appears limited in this case study, but might be useful to allow further optimization of model structure, identification of additional covariates, or conditioning of the model on smaller, more specific subpopulations. Further studies are ongoing to investigate the feasibility and performance of this approach in other drugs and populations.
References:
[1] Chan D et al. Int J Pharmacokinetics 2017
[2] Gonzalez D et al. Clin Translational Science 2017
[3] Darwich AS et al. Clin Pharmacol & Therapeutics 2017
[4] Keizer RJ et al. CPT-PSP 2018
[5] Bukkems LS et al. Int J Antimicrob Agents 2018
[6] Keizer RJ et al. PAGE 2018
[7] Thomson A et al. J Antimicrob Chemotherap 2009
[8] Buelga DS et al. Antimicrob Agents and Chemotherapy 2005
[9] Carreno JJ et al. Antimicrob Agents and Chemotherapy 2017