Does sample size, sampling strategy, or handling of concentrations below the lower limit of quantification matter when externally evaluating population pharmacokinetic models?
Mehdi El Hassani, Amélie Marsot
Université de Montréal
Objectives: As precision dosing becomes increasingly prevalent in clinical practice, selecting the most appropriate population pharmacokinetic (popPK) model for a given patient population is a critical step. External evaluation (EE) is considered the most rigorous method for assessing a model's predictive performance. However, the lack of guidelines and standardized methodologies for conducting external evaluations, as well as the unknown impact of study design factors such as sample size, sampling strategy, and handling of concentrations below the limit of quantification (BLQ), make it challenging to select models for clinical use. The objective of this study is to evaluate the impact of the number of patients, the number of samples per patient, sampling times, and the way BLQ plasma concentrations are handled on the outcomes of the EE of three popPK models using aminoglycosides as an example.
Methods:
Models to be evaluated : To evaluate the impact of sampling strategy used during model development on EE outcomes, three different popPK models were evaluated : 1) Smit et al. (1) – developed in bariatric surgical patients with rich sampling; 2) Koloskoff et al. (2) developed in cystic fibrosis (CF) patients using Cmax and Cmin TDM data; and 3) Alghanem et al. (3) – developed in CF patients using Cmax, Cmin, and random TDM data.
Data: Two virtual patient populations were simulated using two distinct popPK models of aminoglycosides : 1) virtual patients undergoing bariatric surgery (4) and 2) virtual CF patients (5). The first virtual population was used to evaluate the Smit et al. model and the second was used to evaluate both Koloskoff et al. and Alghanem et al. models . Virtual populations of 12, 30, 100, 500, and 5000 patients were simulated according to four sampling scenarios: rich sampling, a single sample per patient (Cmax or Cmin), or two samples per patient (Cmax and Cmin). Three approaches were used to handle simulated data below the limit of quantification (LLOQ): delete BLQ data (method M1), impute BLQ data as LLOQ/2 (M5 method), and a likelihood-based approach (method M3).
Analyses: Predicted concentrations for the population were compared to corresponding virtual observed concentrations by calculating the relative prediction error. The median prediction error (MDPE) was used to evaluate bias, while the median absolute prediction errors (MADPE) was used to evaluate imprecision.
Results:
For a given sampling strategy, number of patients did not have an important impact on bias and imprecision results, regardless of the evaluated model and method used to handle BLQ data. For example, while evaluating the model developed by Alghanem et al. (rich sampling scenario, method M1), MDPE ranged from -10.7% to -18.8%, and MDAPE ranged from 30.2% to 38.5%.
Increasing the number of samples per patient in the evaluation dataset did not improve the predictive performance of the three evaluated models. For example, while evaluating the model developed by Alghanem et al. (single Cmax sample scenario, method M1), MDPE ranged from -5.1% to 17.3% and MDAPE ranged from 16.1% to 22.2%, values similar to those obtained with the rich sampling scenario.
In general, evaluating a model developed with rich sampling did not result in better predictive performance than the other two models developed with TDM. For example, while evaluating the model developed by Smit et al. (rich sampling scenario, M5 method), MDPE values ranged from -25.9% to 39.1% and MDAPE values ranged from 36.8% to 51.7%. These values were higher than those obtained evaluating the Alghanem et al. model in the same scenario (MDPE ranging from -0.9% to 11.0% and MDAPE ranging from -37.1% to 47.0%).
For a given model, MDPE values for the Cmin only scenario using the M3 method were lower than those obtained with the M1 and M5 methods, which performed similarly. The M3 method did not improve Cmin MDAPE values.
Conclusions:
The results suggest that a large sample size may not be necessary for an EE study. Furthermore, a model developed with rich sampling does not appear to have superior predictive performance to a model developed with TDM in the context of an external evaluation study, suggesting that a TDM-developed model would be more generalizable. The study also showed that the method used to handle BLQs may have an impact on the outcomes of the EE. This study is a first step towards developing guidelines for external evaluation of popPK models for clinical use.
References:
- Smit C, Wasmann RE, Wiezer MJ, van Dongen HPA, Mouton JW, Brüggemann RJM, et al. Tobramycin Clearance Is Best Described by Renal Function Estimates in Obese and Non-obese Individuals: Results of a Prospective Rich Sampling Pharmacokinetic Study. Pharm Res. 2019;36(8):112.
- Koloskoff K, Thirion DJG, Matouk E, Marsot A. New Recommendations of a Height-Based Dosing Regimen of Tobramycin for Cystic Fibrosis in Adults: A Population Pharmacokinetic Analysis. Ther Drug Monit. 2022.
- Alghanem S, Paterson I, Touw DJ, Thomson AH. Influence of multiple courses of therapy on aminoglycoside clearance in adult patients with cystic fibrosis. J Antimicrob Chemother. 2013;68(6):1338-47.
- Smit C, Wasmann RE, Goulooze SC, Hazebroek EJ, Van Dongen EPA, Burgers DMT, et al. A Prospective Clinical Study Characterizing the Influence of Morbid Obesity on the Pharmacokinetics of Gentamicin: Towards Individualized Dosing in Obese Patients. Clin Pharmacokinet. 2019;58(10):1333-43.
- Crass RL, Pai MP. Optimizing Estimated Glomerular Filtration Rate to Support Adult to Pediatric Pharmacokinetic Bridging Studies in Patients with Cystic Fibrosis. Clin Pharmacokinet. 2019;58(10):1323-32.