Handling inter-occasion variability in model implementation for Bayesian forecasting: A comparison of methods and metrics.
Sebastian G. Wicha (1), Stefanie Hennig (2)
(1) Dept. of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Germany, (2) School of Pharmacy, University of Queensland, Brisbane, Australia
Objectives: Inter-occasion variability (IOV) can substantially impact the accuracy and precision in Bayesian forecasting (BF) in the context of therapeutic drug monitoring. A number of approaches exist to handle IOV when utilising a model for BF, which include ignoring IOV, weighting functions, or variations of accounting for IOV during Bayesian estimation. In this study, we aimed to compare five methods for handling IOV using different metrics in simulations and in a real dataset example.
Methods: A 1-compartment population PK model (CL: 5 L/h, V: 20 L, interindividual variabilities (variance) IIVCL: 0.1, IIVV: 0.1, varying IOVCL: 0.0-0.1, proportional unexplained residual variability (RUVprop) 10 %CV) was used for simulations using a rich (8 samples over 8-hourly dosing) and a sparse sampling design (2 samples at 1 h and 7 h post dose) in 1000 subjects. The real dataset arose from 423 patients and a total of 2422 samples developed a 2 compartmental PK model [1]. All simulations, estimations and forecasting was performed in NONMEM® 7.4.1.
Forecasting of occasion 6 PK for every individual using data from occasions 1-5 (simulation study) or of occasion 3 PK from occasions 1-2 data (real data) was assessed.
The methods to handle IOV tested here were:
(i) ‘True’ model with IIV and IOV, quantifying ηIIV and ηIOV’s, but using only ηIIV for forecasting
(ii) IIV + IOV: adding ω²IOV to ω²IIV together
(iii) IIV-only1: re-estimation of a model without IOV, using the new parameters for forecasting
(iv) IIV-only2: setting ω²IOV to zero
(v) IIV-only3: weighting down samples from past occasions by doubling RUV for each past occasion
The metrics to evaluate the forecasting accuracy were:
(a) rBias/rRMSE calculated based on the individual predicted vs. observed concentration at the forecasted dosing occasion, and
(b) rBias/rRMSE calculated based on the estimated individual PK parameter (without ηIOV) versus the true parameter (simulation study) or the individual PK parameter determined from the final published model (real data).
Results: Increasing IOV increased rBias/rRMSE in all metrics. In simulation (IOV of 0.1, rich design) metrics (a) displayed a positive bias in all scenarios with method (v) being least biased (rBias: 44%, rRMSE: 477%), followed by (i) (46.7%, 469%), (iii) (63.1%, 625%), (ii)=(iv) (80.6%, 693%). For metrics (b), individual CL determined by method (i) was least biased (CL: 0.4%, 13%; V: -0,74%, 4.3%), followed by (iii) (CL: 3.2%, 15.1%; V: 18.5%, 23.5%), (iv) (CL: -6.2%, 15.2%; V: 9.9%, 16.8%), (ii) (CL: -6.2%, 15.2%; V: 9.9%, 16.9%), (v) (CL: 10.1%, 23.6%; V: 23.7%, 30.8%). Similar results were obtained with the sparse simulation data.
Using real data, metrics (a) also displayed a positive bias in all scenarios. Method (v) was least biased (17.2%, 146.4%), followed by (i) (20.6%, 154.5%), (iii) (27.5%, 179.7%), (iv) (30.8%, 168.4%) and (ii) (32.4%, 173.5%). For metrics (b), method (i) was least biased (CL: 1.2%, 6.0%), followed by (ii) (CL: -2.2%, 13.3%), (iv) (CL: -2.6%, 10.8%), (v) (CL: 2.8%, 7.9%) and (iii) (CL: 3.2%, 11.7%).
Conclusion: Similar trends in forecasting accuracy were observed in the simulation study and the real dataset, but less marked in the latter. Metrics (a), although popular and frequently used, was intrinsically biased in presence of IOV and hence should be interpreted with caution. Metrics (a) suggested the weighting approach (v) to outperform the true model (i) in the simulation study. Comparisons on the forecasting performance of models on the level of estimated vs. true individual PK parameters, i.e. metrics (b) might be more meaningful, but susceptible to shrinkage. Overall, method (i) displayed the best forecasting performance. Method (iii), where IOV was not estimated may be preferable over the weighting method (v) in presence of IOV.
References:
[1] Llanos-Paez CC, Staatz CE, Lawson R, Hennig S. A Population Pharmacokinetic Model of Gentamicin in Pediatric Oncology Patients To Facilitate Personalized Dosing. Antimicrob Agents Chemother. 2017;61(8).
