Impact of covariate model selection methods on covariate effects and their uncertainty in population pharmacokinetic analysis
Morgane Philipp (1,2), Simon Buatois (3), Sylvie Retout (2,3), France Mentré (1)
(1) Université Paris Cité, INSERM, IAME, UMR 1137, Paris, France, (2) Institut Roche, Boulogne-Billancourt, France (3) Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, Switzerland
Introduction:
Covariate analysis is a key step for drug development as it allows, notably, to adjust the dose in subpopulations of interest. Clinicians typically used forest plots to support their decision; the 95% confidence interval (CI) of the change in parameter value is then expressed as a ratio for given values or categories of the covariates and relatively to a reference value. Consequently, a precise and accurate estimation of covariate effect ratios (CER) and their associated uncertainty is critical.
In pharmacometrics, there are many covariate modeling methods such as the stepwise covariate model (SCM) [1], the most commonly used, and the full model (FM) [2]. Until now, studies comparing different methods have focused mainly on the correctness of the selected covariate model. However, none was conducted on assessing the associated uncertainty.
The goal of this work is to assess and compare the correctness of the reported effects and their uncertainties for SCM and FM approaches.
Methods:
A simulation study inspired from a real case study conducted in 387 hemophilia A patients treated with emicizumab was performed. From the one compartment model with first-order absorption and linear elimination population PK model developed by Retout et al. [3], two simplified models were derived i.e. a base model and a covariate model with 4 covariate-parameter relationships included using a power relationship for continuous covariates.
A total of 200 datasets were simulated with the base model and the covariate model. The number of patients, sampling schema and covariates distribution were based on the real data. SCM and FM were applied to each of the simulated datasets using a covariate set of 12 covariate-parameter relationships including the 4 simulated. SCM was run with p_forward = 0.05 and p_backward = 0.01 and FM with 5 retries using PsN version 5.3.0 and NONMEM version 7.4 with FOCEI method.
CER were computed for both continuous covariates (between the covariate effect value calculated at the 10th or 90th quantile and the one calculated at the median of the observed covariate distribution) and categorical covariates (between the covariate effect value of one category and the one for the reference category).
Relative RMSE (RRMSE) and coverage rates of CER were calculated to evaluate the precision and accuracy of the estimates and their associated uncertainty. Coverage rates were computed as the proportion of the 90% CI around the estimated CER that include the simulated CER.
Results:
Overall, RRMSE of CER were low for the two approaches with values all below 20%. However, SCM gave slightly larger RRMSE in case of covariate effects simulated at a value different from 0 compared to those obtained with FM with e.g. the RRMSE of the age CER computed at the 10th quantile on the apparent clearance equal to 6% and 4% for SCM and FM, respectively. On the other hand, SCM yielded slightly lower RMSE in case of covariate effect simulated at 0 compared to FM with e.g. the RRMSE of the age CER computed at the 10th quantile on the absorption rate constant equal to 7% and 15% for SCM and FM, respectively, when simulating with the base model.
Coverage rates of CER were all between 0.80 and 1. When simulating with both the base and covariate model, FM gave most of the coverage rates in the expected range ([0.85, 0.94] i.e. the 95% Cl for a probability of success of 0.9) except for 23% of them with some slightly lower values. For simulations with both the base and covariate model, when no covariate effects were simulated, SCM consistently yielded coverage rates close to 1 as these covariates were almost always not selected. For simulations with the covariate model, when covariate effects were simulated, SCM provides almost all coverage rates in the expected range except for 5% of them with some slightly lower values.
Conclusion:
The reported uncertainty for SCM and FM approaches were both correct. SCM performed slightly better when no effects were simulated, probably because of its selection process, while FM performed slightly better when some effects were simulated.
These methods deserved to be evaluated in a context of more complex simulated covariate model or sparse data.
Other selection methods such as SCM plus (an improved version of SCM) [4] and full random effects model (FREM) [5] will be investigated.
References:
[1] Ahamadi, M., Largajolli, A., Diderichsen, P.M. et al. Operating characteristics of stepwise covariate selection in pharmacometric modeling. J Pharmacokinet Pharmacodyn 46, 273–285 (2019). https://doi.org/10.1007/s10928-019-09635-6
[2] Xu, X. S., Yuan, M., Zhu, H. et al. Full covariate modelling approach in population pharmacokinetics: understanding the underlying hypothesis tests and implications of multiplicity. Br J Clin Pharmacol 84, 1525– 1534 (2018). https://doi.org/10.1111/bcp.13577
[3] Retout, S., Schmitt, C., Petry, C. et al. Population Pharmacokinetic Analysis and Exploratory Exposure–Bleeding Rate Relationship of Emicizumab in Adult and Pediatric Persons with Hemophilia A. Clin Pharmacokinet 59, 1611–1625 (2020). https://doi.org/10.1007/s40262-020-00904-z
[4] Svensson, R.J., Jonsson, E.N.. Efficient and relevant stepwise covariate model building for pharmacometrics. CPT Pharmacometrics Syst Pharmacol 11, 1210- 1222 (2022). https://doi.org/10.1002/psp4.12838
[5] Yngman, G., Nyberg, H.B., Nyberg, J. et al. An introduction to the full random effects model. CPT Pharmacometrics Syst Pharmacol 11, 149– 160 (2022). https://doi.org/10.1002/psp4.12741
