Performance evaluation of the full Automatic Model Development (AMD) tool when the true model is known
Zrinka Duvnjak, Franziska Schaedeli Stark, Valérie Cosson, Sylvie Retout, Emilie Schindler, João A. Abrantes
Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, Switzerland
Introduction: The Automatic Model Development (AMD) tool in Pharmpy is an open source tool for automatic population pharmacokinetic (PK) model building [1, 2]. It covers building of all components of common PK models, and using it in a drug development setting can save time for pharmacometricians to address more complex modelling tasks.
Objectives: To assess the performance of the AMD tool using simulated rich population PK datasets in a clinical drug development context.
Methods: A previously published PK model was adjusted to create 10 true models (scenarios) [3]. All true models consisted of 2 distribution compartments (CMT), but differed in absorption complexity, elimination type, complexity of inter-individual variability (IIV) structures, magnitude of proportional RUV and presence/absence of food effect. For each scenario, 30 dataset replicates were simulated, assuming typical phase I study designs: a single ascending dose study (6 cohorts of 6 subjects with ~24 samples) and a multiple ascending dose study (4 cohorts of 9 subjects with ~38 samples). Random noise was added to the nominal sampling times. The AMD tool (modelsearch, IIV, RUV, and for some scenarios COV search subtools) was run with each dataset (default tool settings) and with initial estimates derived by NCA, resulting in 30 final “AMD models” per scenario. The true model was estimated with each simulated dataset (“reference model”). True, reference and AMD models were compared in terms of model structure, primary parameters, population and individual prediction-derived secondary parameters (e.g. Cmax, AUC_0-tau), BIC, RSE and condition number. Models, datasets and results were generated in an automated manner using pharmr (0.86.0), assemblerr (0.1.2) and qpNCA (1.1.6) in R, and NONMEM (7.5.1) in the validated environment Improve [1,2,4-8].
Results: This poster primarily highlights the outcomes of a scenario characterised by slow and complex absorption, and it offers a concise summary of the results for the remaining scenarios. The AMD tool successfully built models for all simulated scenarios. For the scenario highlighted, the true disposition model structure (i.e. 2-CMT model with linear elimination) was always identified by the AMD tool, and the absorption model mostly differed in the number of transit CMT from the true model (1 instead of 3 transit CMT). Diversity in AMD selected IIV structures was large, and replicates with IIV structures different from the true structure (93%) did not differ in individual PK predictions. The true RUV structure was selected for 80% of replicates. The NONMEM covariance step was obtained for more AMD than reference models and the estimated parameters were more precise. The condition number was <1000 for all replicates where it was obtained (28/30). The median BIC difference and range between AMD and reference models was -2 [-8,10].
Conclusions: The AMD tool was able to generate models that describe well the datasets for all scenarios. In the scenario highlighted, the AMD tool selected models that are parsimonious, and have good parameter precision. In all scenarios, even though components of AMD models were sometimes different from the true model, both individual and population predicted PK profiles derived from AMD models were accurate for most of the replicates. This work showcases the usefulness of such a tool in an automated modelling and simulation environment in drug development [8].
Acknowledgments: The authors would like to thank Simon Buatois and Nicolas Frey for their contributions to the development of the AMD tool in collaboration with Uppsala University, the Uppsala Pharmacometrics Research Group, and Scinteco for their assistance during the technical implementation of the framework described in this abstract.
References:
[1] Chen X, et al. Development of a tool for fully automatic model development (AMD). PAGE 30 (2022) Abstr 10091 [www.page-meeting.org/?abstract=10091]
[2] AMD tool [Internet]. https://pharmpy.github.io. [cited 2023 Mar 7]
[3] Cosson V, et al. Population Pharmacokinetic and Exposure-dizziness Modeling for a Metabotropic Glutamate Receptor Subtype 5 Negative Allosteric Modulator in Major Depressive Disorder Patients. Clinical and translational science vol. 11,5 (2018): 523-531. doi:10.1111/cts.12566
[4] Nordgren R, et al. Pharmpy: a versatile open-source library for pharmacometrics. PAGE 30 (2022) Abstr 10096 [www.page-meeting.org/?abstract=10096]
[5] Nordgren R, et al. Pharmpy and assemblerr - Two novel tools to simplify the model building process in NONMEM. PAGE 29 (2021) Abstr 9656 [www.page-meeting.org/?abstract=9656]
[6] assemblerr [Internet]. https://github.com/UUPharmacometrics/assemblerr. [cited 2023 Mar 7]
[7] qpNCA [Internet]. https://github.com/qPharmetra/qpNCA. [cited 2023 Mar 7]
[8] Abrantes J, et. al. ADaMO: End-to-end automation of Pharmacometric modelling in drug development, from dataset building to output generation. PAGE 30 (2022) Abstr 10051 [www.page-meeting.org/?abstract=10051]
