Visualising and integrating automated model development within a validated environment
Michael Hackl1, Wolfgang Schwarzenbrunner1, Sylvie Retout2, João A. Abrantes2, Christian Flandorfer1
(1) scinteco gmbh, Austria (2) Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, Switzerland
Objective: The utilization of tools for automated model development results in a challenge to re-think and update common workflows and requirements for a validated system like the improve platform [1]. As part of the ADaMO project [2], we aim to integrate pharmR and PharmPy [3,4], which are tools for automated population pharmacokinetic model development, into the general improve workflow. In order to fit our requirements for development in a validated system we need to preserve long time reproducibility of the modeling runs evolving from automation as well as the version locked environment for the execution of pharmR.
Methods: To isolate the integrations from the development of pharmR/pharmPy, the system should be set up as non-invasive as possible. Thereby additional integration effort is only needed when updates in pharmR or PharmPy affect the structure of result files or file storage while still executing all NONMEM runs in a validated environment. It also enables initial development while pharmR and pharmPy are not released.
The automated model development process should be integrated from base to final model in order to allow the modeler to manually inspect and possibly add action at each iteration or “step” to alternatively create new branches or finish the model development manually.
To aid modelers in inspecting, adapting and reviewing the automated model development it should be visualized in an interactive Data Manipulation Graph. It allows the user to follow the steps and decisions of the model development and gives the possibility to manually explore alternatives at any step.
An agile process following the development of the tools pharmPy and pharmR. The “release early, release often” objective should help to gather user feedback.
Results: The pharmR containers allow the deployment and run of multiple different versions of RStudio/pharmR/PharmPy and R in parallel without local installations and provides access to preconfigured systems that are ready to run with one click.
While default settings are generally defined by administrators, the exact version of the NONMEM installation within the validated improve system, the selection of the job queue type in the grid and the number of cores used can be specified by the user at startup time of the container. The non-invasive injection of NONMEM executions is achieved through command line proxies.
The NONMEM executions are performed, stored and executed in the improve system and the results appear automatically in the shape of an analysis tree where the hierarchical relationships between models are preserved. Automatically generated steps can be easily compared to other runs or used in reports. Branching (i.e. create a model that can be modified from a previously automatically generated model) is possible at any point and there is no possibility for the user to interfere with the run after it has been started.
An interactive Data Manipulation Graph was introduced to visualize, filter and explore the executions. This allows grouping the visualization by different sub-tools used in the automated model development and show hierarchies following the decisions made by the automation system. Result files are read out directly after generation to annotate steps in the graph.
All NONMEM runs of a specific amd sub-tool that are used as a base for follow up tool usage are marked as key step, and the final model produced using the Pharmpy/PharmR is marked as the final model in the analysis tree.
Conclusion: A non-invasive and seamless integration of pharmR and PharmPy in the improve platform, supporting parallel multi-core NONMEM executions, has been successfully achieved, allowing a hierarchical and interactive visualisation and easy extensions of the automatically produced NONMEM runs.
Acknowledgment: This work was supported by F. Hoffmann-La Roche Ltd., Basel, Switzerland. We want to thank the Uppsala University Pharmacometrics group and the testers at Roche, Emilie Schindler, Franziska Schaedeli Stark and Valérie Cosson, for their contribution.
References:
[1] Usage of a validated environment to support modeling and simulation activities. Buchheit et al. ACoP11. 2020.
[2] Development of a tool for fully automatic model development (AMD). Chen et al. PAGE 30 (2022) Abstr 10091 [www.page-meeting.org/?abstract=10091]
[3] ADaMO: End-to-end automation of Pharmacometric modelling in drug development, from dataset building to output generation. Abrantes et al. PAGE 30 (2022) Abstr 10051 [www.page-meeting.org/?abstract=10051]
[4] Pharmpy: a versatile open-source library for pharmacometrics. Nordgren et al. PAGE 30 (2022) Abstr 10096 [www.page-meeting.org/?abstract=10096]