Comparison of Pumas and NONMEM in parameter estimation of a complex pharmacodynamic model
Zhengqi Tan (1), Ayyappa Chaturvedula (1), Julie Desrochers (2), Kirsten Riber Bergmann (2), Yassine Kamal Lyauk (2), Rune Viig Overgaard (2)
(1) Pumas-AI, Inc, USA, (2) Novo Nordisk A/S, Denmark
Introduction: Pharmaceutical modeling and simulation (Pumas) software is a pharmacometrics software program developed on Julia language platform (1). Modeling and simulation results generated on Pumas are increasingly being submitted to and accepted by regulatory agencies. As users of Pumas mostly transitioning their prior models from NONMEM (2) based modeling, it is important to match the results for quality control purposes especially in regulatory submissions. Our objective was to compare parameter estimation and results of a complex pharmacodynamic model between NONMEM and Pumas.
Objectives: Compare the results from the FOCEI based estimation implemented in Pumas with NONMEM.
Methods: Pharmacodynamic model included combination of placebo, immediate and slow response defined using Emax and indirect response model components. Average plasma concentration (Cavg) was used as an exposure metric to drive pharmacodynamic effect. Sex was used as a covariate on immediate fraction of response. Placebo model included time effect parameter. Between subject variability (BSV) was estimated on the overall response using exponential error model. Additive error was used for residual variability. Percent change from baseline of the response was used as dependent variable. The dataset resulting from a phase 2 study with several dose levels where response was measured over 52 weeks was used for model fitting. The observed data was randomly sampled to create following scenarios:
- All data (n=544) up to 52 weeks
- All data (n=544) up to 28 weeks
- All data (n=544) up to 34 weeks
- Randomly sampled 60 subjects for 34 weeks
- Randomly sampled 350 subjects for 34 weeks
Percent difference (%diff) was calculated for all parameter estimates using NONMEM estimate as reference value. Relative standard error (%RSE) of estimated parameters, -2LL, and basic diagnostic plots were compared in all instances between Pumas and NONMEM. For Pumas part, data wrangling, modeling, and post processing of results were conducted within Pumas (version 2.2.1) implemented in fully compliant JuliaHub environment. For NONMEM part, data wrangling and post-processing of results were conducted in R (version 4.2.2) and modeling fitting was conducted in NONMEM (version 7.5).
Results: All the estimated thetas, BSV and residual variability were practically identical between Pumas and NONMEM (%diff<0.5%). NONMEM was unable to estimate %RSE for the time effect parameter for the data scenario using data up to 28 weeks, whereas Pumas was able to estimate to be 140%. %RSE for all parameters, -2LL and basic diagnostic plots were identical between NONMEM and Pumas software programs in all scenarios tested.
Conclusion: The parameter estimates, -2LL, precision of parameter estimates, and diagnostics from Pumas software are identical to NONMEM for a complex pharmacodynamic model tested in various scenarios. These results support the use of Pumas for drug development and regulatory purposes.
References:
[1] Rackauckas, Chris, Yingbo Ma, Andreas Noack, Vaibhav Dixit, Patrick Kofod Mogensen, Simon Byrne, Shubham Maddhashiya, et al. "Accelerated Predictive Healthcare Analytics with Pumas, a High Performance Pharmaceutical Modeling and Simulation Platform." BioRxiv, November 30, 2020, 2020.11.28.402297. https://doi.org/10.1101/2020.11.28.402297
[2] Beal SL et al. 1989-2011. NONMEM Users Guides. Icon Development Solutions, Ellicott City, Maryland, USA.