bbr.bayes: An Open-Source Tool to Facilitate an Efficient, Reproducible Bayesian Workflow Using NONMEM
Tim Waterhouse (1), Kyle Meyer (1), Seth Green (1), Curtis Johnston (1), Bill Gillespie (1), Kyle Baron (1), Jonathan French (1,2), Katherine Kay (1), Tim Davis (1), Brian Davis (1), Matthew Riggs (1)
(1) Metrum Research Group, USA, (2) Johnson & Johnson Innovative Medicine, USA
Objectives: With the introduction of Monte Carlo Bayesian methods in NONMEM® 7 [1], Bayesian approaches to modeling have become more accessible to those in the pharmacometrics community who are already familiar with NONMEM®. In addition to accessibility, NONMEM® provides a higher degree of optimization than other Bayesian tools for the types of datasets and model structures often encountered in pharmacometrics settings. These reasons make NONMEM® an attractive option for the pharmacometrician wishing to perform Bayesian analyses. However, constructing and managing control streams for multiple Markov chain Monte Carlo (MCMC) chains and then appropriately processing the full posterior distribution from model output for diagnosing, summarizing, and applying model fits [2] can be challenging. The objective of this work was to support good practice approaches and provide the pharmacometrics community with the R package bbr.bayes, which works in concert with other open-source tools to enable an efficient and reproducible Bayesian workflow with NONMEM®along with a set of illustrative examples to guide its appropriate use.
Methods: Metrum Research Group (MetrumRG) previously developed bbr [3], an R package for managing modeling and simulation projects, and has extended this with a new package, bbr.bayes [4], for accommodating Bayesian analyses. The bbr.bayes package facilitates traceable and reproducible Bayesian workflows in NONMEM® (and Stan [5]) by automating creation and submission of multiple MCMC chains as well as integrating harmoniously with (i) existing MeRGE [6] packages for data handling and reporting, (ii) mrgsolve [7] for generating simulation-based diagnostics, and (iii) packages from the Bayesian modeling community such as posterior [8] and bayesplot [9] for efficient handling of outputs like posterior draws and generating MCMC diagnostic plots. We have assembled example code and accompanying documentation for typical tasks in a NONMEM® Bayesian workflow to illustrate the functionality of bbr and bbr.bayes working in concert with these other packages. While these tasks overlap with many of those considered for a typical analysis using maximum likelihood estimation [10], these Bayesian-specific examples focus on the use of the full Bayesian posterior in downstream activities such as construction of model diagnostics, MCMC diagnostics, parameter tables, and forest plots.
Results: The bbr.bayes package reduces much of the friction associated with a Bayesian pharmacometrics analysis in NONMEM® and promotes good practice applications. In addition to managing the multiple MCMC chains required for such an analysis in a traceable and reproducible manner, the package provides functionality for generating simulation-based diagnostic items using the full posterior including individual and population predictions, normalized prediction distribution errors, and expected weighted residuals. A complete, reproducible example of a NONMEM® Bayes workflow is hosted in a publicly-available, version-controlled repository on GitHub encompassing multiple states and stages of a modeling and simulation project. Similarly, source code for bbr.bayes is hosted in a public GitHub repository. In addition to the scripted example, vignettes and user guides provide step-by-step directions detailing how bbr.bayes and other R packages facilitate key parts of the modeling and simulation analysis workflow by utilizing the full Bayesian posterior [11].
Conclusions: MetrumRG developed the open-source R package bbr.bayes to support traceable, reproducible, and scalable Bayesian pharmacometrics analyses in NONMEM®. Examples of how to use these tools in conjunction with best practice recommendations are provided to the pharmacometrics community.
References:
[1] SL Beal, LB Sheiner, AJ Boeckmann, RJ Bauer, eds. NONMEM 7.5 Users Guides. ICON plc; 1989–2020.
[2] Johnston CK, Waterhouse T, Wiens M, Mondick J, French J, Gillespie WR. Bayesian estimation in NONMEM. CPT Pharmacometrics Syst Pharmacol. 2024; 13: 192-207. doi:10.1002/psp4.13088
[3] https://metrumresearchgroup.github.io/bbr/
[4] https://metrumresearchgroup.github.io/bbr.bayes/
[5] https://merge.metrumrg.com/expo/expo2-stan/
[6] https://www.metrumrg.com/merge-expo/
[7] https://mrgsolve.org/
[8] https://mc-stan.org/posterior/
[9] https://mc-stan.org/bayesplot/
[10] Kay K, Baron K, Green S, Callisto S, Johnston C, Barrett K, Pastoor D, Rogers J, Ruiz-Garcia A, Waterhouse T, Wiens M, Riggs M. A Suite of Open-Source Tools to Guide Efficient Pharmacometric Analyses. American Conference on Pharmacometrics (ACoP13); October-November 2022; Aurora, Colorado.
[11] https://merge.metrumrg.com/expo/expo3-nonmem-bayes/
