On the error of the Laplace approximation when lags and duration parameters have between-subject variation
Andreas Noack (1,2), Patrick Kofod Mogensen (1,2), and Vijay Ivaturi (1)
(1) Pumas-AI, USA, (2) JuliaHub, USA
It is well known that estimation based on the Laplace approximation, such as the First Order Conditional Estimation (FOCE), of models with lag or duration parameters is challenging, see [1, 2], because of the discontinuity in the compartment concentration affected by the lag or duration parameter. This poster provides further insight into the reasons for these challenges. We focus on the situation where the lag or duration parameter is modeled with between-subject variability. We argue that, in this case, the Laplace approximation-based methods are generally inappropriate.
Objectives: Provide details on why the estimation of models based on lag and duration parameters with between-subject variability often fails when based on the Laplace approximation.
Methods: We analyze the problem along two dimensions. First, the individual likelihood function is analyzed both algebraically and visually to demonstrate how lags and duration parameters influence the shape of the individual likelihood function. Second, the effect on the population likelihood is quantified through a simulation exercise in both the Pumas and NONMEM software packages.
Results: The analysis of the individual likelihood demonstrates that lag and duration parameters introduce discontinuities in the concentration curve in the compartment they apply to and that this discontinuity introduces non-differentiable points in the individual log-likelihood functions. The Laplace approximation is based on a quadratic approximation of the individual log-likelihood, so the non-differentiable points invalidate the use of the Laplace approximation. We demonstrate that this can result in both large approximation errors but also, more problematically, lead to highly erratic objective functions. The reasons for these issues are twofold. The first issue is that when the minimum of the joint log-likelihood is one of the non-differentiable points, the zero gradient assumption of the Laplace approximation is necessarily violated. Furthermore, the Hessian is not defined is such points so even if a computer program returns some kind of Hessian approximation, the Laplace approximation is expected to be poor. The second issue is that the lag and duration parameters are likely to introduce several local minima around the non-differentiable point. This is the reason for the erratic objective function since the point around which the Laplace approximation is compute will change with the population parameters.
Conclusions: When modeling lag and duration parameters with between-subject variability, we recommend that estimation methods based on the Laplace approximation are avoided. They can introduce large errors in the approximation and they may fail to converge to a value close to the true maximum likelihood estimate. We recommend that the modeler instead uses exact methods or completely avoids modeling approaches that introduce non-differentiabilities in the likelihood. E.g., by using delay differential equation methods such as transit compartment models.
References:
[1] Savic, Radojka M., Daniël M. Jonker, Thomas Kerbusch, and Mats O. Karlsson. "Implementation of a transit compartment model for describing drug absorption in pharmacokinetic studies." Journal of pharmacokinetics and pharmacodynamics 34 (2007): 711-726.
[2] Mould, Diane R., and Richard Neil Upton. "Basic concepts in population modeling, simulation, and model‐based drug development—part 2: introduction to pharmacokinetic modeling methods." CPT: pharmacometrics & systems pharmacology 2, no. 4 (2013): 1-14.