Make models great again by optimally restricting parameters to make non-identifiable models provably identifiable
Mohamed Tarek
(1) PumasAI Inc, USA; (2) University of Sydney Business School, Australia
Objectives:
One of the goals of statistical learning is to identify the underlying parameter values in a parametric model that best fit the observed data. In many practical scenarios, some parameters in a model may not be identifiable.
The goal of this study is to propose a new methodology to identify the minimum number of parameters that should be fixed in a model to ensure local practical identifiability (LPI) in the neighbourhood of given parameter values $\theta$. This method can be used to guide pharmacometric model development or to restrict quantitative system pharmacology (QSP) models to make them identifiable.
Methods:
The expected Fisher Information Matrix (FIM) is an important diagnostic which can be used to detect LPI given the model and the experiment design [1]. The first order approximation of the expected FIM [2,3] has also been used successfully to analyze the LPI of pharmacometrics nonlinear mixed effect (NLME) models [4,5,6].
The positive definiteness (non-singularity) of the expected FIM $M(\theta)$ at parameters $\theta$ is a sufficient condition for LPI at $\theta$. Under more strict assumptions which are more difficult to verify, the positive definiteness of $M$ is even a necessary condition for LPI [1]. Restricting the parameters in a model is one of the simplest ways to make a non-identifiable model identifiable. Given that the FIM of a restricted model is the sub-matrix of the full model's FIM, this work tries to find the largest sub-matrix of $M$ that is non-singular by formulating this task as an optimization problem. The goal is to find the smallest subset of the parameters to be fixed to make the model identifiable.
Let $x_i \in \{ 0, 1 \}$ be a binary decision variable that is 0 if parameter $\theta_i$ is to be estimated and 1 if the parameter is fixed to its given value. The LPI problem is formulated as the following mixed-integer semi-definite programming (MISDP) problem: minimize $\sum_i x_i$ subject to $C + 100 \cdot \text{diag}(x) - \epsilon I \succeq 0$, where $C$ is the scaled version of $M$ such that the diagonal elements larger than a threshold (1e-3 in this work) are all normalized to 1 by scaling the corresponding rows and columns. $\text{diag}(x)$ is the diagonal matrix with diagonal $x$. $\succeq 0$ is the positive semi-definite constraint and $\epsilon$ is a small tolerance to ensure that the matrix is strictly positive definite. Adding 100 to a diagonal element of $C$ is analogous to specifying a strong prior for this parameter practically eliminating it from the model. An additional constraint can be added to sequentially discover distinct ways to make the model identifiable giving unique suggestions to fix different combinations of parameters.
To solve the above MISDP, an outer approximation mixed integer conic solver was used [7] with an interior point conic sub-solver [8]. The method proposed here was implemented in the OptimalDesign module of the Pumas suite of products.
Results:
To test the methodology, 2 test models were used. The first test model was a toy 1 compartment PK model with a depot and a central compartment. An "intentional typo" was introduced assuming the observed concentration is $\text{dv} \sim N(\text{Central}, \sigma)$ instead of the correct $\text{dv} \sim N(\text{Central} / \text{Vc}, \sigma)$, dividing by the volume of distribution parameter $\text{Vc}$. This typo made the model structurally non-identifiable. The second test model was a 5-compartment lung model [9,10]. The proposed method was able to suggest various parameter combinations to fix making both test models identifiable, verified by simulation. The runtime was also reasonably fast converging in well under a minute using a low tolerance when analyzing the lung model. The toy model took less than a second to analyze.
Conclusions:
Choosing which parameters to fix in a non-identifiable model to make it identifiable can be posed as a tractable MISDP problem. By enforcing the expected FIM sufficient (but not necessary) condition, it can be overly conservative recommending the fixing of some parameters which are not strictly necessary to fix to make the model identifiable. However in practice, the proposed method seems to give reasonable suggestions for which parameters to fix given our test models. This method can potentially be used to restrict large QSP models as part of NLME model development.
References:
[1] Thomas J. Rothenberg. Identification in parametric models. Econometrica, 1971.
[2] F. Mentre, A Mallet, and D. Baccar. Optimal design in random-effects regression models. Biometrika, 1997.
[3] S. Retout and F Mentre. Further development of the fisher information matrix in nonlinear mixed-effects models with evaluation in population pharmacokinetics. Journal of biopharmaceutical statistics, 2003.
[4] V. Shivva, K. Korell, I. Tucker, and S. Duffull. An approach for identifiability of population pharmacokinetic-pharmacodynamic models. CPT Pharmacometrics & Systems Pharmacology, 2013.
[5] Stephen Dufful, A workflow for resolving model internal consistency in use-reuse settings (aka repairing unstable models). PAGANZ, 2024.
[6] Dan Wright. The identifiability of a turnover model for allopurinol urate-lowering effect. PAGANZ, 2024.
[7] Coey, Chris and Lubin, Miles and Vielma, Juan Pablo. Outer approximation with conic certificates for mixed-integer convex problems. Mathematical Programming Computation, 2020.
[8] Chris Coey and Lea Kapelevich and Juan Pablo Vielma. Solving natural conic formulations with Hypatia.jl. INFORMS Journal on Computing, 2023.
[9] R. Hendrickx, E. L. Bergstrom, D. L. I. Janzen, M. Friden, U. Eriksson, K. Grime, and D. Ferguson. Translational semi-physiological model to predict human plasma and lung pharmacokinetics and pulmonary efficacy after oral drug inhalation of bronchodilators. CPT Pharmacometrics & Systems Pharmacology, 2018.
[10] David Lars Ivan Janzen. Structural Identifiability and Indistinguishability in Mixed-Effects Models. PhD Thesis, 2016.