Population optimal experimental design for discrete type data
Joakim Nyberg, Mats O. Karlsson, Andrew C. Hooker
Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden
Objectives: Models with discrete type data are increasingly used in the field of drug development, especially to model pharmacodynamics.
To increase the efficiency in various stages in drug development, optimal experimental design has been used [1]. This approach is built upon the Cramer-Rao inequality that states that the inverse of the Fisher Information Matrix (FIM) is a lower bound of the uncertainty of the parameters in a model, given a specific design. Optimal design has extensively been used to optimize different type of continuous repeated measurements with mixed effect models but little research has been done on optimizing mixed effect models with repeated discrete type data. Nestorov et al [2] did optimize over a categorical model but without random effects and Ogungbenro et al [3] presented a method for calculating the sample size and power with random effects models but dependent on the distribution of the data.
The aim of this investigation is to develop and investigate methods for optimizing mixed effects models with discrete type data independent on the underlying data distribution using the Laplace approximation and Monte Carlo techniques using the optimal design software PopED [4,5].
Methods: Two models were used in this exercise. A dichotomous model with the probability of having response that was dependent on the dose, e.g. a high dose increased the probability of having a response. Inter individual variance was assigned to the parameter in an additive way with a variance of 0.1. This variance gives rise to ~25% difference in the probability for the response/non-response for 95% of the individuals with a low dose. The design contained 50 individuals with 30 observations each. However the 30 observations were split into 3 doses with 10 observations each. One dose (10 obs.) was fixed to a placebo dose and the other two doses were optimized between 0-1 units.
The second model that was investigated was a count model with a dose effect on the variance of the Poisson distribution. The Poisson variance was dependent of a baseline parameter with a random effect, a Dose50 parameter and the dose. The baseline had an exponential random effect with a CV of ~32%. The design had 20 individuals with 90 observations each, again split into 3 dose levels (30 obs/dose) and 30 observations were fixed to a placebo dose. Similar to the dichotomous model the remaining two doses were optimized between 0-1 units.
To calculate the Fisher information matrix (FIM) for these types of models, two different approaches were considered: 1) Approximate the likelihood with the Laplace approximation or 2) calculate the exact likelihood with Monte Carlo (MC) integration techniques. Calculating the likelihood enables computing of an observed FIM, i.e. a FIM dependent on a certain set of data. Further the expectation of the observed FIM is calculated with Monte Carlo integration over all data. To be able to have stable numerical derivatives and stable design optimization the sampling technique reused the samples from the first iteration (similar to the technique presented in Kuhn and Lavielle [6]). The code performing the FIM calculation was implemented in PopED as a penalty function and the random search and line search (with default settings) in PopED was used for the design optimization. The optimization criteria used was D-optimal (optimizing the determinant of FIM). Latin Hypercube (LH) sampling was used in the MC calculation of the likelihood to speed up and stabilize the likelihood calculation. The number of LH samples differed between the different models but was between 40-200 individual samples. The FIM was calculated both using the expectation of the gradient product of the first derivative of the ln likelihood with respect to the parameters as well as the expectation of the negative 2nd order derivative of the ln likelihood with respect to the parameters.
NONMEM [7] was used to investigate the uncertainty (covariance) by stochastic simulation and re-estimation (SSE). This was done by doing 1000 SSE and calculating the uncertainty from the parameter estimates. Numerous investigations of different observed FIM, calculated with NONMEM and the MATRIX=R option, was compared to the observed FIM with the same data calculated by PopED with either Laplace or MC.
Results: The likelihood approximation and calculation with 1) and 2) were successfully implemented and further the observed FIM and the FIM was computed. Slight differences between the exact likelihood and the Laplace approximation were observed. As an example; the -2 ln likelihood for a specific data set with the dichotomous model: 1631.912 (NONMEM), 1631.912 (PopED Laplace) and 1631.877 (PopED MC with 100 000 samples).
The observed FIM for both models showed similar results with NONMEM's R-matrix compared to the Laplace approximated observed FIM and the MC integrated observed FIM calculated with PopED. Further the FIM calculated with Laplace and MC was similar to the covariance calculated with NONMEM. However, a lower bound was observed, e.g. the sum of CV(%) for the parameters in the dichotomous model: 83.8% (NONMEM SSE), 80.3% (PopED Laplace) and 80.5% (PopED MC).
The D-optimal design (PopED Laplace) for the dichotomous model was found to have 10 more observations for the placebo dose than the original design and the last 10 observations were placed at 0.50 units. This gave a determinant of the FIM = 5.91E+05. The D-optimal design for PopED MC had also 10 observations at the Placebo dose but the last 10 observations were placed at 0.44 units with a determinant of the FIM = 6.00E+05.
The D-optimal design for the count data model was estimated to two high doses of 1 unit with both PopED Laplace and PopED MC. The determinant of the optimal design was |FIM|= 5.4E+07 and 6.0E+07 for Laplace PopED and Laplace MC respectively.
Calculation of the FIM with either first derivative of the likelihood with respect to the parameters or the 2nd order approach showed similar results but was dependent of the number of samples used to calculate the expectation of FIM over data.
Conclusions: A method for calculating optimal designs for discrete data mixed effects models was implemented. This method could be used for any type of model with a user specified likelihood that need a high order approximation to the likelihood (2nd order or exact).
The Laplace approximated likelihood showed similar results as the MC integrated likelihood. However, the optimal design differed for the dichotomous model between the Laplace approximation and the MC method. This difference was due to the approximation of the likelihood which indicates that the method of calculating the likelihood might be important.
The predicted covariances from the method also seemed to agree well with the covariances obtained with simulation and estimation. The largest difference was seen in the expected uncertainty of the random effect. The difference was very minor (a few CV(%)) and is expected because the Cramer-Rao inequality is a lower bound of the imprecision.
References:
[1]. Mentré, F., A. Mallet, and D. Baccar, Optimal design in random-effects regression models. Biometrika, 1997. 84(2): p. 429-442.
[2]. Nestorov, I., Graham, G., Duffull, S., Aarons, L., Fuseau, E., Coates, P. (2001). Modeling and stimulation for clinical trial design involving a categorical response: A phase II case study with naratriptan. Pharm. Res. 2001 18(8): p. 1210-1219.
[3]. Ogungbenro K. and L. Aarons. Sample Size Calculations for Population Pharmacodynamic Experiments Involving Repeated Dichotomous Observations. J. Biopharm. Stat. 2008. 18(6): p. 1212-1227.
[4]. Foracchia, M., et al., POPED, a software for optimal experiment design in population kinetics. Comput Methods Programs Biomed, 2004. 74(1): p. 29-46.
[5]. PopED, version 2.08 (2008). http://poped.sf.net/.
[6]. E. Kuhn, M. Lavielle. Maximum likelihood estimation in nonlinear mixed effects models. Comput. Statist. Data Anal. 2005. 39(2): p. 187-201.
[7]. NONMEM Users Guides, (1989-2006). Beal, S.L., Sheiner L.B., Boeckmann, A.J. (Eds.) Icon Development Solutions, Ellicott City, Maryland, USA.
