Full Covariate Models as an Alternative to Methods Relying on Statistical Significance for Inferences about Covariate Effects: A Review of Methodology and 42 Case Studies.
Marc R. Gastonguay
Metrum Research Group & Metrum Institute, Tariffville, CT, USA
Objectives: Covariate (COVT) modeling in population (POP) pharmacokinetics (PK) and pharmacodynamics (PD) has historically been characterized by stepwise hypotheses testing based methods. More recently, a full COVT model (FCM) approach has been proposed for COVT effect inferences in POP PK and PD. The objective of the current work was to perform a systematic review of analyses utilizing the FCM in order to contrast inferences based on statistical significance vs. clinical importance.
Methods: This analysis included the review of 42 case studies of FCM for POP PK (32) or PD (10) analyses, utilizing data from end of Phase 2b or end of Phase 3 development, and conducted within the last 8 years. In all cases, FCM were constructed according to previously described methods [1]. Each analysis included a data reduction step, FCM construction in NONMEM (v. 5, 6 or 7) and determination of 95% CI for parameter estimates (bootstrap or NONMEM asymptotic standard errors). Cases were summarized according to the following characteristics: 1) successful search minimization (MIN), 2) successful covariance matrix of the estimates ($COV), 3) statistical significance of COVT effects, and 4) clinical importance of COVT effects based on magnitude and precision of COVT effect estimates, relative to a clinical effect reference.
Results: Across analyses, the median (MED) number of COVT effects included in the source data was 14 (range 4-60), while the MED number of COVT in each FCM was 6 (range 1-19). 100% of models had successful MIN, and 98% had successful $COV. A total of 258 COVT effects were estimated. 48% of COVT were statistically significant (SS), while 52% were not (NSS). 24% of covariate effects were clinically important (CIMP), 48% were not clinically important (NCIMP), and 28% were insufficiently informed (II). Statistical significance was not a good predictor of clinical importance, where 16% of COVT effects were SS but NCIMP, and 9% of COVT effects were SS but too imprecise to assess clinical importance (e.g. II). Similarly, a lack of statistical significance was not a good indicator of lack of clinically important effect. 20% of COVT effects were NSS, but were not precise enough to rule-out clinical importance (e.g. II).
Conclusions: The FCM method provides for a more useful and accurate inference about COVT effects, when compared to methods based solely on statistical significance procedures.
References:
[1] Gastonguay, M. R. The AAPS Journal. 2004 (6), S1, W4354.
