PMX perspective on Estimands
Mick Looby
Novartis, Switzerland
The concept of estimands in clinical drug development has re-emerged with the publication of the draft ICH E9 addendum. The concept is driven by efforts to address potential short-comings in intention to treat (ITT) analysis - only the randomized treatment assignment is taken into account in differentiating treatment groups at the outcome of a clinical trial. The new framework aims at facilitating a precise description of the treatment effect of interest by not only defining the population, the variable, and the population summary, but also explicitly accounting for events which occur after randomization, e.g. treatment discontinuation due to an adverse event, the use of concomitant medication, etc. The estimand discussion shifts the attention from the HOW to the WHAT in the design and analysis of confirmatory trials.
These questions of the what and the how in drug development occupied Lewis Sheiner in the early nineties, Sheiner (1991). In particular, he drew attention to the properties and short-comings of ITT analyses and how these may be addressed using model based methods. Sheiner and Rubin (1995) introduced in this context the terms “use-effectiveness” – the causal effect of prescribing a drug, and “method-effectiveness” – the causal effect of actually taking a drug. The authors stated that ITT may provide a valid estimate of the former under specific circumstances, but never of the latter, which may be more important in drug development and ultimately clinical usage.
The terms use- and method-effectiveness were first used in the context of contraceptives. Use-effectiveness tells us about the typical effectiveness of treatments in real life: this helps policy makers to make recommendations on treatments. However, to assess the true clinical potential of a medicine or intervention, it is necessary to estimate method effectiveness. Achieving this goal is not trivial. It typically requires models and assumption rich analyses. In some cases, particularly in a confirmatory setting, it is impossible. Irrespective of the challenges in estimating method effectiveness, regulatory approval requires confirmatory trials to asses use effectiveness. While, the Draft E9 addendum focuses on achieving this goal even in the presence of significant inter-current events, I believe the focus on the clinically relevant questions at the study design stage is beginning to have benefits. For example, at a workshop in dose finding in 2014, the EMA stated: ”Traditional statistical pairwise comparisons in phase 2 trials to support dose selection by testing for statistically significant differences between the groups are not a regulatory requirement and are suboptimal in terms of dose selection”. “Mathematical, statistical and pharmacological methodologies to characterise D-E-R and optimal dose selection are scientifically well developed, available for application and welcomed by regulators. These should be tailored to the specific development needs”. As with many topics, this development of using the most appropriate analyses to address the key questions was anticipated by Sheiner (1979) with the learn and confirm paradigm.
In this talk, we will present two examples carried out prior to the publication of the E9 addendum. These examples demonstrate that PMX analyses, as anticipated by Sheiner, can play a central role on drug development and approval, especially when these analyses are closely coordinated with statistics.
The first example will show how a PMX analysis strategy can be used to learn about method-effectiveness efficiently across trails and ultimate lead to the confirmation and approval of novel regimens.
The second example demonstrates how a PMX analysis can be used to address method-effectiveness of a novel drug combination in transplant medicine and ultimately enabling regulatory approval that would have been unlikely otherwise.
The estimand framework is an attempt to restore “the intellectual primacy to the questions we ask, not the methods which we answer them” Sheiner (1991). The discussion round the WHAT in clinical development is core to PMX and the discipline can be central to bringing focus to these questions together with our statistical colleagues. It is my firm belief that ‘opening the box’ on the ‘what questions’ in drug development through the estimand framework will ultimately lead to wider use of model-based methodologies, because they are often the most effective or only way to address key questions such as method-effectiveness. It is also my belief that this transformation presages ever close integration of PMX and statistics.
References:
[1]. Sheiner LB Clinical Pharmacology and Therapeutics (1991) The intellectual health of clinical drug evaluation.
[2]. Sheiner LB, Rubin DB. Clin Pharmacol Ther. 1995 Jan;57(1):6-15. Intention-to-treat analysis and the goals of clinical trials.
[3]. Sheiner LB Clin Pharmacol Ther. 1997 Mar;61(3):275-91. Learning versus confirming in clinical drug development.
[4]. Sheiner LB Br J Clin Pharmacol 2002;54:203-11. Is intent-to-treat analysis always (ever) enough?
