Background on estimands and why are they important?
Mouna Akacha
Novartis, Switzerland
The topic of ‘estimands’ is shaking up the biostatistics community as it is at the heart of a draft addendum (ICH, 2017) to the ICH* E9 guideline (ICH, 1998) – the holy grail of pharmaceutical statistics. Broadly speaking, an estimand for a clinical trial represents “WHAT is most important to estimate in order to address the scientific question of interest” (Ruberg & Akacha, 2017).
The draft addendum presents a new framework which aims at facilitating a precise description of the treatment effect of interest by not only defining the population, the variable, and the population summary, but also by explicitly accounting for so-called “intercurrent events” which occur after randomization, e.g. treatment discontinuation due to an adverse event or the use of concomitant medications. Employing “estimand thinking” at the study design stage, whereby the team considers the impact of such intercurrent events on the WHAT will help in describing drug effects more precisely. The estimand framework may thus restore some of the intellectual health of clinical drug evaluation as primacy is re-assigned to “the questions we ask, not the methods by which we answer them” (Sheiner, 1991). Established practices in pharmaceutical research suggest that relatively more focus has been paid to the ‘HOW’ rather than to the ‘WHAT’.
Furthermore, recent discussions on estimands have highlighted that some established paradigms in the pharmaceutical industry may not always be aligned with clinically meaningful treatment effect measures. One of these paradigms is the so-called intention-to-treat (ITT) approach – by some considered the “steadfast beacon in the foggy vistas of biomedical experimentation” (Efron, 1998).
The ITT approach came to prominence decades ago and was institutionalized in drug development with the publication of the ICH-E9 guideline in 1998 (ICH, 1998). The resulting analysis yields a treatment effect estimate that is often described as the effect of the treatment assigned at randomization or the treatment-policy effect. Despite its ubiquitous use in randomized, controlled clinical trials, the ITT analysis and its interpretation are neither consistently applied nor well understood by many. Indeed, the resulting estimates are sometimes difficult to interpret and may not provide an intuitive or clinically meaningful estimate of treatment effects (Sheiner(2002), Keene(2011)).
Alternative estimands to the treatment-policy estimand which are potentially more clinically meaningful are discussed in the draft addendum. However, deviations from the treatment-policy estimand implied through the ITT approach should not be taken lightly. For example, some estimands may be very relevant from a clinical perspective, but associated quantitative methods are complex and rely on various assumptions that cannot be verified from the data (Sheiner(2002), Nedelman et al. (2007)). In such cases, sensitivity analyses play an important role. Not surprisingly, the role of “sensitivity analyses” is also discussed in detail in the draft addendum.
More generally, the draft addendum is not about ITT or about statistics - rather it provides everyone involved in drug development with a language to have an early, more informed and transparent discussions with all key stakeholders (clinicians, regulators, payers, patients, etc.) to align on clinically meaningful trial objectives at the right time, i.e., during the protocol design phase (Akacha & Kothny, 2017). This is a very important opportunity for quantitative scientists who can guide the discussion, moderate it, and raise important questions that will facilitate the choice of clinically meaningful estimands, targeted designs and appropriate analyses.
*ICH = International Conference on Harmonization
References:
1. Akacha, M. & Kothny, W. Opinion paper on estimands: Opportunities and challenges. Clin. Pharmacol. Ther. 102, 894-896 (2017).
2. Efron: Forword – Limburg Compliance Symposium. Stat. in Med.17, 249 -250 (1998).
3. International Conference on Harmonisation. ICH Harmonised Tripartite Guideline: Statistical principles for clinical trials E9. <http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/ Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf>. Accessed June1, 2015.
4. International Conference on Harmonisation. Draft ICH E9 (R1) Technical Document: Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials. <http://www.ema.europa.eu/docs/en_GB/document_library/ Scientific_guideline/2017/08/WC500233916.pdf>. Accessed on 18 Sep 2017.
5. Keene, O.N. Intent-to-treat analysis in the presence of off-treatment or missing data. Pharm. Stat. 10, 191–195 (2011).
6. Nedelman, J.R., Rubin, D.B. and Sheiner, L.B. Diagnostics for confounding in PK/PD models for oxcarbazepine. Stat. in Med.26, 290 -308 (2007).
7. Ruberg, S.J. & Akacha, M. Considerations for evaluating treatment effects from randomized clinical trials. Clin. Pharmacol. Ther. 102, 917-923 (2017).
8. Sheiner, L.B. The intellectual health of clinical drug evaluation. Clin. Pharmacol. Ther. 50, 4-9 (1991).
9. Sheiner, L.B. Is intent-to-treat analysis always (ever) enough? J. Clin. Pharmacol. 54, 203-211 (2002).
