Item Response Theory Analysis of the Scale for the Assessment and Rating of Ataxia in Autosomal Recessive Cerebellar Ataxias
Alzahra Hamdan (1), Andrew C. Hooker (1), Xiaomei Chen (1), Andreas Traschütz (2,3), Rebecca Schüle (2,4), Matthis Synofzik (2,3), Mats O. Karlsson (1)
(1) Pharmacometrics Research Group, Department of Pharmacy, Uppsala University, Sweden (2) Department of Neurodegenerative Diseases, Center for Neurology and Hertie Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany (3) German Center for Neurodegenerative Diseases (DZNE) Tübingen, 72076 Tübingen, Germany (4) Division of Neurodegenerative Diseases, Department of Neurology, Heidelberg University Hospital, Germany
Background and objectives: Autosomal Recessive Cerebellar Ataxias (ARCAs) is a heterogenous group of rare and ultra-rare neurodegenerative genetic diseases that mainly affect cerebellum and its associated tracts [1]. Given that disease-modifying therapies are now on the horizon targeting an increasing number of genetic ataxias [2], a reliable clinical outcome assessment (COA) is needed for clinical studies for the treatment of such rare diseases. The Scale for the Assessment and Rating of Ataxia (SARA) is by far the most widely used COA for assessing the severity and progression of cerebellar ataxias [3]. However, an analysis of its performance in reflecting disease severity on the item level has not been fully conducted. The aim of this work was to analyze the properties of the SARA using Item Response Theory (IRT) and assess its applicability across genetic ARCA subpopulations.
Methods: A unidimensional IRT model was built to analyze SARA sub-scores data taken from the ARCA registry database [4], which comprises 990 patients with a total of 1932 visits, and mostly from a total of 115 defined genetic diagnoses. Item Characteristic Functions were used to model the relationship between the probability of a particular item response and the underlying individual’s latent variable (LV); i.e., disease status. The properties of SARA items were captured in the item-specific parameters; item discrimination and item difficulty. The informativeness of individual items was also evaluated by calculating the amount of Fisher information provided by each item over the entire LV range. The unidimensionality assumption was thoroughly assessed by evaluating both the correlations between item responses and correlations between item model residuals. Furthermore, the correlations among items for observed and model-predicted data were compared, respectively. The IRT model was implemented using NONMEM version 7.5 [5]. The model code and diagnostics were generated using the piraid package in R [6].
A subpopulation analysis was conducted to evaluate applicability of the developed IRT model to the different genetic ARCA subpopulations. The individual likelihoods, resulting from the maximum likelihood estimation for the IRT model fitted to the whole ARCA dataset, were compared between each of the genetic subpopulations and the entire ARCA population. The difference in means of the individual Objective Function Values (iOFVs) between a specific subpopulation and the entire population was calculated along with the 95% confidence interval (CI).
Results and discussion: A unidimensional IRT model was successfully established modelling the relationship between item responses and one single latent variable representing disease status. The item discrimination parameters varied between 1.5 and 2.9 with a mean of 2.1. Compared to previously published IRT models, SARA item discrimination parameters are higher than many discrimination values of other neurological diseases’ COAs [7–10]. The difficulty parameters of each response category were sufficiently spaced suggesting that the item categories were well designed allowing clinicians to precisely rate the patient’s performance in each task. The item information profiles have shown that all SARA items are contributing to the informativeness of SARA with varying levels across the LV scale.
The unidimensionality checks have shown the adequacy of the unidimensionality assumption. This is indicated by the similarly high data correlation coefficients among items (range: 0.36-0.69, median: 0.52). The correlation patterns were also mimicked by model simulations (range: 0.43-0.61, median: 0.51). The cross-item residual correlations were slightly negative for both the original dataset (range: -0.3-0.14, median: -0.14) and simulated datasets (range: -0.20- -0.08, median: -0.12) which further supports the adequacy of the model.
The subpopulation analysis has shown an absence of evidence for any model misfits on the subpopulation level and hence the model applicability to all ARCA subpopulations. This is indicated by the 95% CI of the mean difference in iOFVs encompassing zero in all studied subpopulations.
Conclusions: The IRT analysis has shown that SARA is a well-performing assessment as indicated by its item parameters, Fisher information profiles, and the agreement with the unidimensionality assumption. The developed IRT framework provides a thorough description of SARA characteristics facilitating its utilization as a COA in future treatment trials and across a large number of genetic ataxias.
Acknowledgment: This work was supported by the European Joint Programme on Rare Diseases (EJP RD) Joint Transnational Call 2019 for the EJP RD WP20 Innovation Statistics consortium “EVIDENCE-RND”. Moreover, work in this project was supported by the Clinician Scientist programme "PRECISE.net" funded by the Else Kröner-Fresenius-Stiftung.
References:
[1] Synofzik M, Németh AH. Chapter 5 - Recessive ataxias. In: Manto M, Huisman TAGM, editors. Handbook of Clinical Neurology [Internet]. Elsevier; 2018 [cited 2023 Mar 7]. p. 73–89. (The Cerebellum: Disorders and Treatment; vol. 155). Available from: https://www.sciencedirect.com/science/article/pii/B9780444641892000056
[2] Synofzik M, Puccio H, Mochel F, Schöls L. Autosomal Recessive Cerebellar Ataxias: Paving the Way toward Targeted Molecular Therapies. Neuron. 2019 Feb 20;101(4):560–83.
[3] Schmitz-Hübsch T, du Montcel ST, Baliko L, Berciano J, Boesch S, Depondt C, et al. Scale for the assessment and rating of ataxia: development of a new clinical scale. Neurology. 2006 Jun 13;66(11):1717–20.
[4] Traschütz A, Reich S, Adarmes AD, Anheim M, Ashrafi MR, Baets J, et al. The ARCA Registry: A Collaborative Global Platform for Advancing Trial Readiness in Autosomal Recessive Cerebellar Ataxias. Frontiers in Neurology [Internet]. 2021 [cited 2022 Aug 26];12. Available from: https://www.frontiersin.org/articles/10.3389/fneur.2021.677551
[5] Beal SL, Sheiner LB, Boeckmann A, Bauer RJ. NONMEM user’s guides (1989–2009). Ellicott City: Icon Development Solutions; 2009.
[6] Nordgren R, Ueckert S. piraid [Internet]. Uppsala University, Pharmacometrics Research Group; 2022 [cited 2023 Jan 3]. Available from: https://github.com/UUPharmacometrics/piraid
[7] Gottipati G, Karlsson MO, Plan EL. Modeling a Composite Score in Parkinson’s Disease Using Item Response Theory. AAPS J. 2017 May;19(3):837–45.
[8] Ueckert S, Plan EL, Ito K, Karlsson MO, Corrigan B, Hooker AC, et al. Improved Utilization of ADAS-Cog Assessment Data Through Item Response Theory Based Pharmacometric Modeling. Pharm Res. 2014 Aug 1;31(8):2152–65.
[9] Novakovic AM, Krekels EHJ, Munafo A, Ueckert S, Karlsson MO. Application of Item Response Theory to Modeling of Expanded Disability Status Scale in Multiple Sclerosis. AAPS J. 2017 Jan 1;19(1):172–9.
[10] Haem E, Doostfatemeh M, Firouzabadi N, Ghazanfari N, Karlsson MO. A longitudinal item response model for Aberrant Behavior Checklist (ABC) data from children with autism. J Pharmacokinet Pharmacodyn. 2020 Jun 1;47(3):241–53.
