Lightning Oral Presentation Australian and New Zealand Stroke Organisation Conference 2026

Developing a novel tool to assess treatment fidelity in complex stroke rehabilitation interventions (139588)

Woon Ki Chew 1 2 3 , Emma Power 4 , David Copland 1 2 3 , Kirstine Shrubsole 1 2 3
  1. Queensland Aphasia Research Centre, The University of Queensland, Brisbane, Queensland, Australia
  2. STARS Education and Research Alliance, Metro North Hospital and Health Service, Brisbane, Queensland, Australia
  3. School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, Queensland, Australia
  4. University of Technology Sydney, Faculty of Health, Sydney, New South Wales, Australia

Background/Aims: Stroke rehabilitation interventions are often complex,1 making treatment fidelity evaluation essential for accurate interpretation and replication of outcomes.2 However, development of robust fidelity measures remains under-reported,3 which undermines confidence in fidelity findings.4 This study aimed to develop a treatment fidelity checklist for a complex post-stroke rehabilitation intervention, the Comprehensive High-dose Aphasia Treatment (CHAT) program.

Methods: Checklist development followed a five-step systematic process: (1) review of previous fidelity measures, (2) development of an intervention framework specifying core treatment content, (3) creation of checklist and scoring guidelines, (4) expert review for content alignment, and (5) pilot-testing with the first five participants in the CHAT trial.3 Iterative changes were recorded, and inter-rater reliability was assessed using percentage agreement and Gwet’s AC1. Item-level agreement identified areas requiring refinement.

Results: The checklist included nine core components with 50 target behaviours. Scoring guidelines specified criteria, definitions, data sources and scoring formulas. Refinements included adding treatment components (e.g. program duration), reordering items, and providing scoring examples. Inter-rater reliability was very good (Gwet’s AC1 = 0.93–1.00, p < 0.0001), with percentage agreement of 80% – 100% across items. Components requiring greater clinical judgement (e.g. evidence-based therapy) showed lower agreement.

Conclusion: This study presents a reliable fidelity tool for complex stroke rehabilitation interventions, which will be applied in the multi-site CHAT trial. The structured process advances transparency, quality and reproducibility of fidelity measure development, and supports clinicians in delivering complex interventions consistently in stroke care practice.

  1. Redfern J, McKevitt C, Wolfe CDA. Development of Complex Interventions in Stroke Care: A Systematic Review. Stroke. 2006;37(9):2410-2419. doi:10.1161/01.STR.0000237097.00342.a9
  2. Bellg AJ, Borrelli B, Resnick B, et al. Enhancing Treatment Fidelity in Health Behavior Change Studies: Best Practices and Recommendations From the NIH Behavior Change Consortium. Health Psychology. 2004;23(5):443-451. doi:10.1037/0278-6133.23.5.443
  3. Walton H, Spector A, Tombor I, Michie S. Measures of fidelity of delivery of, and engagement with, complex, face‐to‐face health behaviour change interventions: A systematic review of measure quality. British journal of health psychology. 2017;22(4):872-903. doi:10.1111/bjhp.12260
  4. Walton H, Spector A, Williamson M, Tombor I, Michie S. Developing quality fidelity and engagement measures for complex health interventions. British journal of health psychology. 2020;25(1):39-60. doi:10.1111/bjhp.12394