Fidelity to Treatment in School-based Interventions

Document Type : Original Article

Authors

1 PhD Student in Educational Psychology, Shahid Beheshti University, Tehran, Iran.

2 Assistant Professor, Department of Psychology, Shahid Beheshti University, Tehran, Iran.

3 Associate Professor, Department of Psychology, Shahid Beheshti University, Tehran, Iran.

Abstract

Methodology is the foundation for designing an effective intervention. In school-based studies, the term "fidelity to the treatment plan" or "intervention plan" includes methodological strategies essential for monitoring practical interventions. It ensures that the intervention is implemented as intended and that its key components contribute to producing desired outcomes. Fidelity to the treatment plan is a multidimensional concept that includes adherence to the intervention, intervention quality, intervention quantity, and intervention process. This framework enables researchers to ensure the validity and reliability of clinical interventions. Reporting on adherence to treatment and the implementation of an intervention plan offers insights into how to execute the intervention, the suitability of the intervention plan within its intended context, and the challenges encountered during implementation and maintenance. Failure to report on these aspects may result in a lack of information for other researchers and users about the project's conditions. Therefore, determining the optimal level of fidelity to the treatment plan in interventions and measurements is crucial. While researchers have increasingly recognized the importance of fidelity to the treatment plan in recent decades, this area still requires more serious attention. Loyalty to the treatment plan in school-based interventions, given the diverse contexts and structures of schools, as well as the dynamic school environment, demands special consideration. Some researchers and reputable publications have suggested that without reporting fidelity to the treatment plan, it is impossible to ascertain whether behavioral changes in subjects are truly influenced by the independent variable. This article aims to outline the dimensions of fidelity to the treatment plan in school-based interventions.

Keywords

References

  •  

    • Foster CE, Horwitz A, Thomas A, Opperman K, Gipson P, Burnside A, Stone DM, King CA. Connectedness to family, school, peers, and community in socially vulnerable adolescents. Child Youth Serv Rev. 2017 Oct; 81: 321-331. Https://‌org/10.1016/j.childyouth.2017.08.011
    • Hossain MM, Nesa F, Das J, Aggad R, Tasnim S, Bairwa M, Ma P, Ramirez G. Global burden of mental health problems among children and adolescents during COVID-19 pandemic: An umbrella review. Psychiatry Res. 2022 Nov; 317: 114814. https://org/10.1016/j.psychres.2022.114814
    • Abrams Kids, mental health is in crisis. Here’s what psychologists are doing to help. American Psychological Association. January 1, 2023; 54(1): 63p.

    https://www.apa.org/monitor/2023/01/trends-improving-youth-mental-health

    • Walker HM. Commentary: Use of evidence-based interventions in schools: Where we've been, where we are, and where we need to go. School Psychology Review. 2004 Sep 1; 33(3): 398-407. https://doi.org/10.1080/02796015.2004.12086256
    • Moir T. Why is implementation science important for intervention design and evaluation within educational settings?. InFrontiers in Education. 2018 Jul 25; (Vol. 3, p. 61). Frontiers Media SA.

    https://doi.org/10.3389/feduc.2018.00061

    • Dobson D, Cook TJ. Avoiding type III error in program evaluation: Results from a field experiment. Evaluation and program planning. 1980 Jan 1;3(4):269-76.

    https://doi.org/10.1016/0149-7189(80)90042-7

    • Harden, Samantha M et al. “Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review.” Systematic reviews 4 155. 8 Nov. 2015,

    https://doi.org/10.1186/s13643-015-0141-0. https://doi: 10.3389/fpubh.2019.00064

    • S. Department of Health and Human Services Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. Introduction to program evaluation for public health programs: A self-study guide. Atlanta, GA: Centers for Disease Control and Prevention, 2011.

    https://www.cdc.gov/evaluation/guide/index.htm

    • Rossi PH. Issues in the evaluation of human services delivery. Evaluation quarterly. 1978 Nov; 2(4): 573-99.

    https:/doi.org/10.1177/0193841X7800200404

    • Linnan L, Steckler A. Process evaluation for public health interventions and research. https://www.wiley.com/en-us/Process+Evaluation+for+Public+Health+Interventions+and+Research-p-9781119022480
    • Basch CE, Sliepcevich EM, Gold RS, Duncan DF, Kolbe LJ. Avoiding type III errors in health education program evaluations: a case study. Health Educ Q. 1985 Winter; 12(4): 315-31. https://org/10.1177/109019818501200311
    • Sanetti LM, Kratochwill TR. Treatment integrity assessment in the schools: An evaluation of the Treatment Integrity Planning Protocol. School Psychology Quarterly. 2009 Mar; 24(1): 24. https://org/10.1037/a0015431
    • Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity Criteria: Development, Measurement, and Validation. American Journal of Evaluation, 24(3), 315–340.

    https://doi.org/10.1016/S1098-2140(03)00057-2

    • Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implementation science. 2007 Dec; 2: 1-9. https://doi.org/1186/1748-5908-2-40
    • Gresham F. M. Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review. 1989; 18: 37–50. https://mtss4success.org/
    • Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: are implementation effects out of control?. Clinical psychology review. 1998 Jan 1; 18(1): 23-45. https://org/10.1016/S0272-7358(97)00043-3
    • O’Donnell CL. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of educational research. 2008 Mar; 78(1): 33-84.

    https://doi.org/10.3102/0034654307313

    • Munter C, Wilhelm AG, Cobb P, Cordray DS. Assessing fidelity of implementation of an unprescribed, diagnostic mathematics intervention. Journal of Research on Educational Effectiveness. 2014 Jan 1; 7(1): 83-113.

    https://doi.org/10.1080/19345747.2013.809177

    • Batsche GM, Kavale KA, Kovaleski JF. Competing views: A dialogue on response to intervention. Assessment for Effective Intervention. 2006 Oct; 32(1): 6-19.

    https://doi.org/10.1177/15345084060320010301

    • Zirkel PA, Thomas LB. State laws and guidelines for implementing RTI. Teaching Exceptional Children. 2010 Sep; 43(1): 60-73.

    https://doi.org/10.1177/004005991004300107

    • Hunley S, McNamara K, editors. Tier 3 of the RTI model: Problem solving through a case study approach. Corwin Press; 2009 Oct 7.

    https://doi.org/10.4135/9781452219028

    • Ledford JR, Wolery M, Gast DL. Controversial and critical issues in single case research. InSingle case research methodology‌. 2014 Mar 26; 377-396. Routledge.Rosenbaum, P. R. (1995). Observational Studies. Springer, New York. https://www.taylorfrancis.com/chapters/edit/10.4324/9780203521892-14/controversial-critical-issues-single-case-research-jennifer-ledford-mark-wolery-david-gast
    • Rosenbaum PR. Quantiles in nonrandom samples and observational studies. Journal of the American Statistical Association. 1995 Dec 1; 90(432): 1424-31.

    https://www.tandfonline.com/doi/abs/10.1080/01621459.1995.10476648

    • Rosenbaum PR. Covariance adjustment in randomized experiments and observational studies. Statistical Science. 2002 Aug; 17(3): 286-327. https://scholar.harvard.edu/files/imbens/files/comment_on_covariance_adjustment_in_randomized_experiments_and_observational_studies_by_paul_rosenbaum.pdf
    • Hulleman CS, Cordray DS. Moving from the lab to the field: The role of fidelity and achieved relative intervention strength. Journal of Research on Educational Effectiveness. 2009 Jan 30; 2(1): 88-110.

    https://doi.org/10.1080/19345740802539325

    • Cordray DS, Pion GM. Treatment strength and integrity: Models and methods. Strengthening research methodology: Psychological measurement and evaluation. 2006: 103-24.

    https://doi.org/10.1037/11384-006

    • George HP, Childs KE. Evaluating implementation of schoolwide behavior support: Are we doing it well?. Preventing School Failure: Alternative Education for Children and Youth. 2012 Aug 1; 56(4): 197-206.

    https://doi.org/10.1080/1045988X.2011.645909

    • Sanetti LM, Cook BG, Cook L. Treatment fidelity: What it is and why it matters. Learning Disabilities Research & Practice. 2021 Feb; 36(1): 5-11. https:// org/10.1111/ldrp.1223
    • Buckley P, Moore B, Boardman AG, Arya DJ, Maul A. Validating a fidelity scale to understand intervention effects in classroom-based studies. American Educational Research Journal. 2017 Dec; 54(6): 1378-413.

    https://doi: 10.3102/0002831217726522

    • فراهانی، حجت‌الله؛ آزادفلاح، پرویز؛ روشن‌چسلی، رسول؛ پورمحمدرضا، تجریشی، معصومه؛ اسرافیلیان، فروغ؛ فتحعلی لواسانی، فهیمه؛ چینی‌فروشان، فاطمه.روش‌شناسی واکاوی «شکست درمان» در پژوهش‌های روان‌درمانی: راهنمای بالین‌گران و پژوهشگران. نشریه روانشناسی بالینی و شخصیت. 1401؛ 19(2)؛ 172 -

    doi: 10.22070/CPAP.2022.15818.1202

     

    • O’Cathain A, Croot L, Sworn K, Duncan E, Rousseau N, Turner K, Yardley L, Hoddinott P. Taxonomy of approaches to developing interventions to improve health: a systematic methods overview. Pilot and feasibility studies. 2019 Dec; 5(1):1-27.

    doi: 10.1186/s40814-019-0425-6

    • شلانی، بیتا؛ آزادفلاح، پرویز؛ فراهانی، حجت‌الله؛ روشن چسلی، رسول؛ اسرافیلیان، فروغ. گام‌های عملی طراحی بسته‌های درمانی، مداخله‌ای و آموزشی در پژوهش‌های روان‌شناختی: رویکرد نقشه‌نگاری مداخله . نشریه روانشناسی بالینی و شخصیت. 1401؛ 20(1): 177 –

    https://‌doi: 10.22070/CPAP.2022.16313.1235

    • Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research. 2011 Jan; 38: 32-43.

    https://doi: 10.1007/s10488-010-0321-0

    • Kane MT. Validation in RL Breenan. Educational measurement 4th Ed. 2006.
    • Wilson M. Constructing measures: An item response modeling approach: An item response modeling approach. Routledge; 2004 Dec 13. https://www.routledge.com/Constructing-Measures-An-Item-Response-Modeling-Approach/Wilson/p/book/9781032261683
    • Jimerson SR, Burns MK, VanDerHeyden AM. Response to intervention at school: The science and practice of assessment and intervention. Handbook of response to intervention: The science and practice of assessment and intervention. 2007: 3-9.

    https://doi: 10.1007/978-1-4899-7568-3_1

    • Sullivan-Bolyai S, Crawford S, Bova C, Lee M, Quintos JB, Johnson K, Cullen K, Hamm T, Bisordi J, Ramchandani N, Fletcher J. PETS-D: Impact on diabetes management outcomes. The Diabetes Educator. 2015 Oct; 41(5): 537-49.

    https://doi: 10.1177/0145721715598383

    • Wiltsey Stirman S, A Gutner C, Crits-Christoph P, Edmunds J, Evans AC, Beidas RS. Relationships between clinician-level attributes and fidelity-consistent and fidelity-inconsistent modifications to an evidence-based psychotherapy. Implementation Science. 2015 Dec; 10(1): 1-0. https://doi: 10.1186/s13012-015-0308-z
    • Rasmussen LM, Patras J, Neumer SP, Adolfsen F, Martinsen KD, Holen S, Sund AM, Martinussen M. Facilitators and barriers to the implementation of EMOTION: an indicated intervention for young schoolchildren. Scandinavian Journal of Educational Research. 2020 Jul 28; 64(5): 728-43.

    https://doi: 10.1080/00313831.2019.1596976

    • Weist MD, Bruns EJ, Whitaker K, Wei Y, Kutcher S, Larsen T, Holsen I, Cooper JL, Geroski A, Short KH. School mental health promotion and intervention: Experiences from four nations. School Psychology International. 2017 Aug; 38(4): 343-62.

    https://doi: 10.1177/0143034317695379

    • Splett JW, Fowler J, Weist MD, McDaniel H, Dvorsky M. The critical role of school psychology in the school mental health movement. Psychology in the Schools. 2013 Mar; 50(3): 245-58. https://doi: 10.1002/pits.21677
    • Holmes SR, Thompson AM, Herman KC, Reinke WM. Designing interventions for implementation in schools: A multimethod investigation of fidelity of a self-monitoring intervention. School Psychology Review. 2020 Dec 27; 50(1): 42-51. https://org/10.1080/2372966X.2020.1870868
    • Peacock GG, Ervin RA, Daly EJ, Merrell KW, editors. Practical handbook of school psychology: Effective practices for the 21st century. Guilford Press; 2009 Sep 2.

    https://www.guilford.com/books/Practical-Handbook-of-School-Psychology/Peacock-Ervin-Daly-Merrell/9781462507771/contents.

    • Solomon BG, Klein SA, Politylo BC. The effect of performance feedback on teachers' treatment integrity: A meta-analysis of the single-case literature. School psychology review. 2012 Jun 1; 41(2): 160-75.

    https://doi: 10.1080/02796015.2012.12087518

    • Harn B, Parisi D, Stoolmiller M. Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools?. Exceptional Children. 2013 Apr; 79(2): 181-93.

    https://doi: 10.1177/001440291307900204

    • Espelage DL, Rose CA, Polanin JR. Social-emotional learning program to reduce bullying, fighting, and victimization among middle school students with disabilities. Remedial and special education. 2015 Sep; 36(5): 299-311.

    https://‌doi.org/10.1177/0741932514564564

    • Powers K, Clarke S, Phillips J, Holmes JA, Cripps R, Craven K, Farrin A, das Nair R, Radford KA. Developing an implementation fidelity checklist for a vocational rehabilitation intervention. Pilot and Feasibility Studies. 2022 Dec; 8(1): 1-2. https://doi: 10.1186/s40814-022-01194-x
    • Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. Bmj. 2000 Sep 16; 321(7262):694-6.

    https://doi: 10.1136/bmj.321.7262.694

    • Hasson H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implementation science. 2010 Dec; 5(1): 1-9. https://doi: 10.1186/1748-5908-5-67
    • Lipsey MW, Cordray DS. Evaluation methods for social intervention. Annual review of psychology. 2000 Feb; 51(1): 345-75.

    https://doi: 10.1146/annurev.psych.51.1.345

    • Baranowski T, Stables G. Process evaluations of the 5-a-day projects. Health Education & Behavior. 2000 Apr; 27(2): 157-66.

    https://doi: 10.1177/109019810002700202

    • Begeny J, Upright J, Easton J, Ehrenbock C, Tunstall K. Validity estimates and functionality of materials and procedures used to monitor the implementation integrity of a reading intervention. Journal of Applied School Psychology. 2013 Jul 1; 29(3): 284-304.

    https://doi: 10.1080/15377903.2013.810187

    • https://mtss4success.org/essential-components
    • Reinke WM, Herman KC, Thompson A, Copeland C, McCall CS, Holmes S, Owens SA. Investigating the longitudinal association between fidelity to a large-scale comprehensive school mental health prevention and intervention model and student outcomes. School psychology review. 2020 Dec 1; 50(1): 17-29.

    https://doi.org/10.1080/2372966X.2020.1870869

    • Elfner Childs K, Kincaid D, Peshak George H. A model for statewide evaluation of a universal positive behavior support initiative. Journal of Positive Behavior Interventions. 2010 Oct; 12(4): 198-210. https://doi: 10.1177/1098300709340699
    • Algozzine B, Horner RH, Sugai G, Barrett S, Dickey SR, Eber L, Tobin T. Evaluation blueprint for school-wide positive behavior support. Eugene, OR: National Technical Assistance Center on Positive Behavior Interventions and Support. 2010 Jan 1. www.pbis.org
    • Cohen R, Kincaid D, Childs KE. Measuring school-wide positive behavior support implementation: Development and validation of the benchmarks of quality. Journal of Positive Behavior Interventions. 2007 Oct; 9(4): 203-13. https://doi: 10.1177/10983007070090040301
    • Chiu AW, Langer DA, McLeod BD, Har K, Drahota A, Galla BM, Jacobs J, Ifekwunigwe M, Wood JJ. Effectiveness of modular CBT for child anxiety in elementary schools. School psychology quarterly. 2013 Jun; 28(2): 141. https://doi: 10.1037/spq0000017
    • Eiraldi R, Mautone JA, Khanna MS, Power TJ, Orapallo A, Cacia J, Schwartz BS, McCurdy B, Keiffer J, Paidipati C, Kanine R. Group CBT for externalizing disorders in urban schools: Effect of training strategy on treatment fidelity and child outcomes. Behavior therapy. 2018 Jul 1; 49(4): 538-50. https://doi: 10.1016/j.beth.2018.01.001
    • Goulet M, Archambault I, Janosz M, Christenson SL. Evaluating the implementation of Check & Connect in various school settings: Is intervention fidelity necessarily associated with positive outcomes?. Evaluation and Program Planning. 2018 Jun 1; 68: 34-46. https://doi: 10.1016/j.evalprogplan.2018.02.004
    • Wilder DA, Atwell J, Wine B. The effects of varying levels of treatment integrity on child compliance during treatment with a three‐step prompting procedure. Journal of applied behavior analysis. 2006 Sep; 39(3): 369-73.

    doi: 10.1901/jaba.2006.144-05

    • Noell GH, Gresham FM, Gansle KA. Does treatment integrity matter? A preliminary investigation of instructional implementation and mathematics performance. Journal of Behavioral Education. 2002 Mar; 11: 51-67. https://doi: 10.1023/A:1014385321849
    • Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American journal of community psychology. 2008 Jun; 41: 327-50. https:// doi: 10.1007/s10464-008-9165-0
    • Collier-Meek MA, Fallon LM, Sanetti LM, Maggin DM. Focus on implementation: Assessing and promoting treatment fidelity. Teaching Exceptional Children. 2013 May; 45(5): 52-9.

    https://doi.org/10.1177/004005991304500506

    • Rathvon N. Effective school interventions: Evidence-based strategies for improving student outcomes. Guilford Press; 2008 Aug 14. https://www.guilford.com/books/Effective-School-Interventions/Burns-Riley-Tillman-Rathvon/9781462526147

Files

Share

How to cite

Statistics