A significant challenge facing autism service providers is the effective and consistent delivery of evidence-based practices to the children they serve. A number of research reviews, including the National Standards Project (Rue, Knox, Welchons, Murzycki, Pollack, & Class, 2015) and the National Professional Development Center on Autism Spectrum Disorder (National Professional Development Center, n.d.), have identified research-supported practices for children with autism. Despite this knowledge base, ensuring staff are consistently using these practices to deliver the best services possible can be difficult.
Child receiving instruction as part of PBIS Autism
To address this challenge, we have developed a specialized Positive Behavioral Interventions and Supports (PBIS) model specifically designed to meet the needs of individuals with autism. PBIS uses a three-tiered system to organize interventions and data-based decision-making to teach and encourage valued behaviors (Dunlap, Goodman, McEvoy, & Paris, 2010). PBIS has been effectively modified for a variety of behavioral problems and settings (Fernandez & McClain, 2014; Jolivette et al., 2014). Our PBIS Autism model utilizes the multi-tiered framework to systematize staff training and implementation of evidence-based practices.
Multi-Tiered Model
The evidence-based practices incorporated into our PBIS Autism model emphasize the use of principles of Applied Behavior Analysis (ABA) to teach critical skills and prevent significant problem behavior. The model focuses on training staff to implement these practices with fidelity, thereby providing children with autism a therapeutic environment that supports positive social behaviors. Tier 1 highlights universal practices that should be in place for all children. These include practices such as using visual supports, creating opportunities for communication, and providing high levels of engagement in meaningful learning activities. For children who are not successful with Tier 1 alone, Tier 2 and 3 supports offer more intensive and individualized supports designed to meet the unique needs of the child. Table 1 outlines the evidence-based practices aligned with each tier within the framework.
Implementation of our PBIS Autism model involves a program of systematic professional development, observations in the natural setting (e.g., classroom, residence), structured data collection during observations, feedback and coaching to help staff reach identified goals for each targeted practice. Professional development seminars focus on providing staff with foundational knowledge in each content area. Instilling that knowledge in day-to-day application revolves around comprehensive coaching and feedback in the natural setting to support consistent implementation of evidence-based practices. Professional development seminars are delivered via online technology and with live trainers, depending on the capacity and needs of the program.
Observations and Data Collection
As part of measuring fidelity to the model, regular observations of evidence-based practices at the Tier 1 level are conducted by PBIS Autism coaches. A key component of the PBIS model is the development internal expertise and capacity by identifying and training coaches within the program on the targeted evidence-based practices, as well as how to conduct observations, collect data, and provide effective and acceptable performance feedback.
The PBIS Autism coaches use a standardized observation tool designed to measure implementation of the evidence-based practices targeted in Tier 1. Each practice has a direct measurement system with specified mastery criteria designed to rate fidelity of implementation of the evidence-based practice by staff. Observations are conducted in the natural environment with all the staff and children in the milieu. Coaches can complete the observation in about 30 minutes. Observations are expected to be completed at least twice per month during initial implementation. Once teams have demonstrated consistent implementation of the evidence-based practices, observations are faded to one time per month for maintenance (though observations would be increased back to two times monthly if performance data begins to illustrate a downward trend in this condition). Observation data is used to assess areas of need, identify goals, tailor coaching experiences and monitor outcomes.
Training the Workforce Using a Behavioral Skills Training Approach
The PBIS Autism approach emphasizes rigorous staff training and performance management components to ensure that staff members are using effective instructional practices consistently in the program. Past research has firmly established that providing workshop trainings to staff in isolation is not effective in ensuring that information disseminated in training is implemented in actual program settings (Reid, Parsons, & Jensen, 2017). Thus, Behavior Skills Training, which includes both training and coaching, serves as the cornerstone of the PBIS Autism model as opposed to relying on verbal or written responses to assess skill acquisition (e. g., passing a post-test quiz) following a workshop.
Behavioral Skills Training (BST) is an empirically supported approach to staff training that promotes consistent implementation of instructional skills learned in natural settings. A critical part of BST includes supplementing traditional training workshops with ongoing modeling of practices, staff observations, and the use of performance feedback (Parsons, Rollyson, & Reid, 2012). This cycle of training and support continues until staff are able to demonstrate targeted instructional skills in practice (Parsons et al., 2012).
Teacher providing positive feedback following a correct response
Two performance checklists were developed for the BST component of the PBIS Autism model to provide feedback to staff on the use of ABA instructional strategies (using reinforcement, prompting procedures, visual supports, lesson formats, and errorless learning).
Checklists are completed by PBIS Autism coaches who themselves have demonstrated competence in performance management and how to effectively deliver performance feedback (as outlined by Parsons et al., 2012).
BST is introduced at the beginning of program implementation on two different levels. First, as previously described, trained coaches collect data twice monthly in the program using a Tier 1 Fidelity Observation Tool. Following the observation, coaches: (a) provide immediate feedback to staff, (b) score the observation data and then report the data to the team using a consultation note (which also includes positive impressions and action plan items), (c) graph the data for the team as well as the PBIS Autism Leadership Team, and (d) meet with staff within the next week to review the consultation note, answer any questions or concerns, and develop action plans collaboratively based upon the data. In between observations, the coaches model targeted practices and strategies for staff in the program, as well as observe and coach staff members when they are working with the children.
At the second level, coaches use the performance checklists to increase the fidelity of direct care staff in using ABA instructional practices in the program. Each staff in the program is observed for a 10-minute period by a coach and the designated performance feedback checklist is completed. These observations continue until staff meet the performance criterion of correct implementation of the skill. At the end of each observation, the coach delivers performance feedback individually, and provides any further information and/or modeled the practices, either individually or to a larger group.
Outcomes From Pilot Implementation
After developing our model, we implemented a pilot on a residential campus serving children with autism beginning in December 2014. To evaluate the model, the leadership team collected data on staff performance related to implementation fidelity to our Tier 1 expectations, focusing on the percentage of Tier 1 performance goals met across 28 months. Baseline data indicated that the average monthly percentage of implementation goals met was assessed at 39%. Following the introduction of BST, this monthly average increased to 93% during the last three months of our outcome assessment (January through March 2017).
Teacher providing reinforcement for a correct response
The leadership team also investigated the impact of increased program implementation on rates of challenging responses (focusing on the nine children who resided in the program throughout the entire pilot). Specifically organizing data on aggression towards others, elopement attempts, and tantrums, the monthly average during the three-month baseline period was 31 instances per child. During the last three months of the pilot, these average monthly rates were reduced to three instances per child. Moreover, the monthly average rates of physical restraints applied with all nine children went from five restraints per month down to 0.3 during the last three months.
Last, the leadership team requested program staff to complete satisfaction surveys anonymously. Results of these satisfaction surveys indicated that staff rated the model favorably. Staff members found the coaching and performance feedback systems to be particularly helpful, and universally stated that they would recommend the model to other professionals.
These outcomes illustrate an association between increased implementation fidelity of our Tier 1 expectations with substantial improvements in child behaviors (both in frequency as well as intensity). To date, this approach continues to be implemented in the pilot site and sites have been added both internally and externally (in 15 different locations across seven different states). Next steps include additional implementation reviews of the PBIS Autism model, as well as exploration of the possibility of delivering the model through telehealth technology.
Todd Harris, PhD, is Executive Director of Autism Services at Devereux Advanced Behavioral Health. Megan Robinson Joy, PhD, BCBA, is State Autism Director at Devereux Advanced Behavioral Health Pennsylvania. Richard G. Allen, PsyD, BCBA-D, NCSP, is job title at Devereux Advanced Behavioral Health and job title at the Philadelphia College of Osteopathic Medicine. Carol Anne McNellis, PsyD, BCBA, is Executive Director at Devereux Children’s Intellectual and Developmental Disabilities Services.
For any additional information about the content of this article, please feel free to contact Dr. Todd Harris at todd.harris@devereux.org or 610-873-4934 or visit www.devereux.org.
References
Dunlap, K., Goodman, S., McEvoy, C., & Paris, F. (2010). School-wide positive behavioral interventions & supports: Implementation guide. Michigan Department of Education, 1-54.
Fernandez, M. A, & McClain, D. (2014, July). Georgia’s juvenile justice system applies new framework to modify youth behavior trends. Corrections Today, 76, 18-19.
Jolivette, K., Patterson, D. P., Swoszowski, N. C., McDaniel, S. C., Kennedy, C., & Ennis, R. P. (2014). SWPBIS in a residential school for students with emotional and behavioral disorders: First years of implementation and maintenance follow-up focus group. Residential Treatment of Children and Youth, 31, 63-79
The National Professional Development Center on Autism Spectrum Disorder. (n. d.). Retrieved from http://autismpdc.fpg.unc.edu/national-professional-development-center-autism-spectrum-disorder
Parsons, M.B., Rollyson, J.H., & Reid, D.H. (2012). Evidence-based staff training: A guide for practitioners. Behavior Analysis in Practice, 5(2), 2-11.
Reid, D.H., Parsons, M.B., & Jensen, J.M. (2017). Maintaining staff performance following a training intervention: Suggestions from a 30-year case example. Behavior Analysis in Practice, 10(1), 12-21.
Rue, H., Knox, M., Welchons, L.W., Murzycki, A., Pollack, E. G., and Class, E.G., et al. (2015). Findings and conclusions: The national standards project: addressing the need for evidence-based practice guidelines for autism spectrum disorders, Phase 2. Randolph, MA: The National Autism Center
