The theme of this issue of Autism Spectrum News is “Advances in Autism Science.” One facet of autism science is the adherence to “evidenced-based practice” (EBP); a requirement that those who work with persons with autism – teachers, psychologists, speech therapists, etc. – use strategies and tactics that have been “experimentally” tested, thoroughly researched, and found to have an empirically-demonstrated improvement in some aspect of the autism condition. Most of the major organizations that promote the treatment of autism support this requirement (e.g., Association for Science in Autism; The Autism Society; American Academy of Pediatrics).
The importance of using evidenced-based practice is supreme. Autism treatment has been termed a “fad magnet (e.g., Jacobson, Foxx, & Mulick, 2005) due to the numerous treatments that are used that have no empirical support of their effectiveness (e.g., Facilitated Communication). Service providers are required to use treatments that have substantial evidence of effectiveness so that there is a strong likelihood of improving language, skills, and behaviors of persons with autism. Treatments for which there is little or no evidence of effectiveness will likely have little to no effect on the person being served, resulting in a waste of time, money, and emotional investment (Zane, Davis, & Rosswurm, 2009).
What constitutes “evidence” of effectiveness? Many professional organizations (e.g., American Psychological Association, United States Department of Education, Coalition for Evidenced-Based Policy, and the Council for Exceptional Children) have published sets of criteria that establishes levels or goals that must be reached to consider a treatment as having evidence of effectiveness. Some of these criteria include well-established scientific principles, such as reliable and valid measurement of a dependent variable; clear identification of the independent variable or treatment; use of a commonly accepted research design (such as the traditional experimental-control group design, or within-subject designs such as reversals or multiple baselines), and careful screening and selection of subjects.
Morris (2009) described Relationship Development Intervention (RDI) as a treatment for autism developed by Dr. Steven Gutstein from Texas. According to Gutstein, RDI has a focus of increasing social awareness through the use of dynamic intelligence (Morris, 2009). RDI methods are employed by the parents of the children in RDI therapy, since the general goal of the treatment is more natural and complete interactions among family members. The developer, Gutstein, calls RDI a “cognitive-developmental” parent training program. The program attempts to impact “experience sharing” and inflexibility in thinking (e.g., Gutstein, 2001). Morris and others (e.g., Gutstein & Sheely, 2002) have outlined the types of methods and goals built into RDI, including dynamic analysis, flexible problem solving, and resilience. As of September, 2009, there are currently over 200 certified RDI therapists. This number has steadily increased over the past several years, suggesting an increasing popularity of this treatment.
However, an important question is whether or not RDI can presently be considered an empirically validated treatment, one that meets the standards for treatments that have proven effective in improving some aspect of autism. Thus, a literature review was conducted using the keywords “Relationship Development Intervention,” Gutstein,” “RDI,” and “social development.” Any published article found related to RDI was assessed as whether or not it met the criterion of a “research” study, such as a test of whether or not RDI intervention caused the change in a group of participants, or possibly a comparison of RDI to other interventions in autism or education.
This search found only one published article that attempted to evaluate the effectiveness of RDI (Gutstein, Burgess, & Montfort, 2007). The purpose of this study was to determine whether children who participated in RDI treatment improved in selected measures related to autism. The authors reviewed the files of 16 children who ranged in ages between 20-96 months, representing various diagnoses of autism (e.g., Asperger, Pervasive Development Disorder – Not Otherwise Specified (PDD-NOS), and Autism). These children had been receiving treatment based on the RDI model for at least 30 months. The authors attempted to measure three variables to assess whether or not the children improved due to the RDI intervention: (1) a subset of 13 items from the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, Dilavore, & Risi, 2002) and the Autism Diagnostic Interview-Revised (ADI-R); (2) the results of a “flexibility interview”, developed by Gutstein, et al., in which parents used a self-report Liekert scale to rate the degree to which they thought their children exhibited “rigidity” in their behavioral adaptation and thinking; and (3) educational placement of the participants, which involved parents and teachers subjectively reporting the type of school placement in which the children resided (ranging from mainstreaming with no special education services to full-time placement with special education support).
Gustein and colleagues collected data on these measurements prior to and following the children’s participation in RDI for an average of 18 months. Following treatment, Gutstein, et al. reported: (a) improvement in ADOS diagnosis, (b) improvement in “age appropriate flexibility” to routines, and (c) more children participating in less restrictive, more mainstreamed educational placements. The authors concluded that RDI was a “promising program for remediating critical experience-sharing difficulties…” of children with autism (p. 409). They hypothesized that the RDI treatment was causally related to the positive changes in the children; that is, that RDI appeared to be responsible for the improvement.
For a treatment to be considered within the category of EBP, there is general agreement that there must exist “multiple” research studies, done by a variety of researchers, that show that treatment to be effective (e.g., Chambless, Baker, Baucom, Beutler, Calhoun, Crits-Christoph, et al., 1998). With only one study published on testing whether RDI results in any improvement in participants with autism, it seems that RDI does not pass this initial criterion. Furthermore, it is unclear whether or not the positive conclusions made by Gutstein and colleagues concerning RDI in this current study are actually warranted. In other words, the methodological rigor of the Gutstein, et al. (2007) study was examined to determine whether the research design met minimal criteria for quality and, thus, believability of the authors’ conclusions.
Upon careful examination of the design and methodology of the Gutstein, et al. (2007) study, it seems as if there are methodological problems with this study that prevent confidence in the conclusions offered by the authors. For example, the research design used in this study involved one group of participants, with measurements taken prior to and after the RDI intervention. This type of design is a “one group pretest-postest design” (e.g., Fraenkel & Wallen, 2009; Gay, Mills, & Airasian, 2009). It is important to note that this type of design is universally considered “weak” in that it does not control for threats to internal validity. This research design offers unconvincing evidence that the treatment was the sole reason for changes in the dependent measures (e.g., Fraenkel & Wallen 2009). Thus, there is an assumption that the participants in the Gutstein, et al. study could have improved on the measures due to reasons unrelated to RDI (such as maturation).
There are two other issues related to the research design that prevents clearly assuming that RDI was responsible for improvement in the participants. First, an important criterion for a well-designed study is proof of treatment implementation (i.e., procedural integrity; Gresham, Beebe-Frankenberger, & MacMillan, 1999). Gutstein and colleagues not only failed to provide detailed information about what exactly the RDI treatment protocols were that were employed, but they failed to provide any check on whether or not the treatment providers actually implemented the RDI strategies as Gustein, et al. intended. Thus, this study fails to meet this particular research quality criterion. A second essential criterion for “believability” of research is that of measurement reliability (e.g, Gay, et al. 2009). Specifically, researchers are required to provide evidence to support the belief that the dependent variables measured in the study were measured reliably. This is often accomplished by having a second independent observer measure the participants at the same time (and then comparing results), or by demonstrating that standardized instruments have pre-determined reliability and validity. In the current RDI study, of the four dependent variables, the authors mentioned that inter-rater reliability was obtained (successfully) with one measure (ADOS), and that the ADI-R developers reported satisfactory reliability. However, the other two dependent variables (flexibility and educational placement) had no reliability measurements reported. In addition, since only a subset of items of the ADOS and ADI-R were measured, the validity of these two assessments was compromised, since the initial strong validity of these assessment tools is based on the tests in their totality.
Conclusion
Due to the weak research methodology used by Gutstein, et al. (2007), the lack of fundamental research methodology, and the existence of only one formal assessment of the effectiveness of this autism treatment, RDI at this time should not be considered to be a treatment that has evidence of effectiveness. There is no existing research base for concluding that RDI has been proven to be effective. Thus, like with other fad therapies and treatments that have no valid effectiveness data, care providers should carefully consider whether RDI is appropriate to use. Researchers must begin to do well-designed research studies attempting to simply determine if RDI is causally related to any improvement of any measure related to autism. Hopefully such studies will be done to determine if RDI is effective. However, until that time, treatment providers and other caregivers would be advised to consider using other treatments that have a proven record of effectiveness (e.g., applied behavior analysis).
Dr. Thomas Zane is an Associate Professor in the School of Education and the Founder and Director of the Center for Applied Behavior Analysis at The Sage Colleges. Dr. Zane earned his Bachelor’s and Master’s degree in psychology at Western Michigan University and his doctorate in Applied Behavior Analysis at West Virginia University. He is a licensed psychologist in New York and Massachusetts. Dr. Zane has published in various journals and books, presented at regional, national, and international conferences, and been an invited lecturer in Ireland and the Republic of China. He is the Director of the Center for Applied Behavior Analysis at The Sage Colleges, and offers a Master’s of Science Degree in Applied Behavior Analysis and Autism, a distance-learning graduate program.
References
Fraenkel, J. R., & Wallen, N. E. (2009). How to design and evaluate research in education, Seventh Edition. New York: McGraw-Hill.
Gay, L. R., Mills, G. E., & Airasian, P. (2009). Educational research: Competencies for analysis and applications, Ninth Edition. Upper Saddle River, NJ: Pearson
Gresham, F. M., Beebe-Frankenberger, M. E., & MacMllan, D. L. (1999). A selective review of treatments for children with autism: Description and methodological considerations. School Psychology Review, 28(4), 559-575.
Gutstein, S. (2001). Solving the relationship puzzle. Arlington, TX: Future Horizons.
Gutstein, S., & Sheely, R. (2002). Relationship development intervention with young children: Social and emotional development activities for asperger syndrome, autism, PDD, and NLD. London: Jessica Kingsley.
Jacobson, J. W., Foxx, R. M., & Mulick, J. A. (2005). Controversial therapies for developmental disabilities: Fad, fashion, and science in professional practice. Mahwah, NJ: Lawrence Erlbaum Associates.
Lord, C., Rutter, M., Dilavore, P., & Risi, S. (2002). Autism Diagnostic Observation Schedule. Los Angeles, CA: Western Psychological Services.
Morris, R. (2009). Relationship development intervention. Autism Spectrum News, Fall 2009.
Zane, T., Davis, C., & Rosswurm, M. (2009). The cost of fad treatments in autism. Journal of Early and Intensive Behavior Intervention, 5(2), 44-51.
My family benefitted immensely from RDI. My family is a testament to it. ABA and all other treatments turn a child into a robot.
The fact that RDI is dynamic and fluid is the reason you are unable to bring it into clear metrics and that is why science has failed in promoting such a beautiful help and robbed children of good lives. Please recognize that.