|
William R. Shadish ACADEMIC 2. Corrections to Errors in Shadish, Cook & Campbell (2002) 7. Graduate Studies in Psychology at UC Merced
|
|
|
Annotated Bibliography
Allison, D. B., Faith M. S., & Franklin R. D. (1995). Antecedent exercise in the treatment of disruptive behavior: A meta-analytic review. Clinical Psychology: Science and Practice. 2, 279 – 303.
The 1980’s provided research proclaiming that exercise is a beneficial supplement to psychological therapy for those with psychological afflictions. This study turns its attention to children with disruptive behavior and the potential to use exercise in these cases. The meta-analysis used 42 different studies (16 of which were group studies, 26 of which were SSD). Exercise was defined as some increased level of physical exertion excluding sport since it does not necessarily imply exertion. The articles had to include an independent variable that was clearly some form of physical exercise applied without exception, a dependent variable that was some measure of undesired behavior, and some objective data collected on the disruptive behavior. The methods for meta-analysis employed a multiple regression approach, in which the effects of treatment on both the level and slope of behavior are estimated after controlling for any baseline trend. The end result is an r-value, which can be broadly interpreted as the partial correlation between treatment and outcome after controlling for trend and can then be converted to a d statistic (Allison, Faith, & Franklin, 1995). The effect size of this study was .33 with a standard deviation of .08.
Baron, A., & Derenne, A. (2000). Quantitative summaries of single-subject studies: What do group comparisons tell us about individual performances? The Behavior Analyst. 23, 101 – 106.
In this article, Baron and Derenne are criticizing a quantitative summary completed by Kollins, Newland, and Critchfield in which they found unexpected results that they believe should be a springboard for new research. Baron and Derenne argue that while they agree that empirical research should result from the work of Kollins and his colleagues, it should not further the idea of quantitative summaries but call into question the procedure that brought about different findings. The authors conclude by stating their indifference to these summaries, as they aren’t worthy of discussion in the arena of outcomes.
Beto, J. A., & Bansal, V K. (1992). Quality of life in treatment of hypertension. American Journal of Hypertension. 5, 125 – 133.
Patients who suffer from hypertension do not normally find themselves subject to symptoms; however, the lifelong therapy that is necessary for most patients results in many side effects that can affect the patient’s quality of life. This research assessed 9 articles that were screened twice for their relevance to the subject at hand. First, they were screened for the selection criteria of blinded, randomized clinical trials with a baseline comparison with treatment, patients as their own controls, and assessment of quality of life by self-administered evaluation, interviewer-assisted self-evaluation, sleep laboratory observations, or standardized psychomotor or cognitive tests. The second screening was for study design, population demographics, control and treatment conditions, and quality of life construct. The methods for meta-analysis were those used by Hedges and Olkin that requires one to calculates an effect size (d) for each study by dividing the mean change between baseline and treatment by the average of the standard deviation (Beto & Bansal, 1992). Little to no positive effects were shown by the meta-analysis.
Campbell, J. M. (2003). Efficacy of behavioral interventions for reducing problem behavior in persons with autism: A quantitative synthesis of single-subject research. Research in Developmental Disabilities. 24, 120 – 138.
Children growing up with autism endure extreme difficulty assimilating into mainstream society due to obvious disabilities, however, when they begin taking part in particularly disruptive behavior such as self-injury behavior, stereotypic behavior, aggression, and property destruction they begin to interfere with their own ability to learn and play. This experiment examined 117 representing 181 individual subjects. The articles were published between 1966 – 1998 and were included for having patients diagnosed with autistic disorder, reported the results of single-case studies, single-case studies that had baseline and treatment phases that were present in the study with repeated data points, and treatment targeting reduction of said behaviors. This research incorporated three different methods for meta-analysis; a MBLR, PND, and PZD statistic was used for each patient in the study. The mean effect size for the MBLR was 76.47 with a SD of 28.56, the mean effects size for the PND was 84.40 with a SD of 23.80, and the mean effect size for the PZD was 42.86 with a SD of 35.59.
Carr, E. G., Horner, R. H. Turnbull, A. P, Marquis, J. G., McLaughlin, D. M., McAtee, M. L, et al. (1999). Positive behavior support for people with developmental disabilities: A research synthesis.
The authors of this book reviewed 107 articles discussing the effectiveness of positive behavior support for students who exhibit problem behavior. They chose to assess the articles published between 1985 and 1996 in terms of four categories: demographics, assessment practices, intervention strategies, and outcomes. The authors had five conclusions. 1: Problem behavior support is widely applicable to people with serious problem behavior. 2: The field is growing rapidly overall, but especially in the use of assessment and in interventions that focus on correcting environmental deficiencies. 3: Using stringent criteria of success, PBS is effective in reducing problem behavior in one-half to two-thirds of cases. 4: Success rates nearly double when intervention is based on a prior functional assessment. 5: Consumer needs that emphasize comprehensive lifestyle support, long-term change, practicality and relevance, and direct support for consumers themselves are inadequately addressed by the research base (Carr, et al., 1999).
Critchfield, T. S., Newland, M. C., & Killins, S. H. (2000). The good, the bad, and aggregate. The Behavior Analyst. 23, 107 – 115.
The purpose of this article is to defend the process of generalizing data from different types of research known as literature review. Previous articles, such as that written by Baron & Derenne (2000), argue that these methods are not valid as they use data from dissimilar research. Critchfield et al. argue that generalities cannot be made without also varying how issues are studied. Several examples of such reviews are listed in hopes of defending their claim.
DuPaul, G. J., & Eckert, T. L. (1997). The effects of school-based interventions for attention deficit hyperactivity disorder: A meta-analysis. School Psychology Review. 26, 5 – 27.
According to previous research, children and adolescents with attention deficit hyperactivity disorder are at high risk relative to the normal population with respect to scholastic and social failure in school settings (DePaul & Eckert, 1997). As a result, schools are in need of treatments that produce enhanced performance. This study used 63 studies (x between-subject, x within-subject, and x single-subject), and although there were different designs used, each sub-set was analyzed separately. In order to be included in the study, the research had to have patients diagnosed with ADD, ADHD, or hyperactive disorders, and be set in a school setting. The method of meta-analysis used with the single-subject research was the method developed by Busk & Serlin (1992). This method assumes a normal distribution and equality of variances and intercorrelations across baseline and treatment phases (DuPaul & Eckert, 1997). The effect size was 1.16 with a median of .80.
DuPaul, G. J., & Eckert, T. L. (1998). Academic interventions for students with attention- deficit/hyperactivity disorder: A review of the literature. Reading and Writing Quarterly: Overcoming Learning Difficulties. 14, 59 – 82.
Students who suffer with attention-deficit hyperactivity disorder (ADHD) have difficulty succeeding in scholastic arenas. In response to their disabilities, many researchers have attempted to develop interventions that would enhance their academic performance. This literature-review focuses on finding out which of these interventions work. The authors reviewed 16 articles and came to the conclusion that none of the interventions mentioned made significant changes to the academic performance of the individuals but did improve their behavior.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Council for Exceptional Children. 71, 165 – 179.
The authors illustrate single-subject research as a key to the future of education due to the recent trends of empirically founded methods. They discuss the principles of single-subject research such as the independent and dependent variables, the fact that the individual participant is the unit of analysis, and the use of baseline conditions.
Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist. 48, 1181 – 1209.
The purpose of this paper is to examine a number of meta-analyses that have analyzed treatment experiments. The authors state that meta-analysis has given stark, dramatic patterns of evidence in terms of the effectiveness of treatments (Lipsey & Wilson, 1993). They discuss the benefits of meta-analysis of previous methods of synthesizing data. The authors admit that meta-analytic methods have their limitations, however, they are capable of providing significant statistics that can determine the effectiveness of treatments in the educational and psychological fields. They conclude from their review of meta-analytic evidence that well-developed psychological, educational, and behavioral treatments generally have meaningful positive effects on the intended outcome variables (Lipsey & Wilson, 1993).
Mastropieri, M. A., & Scruggs, T. E. (1985). Early intervention for socially withdrawn children. The Journal of Special Education. 19, 429 – 441.
While all methods of intervention are of interest to special educators, they pay extra close attention to the results of early intervention research. This meta-analysis used 18 research articles (all of SSD) that had subjects whom were described as handicapped or as socially withdrawn, whose subjects were 66 months of age or less at the onset of the intervention, the study assessed the efficacy of specific behavioral interventions as opposed to the effectiveness of behavioral principles, and that data were presented in a typical graphic display. The studies were systematically analyzed for a number of variables, including description of target subjects, type of intervention, length and intensity of intervention, primary intervener, and several outcome variables, including percent of nonoverlapping data points from baseline to intervention phases and an overall outcome rating based on a 3-point scale (Mastropieri & Scruggs, 1985). The findings of this report conclude that the social interaction effects were stronger when target subjects were directly reinforced for interaction; although target subjects did spontaneously respond to peer confederate initiations, they were less likely to initiate interactions with peer confederates; peer confederates were more likely to interact with target subjects when they were reinforced; and generalization was infrequently assessed and was only effective when assessed immediately after the intervention in the same setting.
Mathur, S. R., Kavale, K. A., Quinn, M. M., Forness, S. R., & Rutherford, R. B. (1998). Social skills interventions with students with emotional and behavioral problems: A quantitative synthesis of single-subject research. Behavioral Disorders. 23, 193 – 201.
Students who suffer from emotional or behavior disorders often suffer from a lack of social skills as well. This may lead them to experience peer rejection, social withdrawal, isolation, loneliness, and problems in establishing and maintaining positive social relationships (Mathur et al., 1998). This meta-analysis used 64 single subject studies that tests interventions that aim to improve their social skills. These studies all had participants who were described as having behavioral problems, emotional/behavioral disorders, serious emotional disturbance, disruptive behavior, social behavior problems, conduct disorders, autism, or delinquency; one or more components of social skills instructional procedures; a valid single-subject research design; and a direct measure of over behavior change. The method of meta-analysis incorporated into this study was the percentage of nonoverlapping data method (PND). The mean PND was 62% with a standard deviation of 33%, which represents mild treatment effectiveness (Mathur et al., 1998).
Maughan, D. R., Christiansen, E. Jenson, W. R., Olympia, D., & Clark, E. (2005). Behavioral parent training as a treatment for externalizing behaviors and disruptive behavior disorder: A meta-analysis. School Psychology Review. 34, 267 – 286.
Children who suffer from ADD, ADHD, and other disabilities resulting in disruptive behavior have difficulty developing proper social skills and effect their schools, teachers, and parents negatively. These children are usually from families characterized by considerable stress, changes in family structure, and inconsistent and highly punitive disciplinary practices (Maughen et al., 2005). Thus, the purpose of this meta-analysis is to test the effectiveness of interventions that involve parent training. The authors used 79 studies (10 between subject, 22 within subject, and 47 single-subject) that had children who exhibited at least one target behavior such as aggression, treatment procedure that included the training of parents in the use of reinforcement and/or time-out and on additional behavioral procedure, a sample including parents of children between the ages of 3 and 16, at least one outcome measure, and single-subject design studies needed to provide a graph displaying raw data. The methods for meta-analysis used for the single-subject research were ITSACORR and Busk and Serlin’s No Assumptions method. The effect size for the ITSACORR method was .59 and .54 when weighed. The effect size for the No Assumptions method was 2.02 and 1.56 when weighed.
Scotti, J. R., Evans, I. M., Meyer, L. H., & Walker, P. (1991). A meta-analysis of intervention research with problem behavior: Treatment validity and standards of practice. American Journal on Mental Retardation. 96, 233 – 256.
Problem behaviors, though present throughout society, are more commonly found in people who suffer from developmental disabilities such as mental retardation. These behaviors often times leaves them struggling to assimilate into mainstream society. This meta-analysis evaluates the practices and the treatment outcomes in studies designed to remedial behavior problems (Scotti et al., 1991). The authors used 403 studies (AB designs 11%, ABAB designs 31%, multiple baselines 40%, ABACAD designs 33%, and multi-element/alternating treatment procedure 6%) that were chosen based on the use of patients who were suffering from some developmental disorder and the presence of a graph that could be associated with methods to be described later. The researchers used two types of effect size analysis: the percentage of nonoverlapping data and the percentage of zero data. While their methods of calculating PND were not disclosed, their methods for calculating PZD was to determine the first data point in an intervention phase to reach zero, and then computing the percentage of data points from the first zero point onwards remaining at zero in that phase.
Scruggs, T. E., Mastropieri, M. A., Cook, S. B., & Escobar, C. (1986). Early intervention for children with conduct disorders: A quantitative synthesis of single-subject research. Behavioral Disorders. 11, 260 – 271.
In order to make adaptation to a school atmosphere an easier transition, several researchers have developed interventions for preschool children suffering from conduct disorders. The purpose of this meta-analysis is to test the efficacy of those interventions. The authors used 16 studies (62% ABAB design, 19% multiple baseline design, and 19% AB or variation designs) that focused on conduct disorders in preschoolers and displayed individual subjects in a time series format (Scruggs et al., 1986). The authors used a percentage of nonoverlapping data (PND) formula for calculating the effects sizes of the different studies. The median PND score was 79 which shows that these interventions are generally successful.
Scruggs, T. E., Mastropieri, M. A., Forness, S. R., & Kavale, K. A. (1988). Early language intervention: A quantitative synthesis of single-subject research. Journal of Special Education. 22, 259 – 283.
One early sign of an individual who will later be categorized as handicapped once entering school is problems with language. Should those issues with language continue into their school-age years, they will have difficulty succeeding in school (Scruggs et al., 1988). This quantitative analysis aims to tests the efficacy of interventions for children suffering from early language problems. The authors used 20 studies that used early interventions, represented their data graphically, and used it on preschool age children suffering from early language deficits. The method of calculating the effect sizes was the percentage of nonoverlapping data (PND) formula. The article reports low PND scores that suggest that these interventions are not very successful.
Shogren, K. A., Fagella-Luby, M. N., Bae, S. J., & Wehmeyer, M. L. (2004). The effect of choice-making as an intervention for a problem behavior: A meta-analysis. Journal of Positive Behavior Interventions. 6, 228 – 237.
Previous studies have suggested that choice making, among other things, is a positive behavior supporter that ought to be used to help extinguish problem behaviors in people suffering from developmental disabilities. This meta-analysis tests the efficacy of those interventions. The authors used 13 studies (12 ABAB studies, 1 multiple baseline study) that were chosen based on the recipient of the intervention had an identified disability, a choice intervention was implemented to reduce problem behavior, occurrences of problem behavior were measure as a dependent variable, and the effect of the intervention on problem behaviors was reported graphically (Shogren et al., 2004). The method of the meta-analysis was the percent of nonoverlapping data (PND) formula. The average PND score was 65.7% with a standard deviation of 41, thus showing a statistically significant sign of success for these interventions.
Stage, S., & Quiroz, D. (1997). A meta-analysis of interventions to decrease disruptive classroom behavior in public education settings. School Psychology Review. 26, 333 – 368.
Previous studies show that disruptive behavior in class leads to poor grades, lack of interest in school, and poor results on standardized tests. This meta-analysis set out to find out which school-based interventions successfully inhibit disruptive behavior in the classroom. The study incorporated 99 studies (48 of which were SSD) that provided an empirical measure of disruptive classroom behavior, settings in public schools, adequate statistical information, and a control group or baseline phase. The method for the meta-analysis was ITSACORR. ITSACORR produces an overall F-test that uses an estimated adjustment to control for autocorrelation and yields a t-test for the change in the y-intercept between the baseline and treatment conditions as well as a t-test for the change in slope between the conditions (Stage & Quiroz, 1997). The effect size of this study was -.78 with a SD of .58.
Swanson, H. L., & Sachse-Lee,C. (2000). A meta-analysis of single-subject-design intervention research for students with LD. Journal of Learning Disabilities. 33, 114 – 136.
This article reports that although there have been a great number of single-subject design intervention research for students with learning disability, no one has put together a comprehensive meta-analysis of the gathered data (Swanson & Sachse-Lee, 2000). The purpose of this meta-analysis is to do so. The analysis used 85 studies chosen based on their having used a single-subject design, provided sufficient clarity of information, participants identified has having learning disabilities, provided help for the participants when they weren’t familiar with material, and was written in English. The researchers used the Rosenthal (1994) formula for the calculation of their effect sizes. Effect sizes were calculated for the three treatment phases in order to compare actual progress. The mean effect size was .87 with a standard deviation of .32 showing a general success for these interventions.
Wehmeyer, M. L. (1995). Intra-individual factors influencing efficacy of interventions for stereotyped behaviours: A meta-analysis. Journal of Intellectual Disability Research. 39, 205 – 214.
Stereotypical behaviors of the mentally retarded inhibit other more developmentally or educationally significant behaviors (Wehmeyer, 1995). Many researchers have attempted to develop interventions that would help to eliminate those repetitious motor responses in order to help those individuals avoid stigmatization and better adapt to living in normal society. The purpose of this meta-analysis is to test the efficacy of these interventions. The analysis encompasses 33 studies that were chosen because they were testing a program for the elimination of stereotyped behaviors in developmentally disabled individuals. The researchers used non-parametric procedures as described in Scruggs et al. (1987), and used the Kolmogorov-Smirnov Two-Sample test for each coded factor. A percentage of nonoverlapping data (PND) and a percentage of zero data (PZD) were calculated for each individual. The results show that an overwhelming number of the interventions work as the average PND score was 81%.
Weinstein, K. S., Bray, M. A. & Kehle, T. J. (2004). [Review of Swanson et al. (1999). Interventions for students with learning disabilities: A meta-analysis of treatment outcomes.] Psychology in the Schools. 41, 273 – 274.
This article is a review of the text Interventions for Students with Learning Disabilities: A Meta-Analysis of Treatment Outcomes written by Swanson, Hoskyn, and Lee (1999). The article gives a quick overview of the text and claims that the text is more comprehensive than previous meta-analytical research in the field, and that it is written for grad students and experts in the field interested in learning disabilities (Weinstein et al., 2004). The writers’ final critique is that the text is an excellent and needed comprehensive study of the effectiveness of interventions for students with learning disabilities (Weinstein et al., 2004).
Xin, Y. P., Grasso, E., Dipipi-Hoy, C. M., & Jitendra, A. (2005). The effects of purchasing skill instruction for individuals with developmental disabilities: A meta-analysis. Exceptional Children. 71, 379 – 400.
Advancements in the teaching of people with developmental disabilities has allowed for them to lead better lives without dependence on constant care. These individuals are now capable of integrating themselves into society as jobs have been provided and offered to them, however, this has raised issues in regards to their ability to handle money. This meta-analysis focused on intervention research testing the overall effects of purchasing, money skills instruction, and various instructional strategies. They used 28 articles based on five inclusion criteria, which were that they had to include an intervention to teach purchasing skills, recipients of the intervention were identified as individuals with developmental disabilities, studies had to be single-subject or group research design studies, studies had to provide baseline and treatment data or pretest and posttest data that allowed for the computation of effect sizes, and only studies published in English were included. The authors used the PND (percentage of nonoverlapping data) method for the meta-analysis. Results showed a positive effect for purchasing skill instruction with a median PND of 87%.
|