To the Graduate Council: I am submitting herewith a dissertation written by Carl William Chaney entitled “An Investigation of the Relationships between Accelerated Reader® and Other Factors and Value-Added Achievement in Tennessee Public Schools.” I have examined the final paper copy of this dissertation for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Doctor of Education, with a major in Educational Administration and Policy Studies. _________________Dan R. Quarles Major Professor We have read this dissertation and recommend its acceptance: C. Glennon Rowell____________________ Mary Jane Connelly___________________ Thomas W. George____________________ Acceptance for the Council: ____________________Dr. Anne Mayhew Vice Provost and Dean of Graduate Studies doncahr doncahr (Original signatures are on file in the Graduate Student Services Office.) An Investigation of the Relationships between Accelerated Reader® and Other Factors and Value-Added Achievement in Tennessee Public Schools A Dissertation Presented for the Doctor of Education Degree The University of Tennessee, Knoxville Carl William (Bill) Chaney May 2002 ii Copyright © 2002 by Carl William Chaney All rights reserved. iii Dedication This dissertation is dedicated to my mother, Dr. Virginia Miles Chaney, who taught school for 66 years without interruption, who always hoped that I would earn a terminal degree, and who never wavered in her encouragement of my intellectual pursuits. iv Acknowledgments I wish to thank the following persons and all others who greatly assisted me in the preparation of this dissertation. Together they provided the encouragement and gentle prodding that sustains efforts of this kind. • Dr. Sharon Henderson Chaney, my wife, whose continuous and loving encouragement compelled me to complete the work. • Dr. Dan R. Quarles, my major professor, who guided my work at each juncture, and my other committee members: Dr. Mary Jane Connelly, Dr. Thomas George, and Dr. Glennon Rowell, who graciously offered their suggestions and perspective. • The Davidson Academy Board of Trustees, my patrons in the pursuit of the doctorate. • Dr. William Sanders, who developed the value-added achievement model, and whose insight and encouragement gave impetus to the study. • Dr. June Rivers Sanders, classmate and friend, who acquainted me with the value- added concept and introduced me to Dr. William Sanders. • Dr. Gary Ubben, who taught me in several classes and posed good questions to guide my thinking. • Laurie Borkon and Eileen Hannigan of The School Renaissance Institute, who kindly assisted me in getting data on implementation of Accelerated Reader in Tennessee schools, and helped me to frame the study. • Beverly Cherry, Darlyne Kent, Stacy Lovell, and Don Winn of Davidson Academy, who helped with data input and organization and background on the use of Accelerated Reader. • Thesis/Dissertation Consultant Heather Doncaster, who helped me format this document for electronic submission. v Abstract This study investigated the relationships between value-added achievement in Tennessee public schools that include grades one through five and selected independent variables. The schools’ use of the reading practice and monitoring software known as Accelerated Reader® (AR) was of particular interest, as considerable research has suggested its effectiveness in raising achievement in reading and other subjects. Data were (1) the dependent variables, cumulative three-year average (1999, 2000, and 2001) Tennessee Value-Added Assessment scores in reading, language, math, science, and social studies; (2) independent variables school enrollment, per pupil annual expenditure for the system, percentage of students in the school eligible for free or reduced price meals, and percentage of minority students in the school; and (3) whether and to what extent AR had been purchased and implemented at each school since August 1, 1999. Four levels of AR ownership/implementation were classified as (1) ownership without any “model classrooms,” (2) having one or two model classrooms, (3) having three or more model classrooms, or (4) being certified as a “model school.” Multiple regression analysis was used to search for statistically significant relationships between the independent demographic variables and AR use and the dependent variables of value-added achievement at the .05 level of significance, in the hope that a useful model could be designed for predicting value-added achievement from AR use, school enrollment, per pupil expenditure, free or reduced-price meal eligibility, and minority enrollment. Analysis of the data uncovered almost no significant relationships or school-level effects. In no instance was AR implementation a significant factor in relation to value- added achievement at the school level. While no useful regression model was developed from this study, one significant finding was that, in schools ending at grade six, school enrollment and especially minority enrollment are negatively correlated with math achievement. vi Table of Contents Chapter 1 INTRODUCTION .......................................................................................1 Background ..........................................................................................................1 Accelerated Reader ..................................................................................................3 The Tennessee Value-Added Assessment System (TVAAS) .................................5 Purpose of the Study ................................................................................................7 The Research Question ............................................................................................8 Significance of the Study .........................................................................................8 Limitations and Delimitations..................................................................................9 Assumptions ..........................................................................................................9 Definition of Terms................................................................................................10 Summary of Chapter 1 ...........................................................................................11 Organization of the Study ......................................................................................12 Chapter 2 REVIEW OF RELATED LITERATURE.................................................14 Introduction ........................................................................................................14 Institute Research...................................................................................................14 Dissertations and Theses........................................................................................17 Journal Articles, Reports, and Papers ....................................................................20 Field Reports from Schools and Districts ..............................................................23 Summary of Chapter 2 ...........................................................................................23 Chapter 3 METHODOLOGY ....................................................................................27 Introduction ........................................................................................................27 vii Table of Contents (continued) Subjects ........................................................................................................27 Data Collection ......................................................................................................27 Methods and Procedures ........................................................................................29 Chapter 4 FINDINGS AND ANALYSIS OF DATA ................................................31 Introduction ........................................................................................................31 Data Analysis and Findings ...................................................................................34 Summary and Analysis of Schools Ending at Grade Five.....................................43 Schools Ending at Grade Six .................................................................................47 Summary – Schools Ending at Grade Six..............................................................53 Schools that include Grades One through Eight ....................................................53 Summary – Schools Including Grades One through Eight....................................55 Analyses of All Schools Combined (Regardless of Ending Grade Level) ............55 Summary – All Tennessee Public Schools Including Grades 1-5 .........................62 Differences in Mean Value-Added Achievement Scores by AR Level ................62 Summary of Findings and Analysis of Data ..........................................................72 Chapter 5 SUMMARY OF THE STUDY, CONCLUSIONS, AND RECOMMENDATIONS FOR FURTHER RESEARCH.................................................75 Summary of the Study ...........................................................................................75 Summary Data on the Schools in the Study...........................................................77 Tennessee Schools and AR....................................................................................78 viii Table of Contents (continued) Multiple regression analyses – schools including grades one through five and ending at grade five ........................................................................................................79 Multiple regression analyses – schools including grades one through six and ending at grade six ........................................................................................................80 Multiple regression analyses – schools including grades one through eight .........81 Multiple Regressions – All Schools Combined .....................................................82 ANOVA’s – All Schools Combined......................................................................83 Conclusions ........................................................................................................84 Recommendations for Further Research................................................................87 References ........................................................................................................88 Appendices ........................................................................................................94 Appendix A: Reading Renaissance® Model Classroom Checklist .......................95 Appendix B: The Renaissance Certification Program at a Glance ......................100 Vita ......................................................................................................102 ix List of Tables Table 1: Classification of Schools by Level of AR Implementation ...................................6 Table 2: Field Reports from Schools and Districts............................................................24 Table 3: Schools in the Study by Grade Structure and Grades Tested ..............................28 Table 4: Descriptive Statistics – Tennessee Public Schools Including Grades One through Five – Value-Added Scores, Per Pupil Expenditures, School Enrollment, Percentage Minority Enrollment, and Percentage on Free- and Reduced-Price Lunch – Fall 2001....33 Table 5: Numbers of Schools by Level of AR Implementation and School Group ..........35 Table 6: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Reading Achievement as the Criterion Variable, Spring 2001..........................................41 Table 7: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Language Achievement as the Criterion Variable, Spring 2001 .......................................42 Table 8: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Math Achievement as the Criterion Variable, Spring 2001...............................................44 Table 9: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Science Achievement as the Criterion Variable, Spring 2001...........................................45 Table 10: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Social Studies Achievement as the Criterion Variable, Spring 2001 ................................46 Table 11: Regression Analysis – TN Public Schools Ending at Grade 6 – Value-Added Reading Achievement as the Criterion Variable, Spring 2001..........................................48 Table 12: Regression Analysis – TN Public Schools Ending at Grade 6 – Value-Added Language Achievement as the Criterion Variable, Spring 2001 .......................................49 x List of Tables (continued) Table 13: Regression Analysis – TN Public Schools Ending at Grade 6 – Value-Added Math Achievement as the Criterion Variable, Spring 2001...............................................50 Table 14: Regression Analysis – TN Public Schools Ending at Grade 6 – Value-Added Science Achievement as the Criterion Variable, Spring 2001...........................................51 Table 15: Regression Analysis – TN Public Schools Ending at Grade 6 – Value-Added Social Studies Achievement as the Criterion Variable, Spring 2001 ................................52 Table 16: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Reading Achievement as the Criterion Variable, Spring 2001 ...................54 Table 17: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Language Achievement as the Criterion Variable, Spring 2001.................56 Table 18: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Math Achievement as the Criterion Variable, Spring 2001 ........................57 Table 19: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Science Achievement as the Criterion Variable, Spring 2001 ....................58 Table 20: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Social Studies Achievement as the Criterion Variable, Spring 2001..........59 Table 21: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value- Added Reading Achievement as the Criterion Variable, Spring 2001 ..............................61 Table 22: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value- Added Language Achievement as the Criterion Variable, Spring 2001............................62 xi List of Tables (continued) Table 23: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value- Added Math Achievement as the Criterion Variable, Spring 2001 ...................................63 Table 24: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value- Added Science Achievement as the Criterion Variable, Spring 2001 ...............................65 Table 25: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value- Added Social Studies Achievement as the Criterion Variable, Spring 2001.....................66 Table 26: One-Way ANOVA – All TN Schools Including Grades 1-5 – Value-Added Achievement by AR Category of Implementation ............................................................68 xii List of Figures Figure 1: All Schools – Levels of AR Implementation .....................................................36 Figure 2: Schools with Grades 1-5 – Levels of AR Implementation.................................37 Figure 3: 1-6 Schools – Levels of AR Implementation .....................................................38 Figure 4: 1-8 Schools – Levels of AR Implementation .....................................................39 Figure 5: All TN Public Schools Including Grades 1-5 – Value-Added Reading Achievement by Level of AR Implementation, Spring 2001 ............................................69 Figure 6: All TN Public Schools Including Grades 1-5 – Value-Added Language Achievement by Level of AR Implementation, Spring 2001 ...........................70 Figure 7: All TN Public Schools Including Grades 1-5 – Value-Added Math Achievement by Level of AR Implementation, Spring 2001 ............................................71 Figure 8: All TN Public Schools Including Grades 1-5 – Value-Added Science Achievement by Level of AR Implementation, Spring 2001 ............................................73 Figure 9: All TN Public Schools Including Grades 1-5 – Value-Added Social Studies Achievement by Level of AR Implementation, Spring 2001 ............................................74 1 CHAPTER 1 INTRODUCTION Background As technology has become increasingly infused into the educational landscape, more and more educational software is being developed and marketed. With this increased variety of technology tools from which to choose, educators need more than ever before to know which software programs can be expected to have a significant impact upon student learning. With an increasing emphasis on accountability and performance, administrators and teachers want to employ proven methods and tools to produce the greatest possible gains in student achievement. It is generally accepted that reading skills are an essential key to learning, and that as students read more, read more broadly, and with greater comprehension, they are able to master all school subjects more quickly and easily. And since it is estimated that America’s K-12 students spend an average of only seven minutes per day reading (Institute for Academic Excellence, 1996), there is obviously room for improvement. In an effort to systematize reading instruction and practice, many schools have implemented computer-based programs to determine students’ reading level, test their comprehension, and track their progress. The most widely used computer reading management program in Tennessee is Accelerated Reader® (AR), a system for managing and encouraging literature-based, developmentally appropriate reading practice for students, promulgated by Advantage Learning Systems (1993). AR is actually the original component of a 2 system now known as Reading Renaissance (RR) and henceforth used synonymously, which in turn is part of a comprehensive school improvement program marketed as School Renaissance. Reading, writing, and math skills are all targeted in this suite of computer applications and school-improvement tools. More than 50,000 schools have purchased AR across the country since its introduction in the early 1990’s. A significant number of Tennessee elementary schools have been using AR for several years, but it is reasonable to assume that the level of program implementation varies from school to school. As with most educational efforts and emphases, significant results are more likely to be observed when the program is vigorously and intensively implemented. Several dozen studies have reported mixed, although generally favorable, results of the effect of AR upon achievement in reading and other subjects, library circulation, attitudes toward reading, and even school attendance. In one of the largest studies to date, the Tennessee Value-added Assessment System (TVAAS) records of almost 63,000 Tennessee students in grades 2-8 in the 1996-97 school year were examined in relationship to a number of AR variables, including numbers of books read, readability level, and AR “points” earned, which are based upon reading comprehension quiz scores (Sanders & Topping, 1999). Analysis of the data revealed that “value added” generally increases with both reading volume and percentage correct on AR quizzes. A 1997 study (Paul, Swanson, Zhang, & Hehenberger) analyzed the Terra Nova achievement test scores and TVAAS gains for students in grades 3-8 in several hundred Tennessee schools, simply comparing schools that had purchased AR with those which had not. In 3 all grades and subjects, schools that owned AR outperformed schools that did not own AR. In this study, however, level of implementation was not considered. It could be argued, also, that this superior achievement is attributable to other factors, or even that a priori more “progressive” schools have a tendency to purchase AR. But a previous study (Paul, Vanderzee, Rue, & Swanson, 1996) analyzed test data for 2,500 AR schools with 3,500 socio-economically matched non-AR schools and found statistically significant differences in overall academic achievement between the AR schools and the control schools. The same study revealed an increase in gains with increased length of time schools had used AR. The question remains, however, whether the factor of AR utilization and implementation—alone or in combination with other variables—has a significant effect upon “value-added” achievement in Tennessee schools. Since hundreds of thousands of dollars are being spent, and thousands of hours for students, teachers, and librarians are being used each year on AR in Tennessee alone, educators really need to know whether the program is effective and, if so, under what conditions and circumstances. Accelerated Reader AR is not a method of teaching the skill of reading, nor of increasing reading speed, as the name might imply. Rather, AR is fundamentally a system of computerized testing and record keeping, whose goal is to increase literature-based reading practice at levels appropriately challenging to the reading development of students. The program is based upon a list of over 25,000 books, each of which has been assigned a reading level, 4 based upon a widely used readability index (Paul, Vanderzee, Rue, & Swanson, 1996). Each book is assigned a “point value” derived from its length and reading level. The basic formula for this calculation is as follows: AR Points = (10 + Reading Level) x (Words in Book/100,000) Procedurally, the student: 1. Selects a book from the list; 2. Reads the book; 3. Goes to a computer, either in the library or classroom on a network, and takes a five- to twenty-item, multiple-choice test, developed and validated by AR, about the content of the book. The test replaces the traditional book report or other assessment tool to give the teacher some reasonable assurance that the student has read the book. Provided that the student scores a minimum of 60 percent on the quiz, the program awards the student AR points according to this formula: AR Points Awarded = (Percentage Correct on Quiz) x (Book’s AR Point Value) The combination of test scores and points earned, along with the student’s reading level, is tracked by the software and reported to teachers, students, parents, and administrators by means of a variety of reports. A thorough explanation of AR methodology is provided by Sanders and Topping (1999), pages 3-5. As AR has evolved and become a component of Reading Renaissance, its sponsors have developed various levels of implementation to recognize those teachers 5 and schools that are adhering to the program’s tenets more vigorously. For purposes of this study, schools’ levels of implementation are classified in Table 1. The Tennessee Value Added Assessment System (TVAAS) TVAAS was developed by then-University of Tennessee Professor William Sanders in the 1980’s, and its deployment mandated by the Tennessee legislature in 1992, to provide unbiased estimates of the influences that school systems, schools, and teachers have upon the academic gains of students in the State’s 138 school systems (Sanders & Topping, 1999). The legal authority also requires the use of fresh, non-redundant, equivalent achievement tests each year for students in Tennessee public schools in grades two through eight. The achievement-testing program is known as TCAP, or Tennessee Comprehensive Assessment Program. The specific achievement test selected for use is the Terra Nova®, published by CTB-McGraw Hill. Students are tested in five major subject areas: reading, math, language, science, and social studies, using both norm- referenced and criterion- or curriculum-referenced items. Test items for each year’s administration are drawn from a large bank of equivalent items, at least 70% of which must be new in relation to the previous year’s tests. Terra Nova® has been subjected to rigorous examination of its psychometric properties and been found to be valid and reliable (Williams, 1989; Bock & Wolfe, 1996). From the largest longitudinally-merged database of student achievement in the United States, TVAAS inputs students’ scale scores derived from the Terra Nova® into a system of mixed-model, multivariate statistical analysis to eliminate the problem of 6 Table 1: Classification of Schools by Level of AR Implementation AR Level Description 0 School has not purchased AR. 1 School has purchased AR and is presumed to be using the program, but does not have any Model Classrooms. 2 Model Classroom certification has been awarded to one or two classes/teachers in the school, based upon specific criteria as set forth in the Model Classroom Checklist (Appendix A). 3 Model Classroom certification has been awarded to three (3) or more classes/teachers in the school, based upon specific criteria as set forth in the Model Classroom Checklist. 4 Model School certification has been awarded to schools in which all classes are utilizing AR, at least five teachers or 30% of reading teachers (whichever is greater) have achieved Model Classroom certification, and the school has met other criteria as forth in the Renaissance Model School Criteria (Appendix B). 7 missing data resulting from students’ missing tests, changing schools, changing districts, or moving out of state. TVAAS estimates of school, system, and teacher effects have been virtually uncorrelated with socio-economic factors and prior achievement (Sanders & Topping, 1999). A thorough discussion of the technical aspects of TVAAS may be found in Sanders, Saxton, & Horn (1997). In the fall of 2000, the State of Tennessee released its first-ever “report cards” on each public school system and school in the State (Report Card, 2000). In addition to demographic and other academic achievement data, schools encompassing grades K-8 posted three-year average TVAAS achievement gains. These value-added scores in reading, math, language, social studies, and science measure how much students in each school learned, on average, as measured against a national norm gain on the Terra Nova achievement test. The 2001 achievement and value-added data are available at on-line at the Web site of the Nashville Tennessean. Purpose of the Study The purpose of this study was to determine the relationship between the use of AR and four other variables and the achievement of students in Tennessee public schools over the past three years. In addition to level of AR implementation, the variables of (1) school enrollment, (2) percentage of students on free or reduced-price meals, (3) racial composition of the student body, and (4) per-pupil expenditure of the school system were examined. 8 Since the achievement test data reflect the average of testing done in the spring of 1999, 2000, and 2001, it seemed logical that, in order for AR/RR to have significant impact upon student achievement, the program would have had to be in place by the beginning of the 1999-2000 school year, at a minimum. In addition, not many schools had purchased and implemented since that time, most having done so earlier. Therefore, the study used a cut-off date of August 1, 1999, for purchase of AR, and schools' levels of implementation were considered as of that date. The Research Question The fundamental research question for this inquiry was this: “What are the effects of AR, at various levels of implementation, in combination with other selected variables, upon student achievement in Tennessee public schools encompassing grades K or 1 through 5 or above?” Significance of the Study The results of this study should help guide educational leaders in Tennessee and other states in their quest for programs that offer potential for significant positive impact upon student achievement. While some previous research points to the efficacy of AR, no statewide analysis of value-added gains in relation to AR implementation has been performed. Findings of this research were expected to indicate under what conditions or circumstances, and at what levels of use, AR can be expected to have a significant impact upon student achievement. 9 Limitations and Delimitations This study was delimited to: 1. The “report card” data on Tennessee public schools as compiled in the fall of 2001 by the Tennessee Department of Education and reported at the Web site of the Nashville Tennessean. 2. Those schools for which three-year average value-added scores were available. 3. The schools reported by the School Renaissance Institute as having, or not having, implemented AR at some level. The study was limited by: 1. The accuracy of the data supplied by the schools report card. 2. The accuracy of the data supplied by School Renaissance Institute. 3. The fact that the implementation data supplied by the School Renaissance Institute did not take into account the total number of classrooms in the school. 4. Lack of indication as to the grade levels certified as having “Model Classrooms.” 5. Lack of indication as to whether any Model Classrooms were at the grade levels tested. Assumptions The researcher assumed the following: 1. Schools that had purchased AR on or before August 1, 1999, but had not received any Model Classroom certifications, were nevertheless utilizing the program to some extent. 10 2. The Tennessee Value Added Assessment System is a valid measure of gains in student achievement from year to year. 3. Use of the three-year-average TVAAS gain statistic as a criterion variable tended to minimize “spikes” and other testing anomalies that examining a single year’s gains might have permitted. 4. The additional predictor variables identified above were the factors most likely to have significant effects within the regression analysis. Definition of Terms The following terms with which some readers may be unfamiliar are used in this study. They are defined as follows: Accelerated Reader® (AR) is a computer-based system for managing and encouraging literature-based, developmentally appropriate reading practice for students, promulgated by Advantage Learning Systems (1993). Institute for Academic Excellence™ is the research and development arm of School Renaissance®, formerly known as Reading Renaissance™, originally known as Advantage Learning Systems®. Model Classroom™ is the designation applied to teachers or classrooms meeting the criteria for AR set forth in the Model Classroom Checklist (Appendix A.) Model School™ is the designation applied to schools meeting the criteria as set forth in Appendix B. 11 Reading Renaissance™ (RR) and School Renaissance™ (SR) are names applied to the systems of implementation of AR and other programs, all marketed under the umbrella term of Renaissance Learning™, which is the current parent company of AR. Tennessee Comprehensive Assessment Program (TCAP) is the state’s mandated system of tracking student achievement in public schools. It consists of standardized norm-referenced and criterion-referenced testing in several subject areas in grades two through 12. Tennessee Value-Added Assessment System (TVAAS) is a statistical procedure developed by Dr. William Sanders for estimating student achievement gains from year to year by comparing scale scores on the Terra Nova® achievement tests, using mixed- model multivariate analysis to control for extraneous variables. TVAAS scores are reported as percentages of expected gains. The system is also used to compare groups of students, teachers, schools, and school systems. The public has access only to school- level data. Terra Nova® Achievement Tests are norm-referenced and criterion-referenced batteries of objective tests published by CTB-McGraw Hill and administered to students in grades two through eight under the TCAP. Summary of Chapter 1 Tennessee’s educators, like their counterparts across the nation, are seeking ways to enhance student achievement. Various programs have been developed to help achieve 12 this goal. One of the most widely used such programs is Accelerated Reader® (AR). While its authors and publisher claim that it is effective in increasing reading practice and comprehension, and improving achievement in other school subjects, scientific research on the subject is incomplete and inconclusive. This study of the relationships between AR implementation, together with other variables, and student achievement in Tennessee was designed to add to the body of knowledge about which programs tend to enhance value-added achievement. Organization of the Study This study was organized in the following manner: Chapter 1 provides a background on the subject, information about AR and its implementation, information about the Tennessee Value-Added Assessment System (TVAAS), the purpose of the study, the research question, the significance of the study, limitations and delimitations of the study, assumptions upon which the study rests, definitions of possibly unfamiliar terms, and a summary of the chapter. Chapter 2 is devoted to a review of the literature extant on AR. Four categories of research are reviewed: (1) research conducted or sponsored by the School Renaissance Institute; (2) dissertations and theses; (3) journal articles, reports, and papers; (4) unpublished “field reports” from schools and districts. Chapter 3 addresses the methodology of the study, the subjects, data collection and classification procedures, and statistical methods and procedures. 13 Chapter 4 includes the findings and analysis of the data, presented in a series of tables and charts with interpretation of results. Chapter 5 is a summary of the study, conclusions, and some recommendations for further research. 14 CHAPTER 2 REVIEW OF RELATED LITERATURE Introduction Considering the number of schools using AR and the monetary and time investment represented, there is not a great deal of independent research on its effectiveness reported in the educational literature. This chapter provides a review of selected writings in these four categories: (1) Large-scale research by the School Renaissance Institute, formerly known as the Institute for Academic Excellence, publishers of AR; (2) a few dissertations and theses which have examined AR with respect to its effect on student achievement, reading practice, attitudes, and library circulation; (3) journal articles, reports, and papers by educators relating to the use of AR; (4) unpublished “field reports” from schools and districts using AR. Within each category, the research tending to favor the use of AR will be reviewed first, followed by the research tending to question its efficacy. Institute Research The “landmark” study by the Institute for Academic Excellence (1996) collected reading performance data for 659,214 students in grades K-12 during the 1994-95 school year. Analysis of the data revealed the following points: • Average reading practice among all students is 7.1 minutes per day. • Reading practice declines markedly after fifth grade. 15 • High school students spend as much time practicing reading as do kindergarten students—about three minutes per day. • When ranked according to the amount of reading practice they do, students in the top 5% read 144 times as much as students in the bottom 5%. • Students in the highest performing states on the National Assessment of Educational Progress (NAEP) engaged in 59% more reading practice than students in the bottom quartile of the states. • Students in schools with populations of 200 or fewer engage in twice as much reading practice as do students in school of 1,000 or more. • Students in private schools practice reading 67% more than students in public schools. An earlier study by the Institute (1993) collected reading and math standardized scores from 10,124 students in grades 1-9 in 136 schools nationwide during the school years 1991-92. Predicted reading improvement ranged from 1.3 to 26.9 percentiles per 100 AR points earned by the students. Greater improvement was found for lower-ability readers, but all students seemed to benefit from reading practice. Predicted math improvement ranged from 1.9 to 11.7 percentiles per 100 AR points earned. Paul, Vanderzee, Rue, and Swanson (1995) collected attendance and standardized test data for 2,511 AR schools and 3,500 socio-economically matched non-AR control schools in Texas. AR schools displayed statistically significant superior performance in all subject areas tested: reading, math, science, social studies, and writing. Gains in 16 academic performance increased with the length of time the school had been using AR. Attendance was also better at AR schools. In a study (Paul, Swanson, Zhang, and Hehenberger, 1997) with similarities to the proposed one, the TCAP/TVAAS scale scores and gains of almost 700 Tennessee schools for the 1995-96 school year were examined in relation to the schools’ use of AR. In this study, AR implementation was simply categorized in one of three ways: (1) an AR school if the school had purchased the program prior to September, 1995; (2) a non-AR school if it had not purchased AR as of June, 1997; or (3) a Transition school if it purchased AR at any time after August, 1995. Only the scale scores and TVAAS gains for one year were used as the criterion variable. In each of the 30 grade-subject pairs, AR schools had higher adjusted mean scale scores than non-AR schools. AR schools had a higher adjusted mean gain in all grades and subjects except in reading, science, and social studies in grade 5. In reading, language, and math in all grades, “Transition” schools performed at a level above that of non-AR schools but below that of AR schools. Yet another Institute study (1999) analyzed the 1998-99 reading practice and achievement data for 12,984 students, grades 1-9, in 50 Idaho elementary, middle, and junior high schools. Compared to a national sample of their peers, the students in AR schools gained an average of 1.84 normal curve equivalents (NCE’s) in reading. An important aspect of this study is that the level of reading growth was found to be nearly twice as high for students in schools with Model Classrooms as it was for those in schools where no staff members had received Reading Renaissance training. 17 Dissertations and Theses Morse (1999) monitored the reading practice and assessment performance of 60 first-, second-, and third-grade students for eight months, using the Standardized Test for the Assessment of Reading (STAR) as a pre- and post-test. She found that students who earned 50 or more AR points had significantly greater achievement gains than those who did not. Analysis of variance by Kunz (1999) on data from the Illinois School Report Card factors of school district size, grade level, reading program, and average reading scores demonstrated significant positive interaction for AR. Bork (1999) compared students in two parochial elementary students in a similar fashion to that of Morse (above) and found moderate positive correlation between reading level as measured by STAR and AR points earned. No significant relationships were found for age, grade, or gender. Howard (1999) wished to learn whether recreational reading, using Accelerated Reader, influenced reading vocabulary, comprehension, and attitude when socioeconomic status was low. Seven Title I schools in urban Southeastern Virginia participated in pre- testing in September/October 1998 and post-testing in May/June 1999. Two independent variables, each with three levels, were manipulated: (1) Type of AR Usage, i.e. low (0–20 points), average (21–74 points), high (75 and above points); and (2) Grade Level, i.e. three, four, and five. Dependent variables reading, vocabulary, and comprehension were measured using the Gates-MacGinitie Tests of Reading, Form L, on 755 students. The dependent variable attitude was measured on 515 students who completed the Elementary Reading Attitude Scale (ERAS). Positive findings were as follows: (1) At pre-testing 18 75% or greater of all students tested below grade level in both reading vocabulary and comprehension. At post-testing, after the AR treatment had been administered for the duration of the school year, the percentage of students testing below grade level for reading vocabulary and comprehension significantly decreased. (2) Results of the Multivariate Analysis of Variance (MANOVA) were significant for Type of AR Usage and Grade Level effects. When Type of AR Usage was considered, significant differences between pre-test and post-test assessment of vocabulary and comprehension were noted. (3) Review of the data for the mean difference in vocabulary and comprehension by Grade Level and Type of AR Usage indicated that as participation in the AR program increased, the mean score differences also increased. (4) Analysis of Variance revealed that only the “Type of AR Usage” effect was significant. Vega (1999) wished to help raise reading levels in a third grade class of at-risk students. Sixty-three percent of the students in the study were reading below grade level in the (1997–1998) school year and had been experiencing low reading abilities since first grade. They participated in AR from October to May. At the end of the study, 75 percent of the subjects were reading on or above grade level. Twenty percent were still reading below grade level, compared with 63 percent at the beginning of the study. Rogers (2000) examined AR in a middle Georgia elementary school to determine its perceived impact on students' reading experiences, attitudes, and habits. The subjects of the study were a selected group of fifth graders and their teachers, with both groups participating in structured oral individual interviews and three student focus group discussions. Standardized test scores were also examined. Analysis of the collected data 19 indicated that the program had a meaningful impact on students. Both teachers and students perceived the program as being successful in getting students of all abilities and interests to read with high frequency and on a wide variety of subjects. AR impacted students’ reading abilities as well, as measured by their high reading scores on a nationally norm-referenced standardized testing instrument and teacher opinion. Nancy Facemire (2000) examined the effect of AR on the reading comprehension scores of third grade students in a socio-economically disadvantaged area of West Virginia. The experimental group of students was encouraged to read and test on books supported by the AR program. The STAR program was used to pre-test and post-test students and the group scores were used to ascertain if significant growth in reading comprehension occurred in the experimental group. Analysis of the data in this study did suggest that a significant difference could be attributed to AR. Mitchell Pratt’s dissertation (1999) used an ex post facto non-randomized design to compare students’ achievement at two Utah elementary schools, one using AR and one not. While he found no statistically significant differences in score on the Utah Core Assessment Series test or the Stanford Achievement test, Pratt points out that AR implementation was not complete. In a limited study, McKnight (1992) examined the reading motivation, habits, and attitudes of a fifth grade class. This study found that the majority of students greatly improved after using AR for 11 weeks. A doctoral study by Teresa Spradley (1998) compared the reading, math, and language achievement of 47 sixth graders in an AR school with those of 47 students 20 randomly selected from three other non-AR schools in the same district, controlling for gender and economic status. ANCOVA’s and multiple linear regression analyses were interpreted to reveal significant increases in reading and language for the students using AR, and in reading, language, and math for students above the poverty level. A correlational study by Holman (1998), however, uncovered no statistically significant relationship between AR points earned and reading comprehension gain on the Iowa Test of Basic Skills for 170 randomly-selected fourth and fifth grade students at Early County (GA) Elementary School. Neither did Mary Knox (1996) find significance at the .05 level for vocabulary, reading comprehension, nor number of books read by 77 fourth- and fifth-graders in AR as compared to students in a teacher-directed reading improvement program. McMillan (1996) selected a sample of 214 fourth grade students from three elementary schools in a mid-urban district. The experimental group participated in AR during the school year, but did not improve their reading comprehension skills significantly (as measured by the Texas Assessment of Academic Skills) compared with those who did not use AR. Library circulation records did reveal that the AR students checked out more books, longer books, and books with a higher reading level. Journal Articles, Reports, and Papers Sanders and Topping’s (1999) analysis suggested that both student reading volume and percentage correct on AR quizzes have a positive impact on teacher effectiveness as measured by TVAAS. Among their conclusions are the following: 21 • In general, value-added rises with increased numbers of books read by students, and increasing percentage correct on AR quizzes. • Teachers completing Reading Renaissance training were significantly more effective than control teachers who had not. • Model Classrooms had higher effectiveness in fourth and fifth grades. Peak and Dewalt (1994) tracked the progress of 50 ninth-grade students who had used AR since third grade. The AR students displayed improved reading attitudes and posted higher reading scores on the California Achievement Test (CAT) than a control group of 50 students. Vollands, Topping, and Evans (1999) reported on quasi-experimental action research that evaluated AR for 51 sixth grade students in two schools in severely socio- economically disadvantaged areas. They found that the program, even when less than fully implemented, yielded gains in reading achievement superior to gains from regular classroom teaching and an alternative intensive method for 38 control students, as well as significant improvement in girls' reading attitudes. A University of Texas study (1998) correlated AR points earned and quiz percentage correct to the probability of students’ passing the reading portion of the Texas Assessment of Academic Skills (TAAS). Fifth grade students at Barbers Hill Elementary School were studied during the 1996-97 school year. Results indicated that students earning more than 55 AR points and scoring more than 85% correct of AR quizzes have a 95% chance of passing the TAAS reading exam. Students below these levels have a 17% chance of passing. 22 Goodman (1999) evaluated the effectiveness of AR as implemented at Gardner Middle School in San Manuel, Arizona, during the period April 1997 to April 1998. Two hundred eighty-two students in the seventh and eighth grades were pre-tested and post- tested with the Gates-MacGinitie Reading Tests Form K (pre-test) and Form L (post- test). The mean pre-test scores and post-test scores were compared, using t-tests to determine if there were statistically significant gains or losses. Students exhibited statistically significant improvement in vocabulary and in total scores (vocabulary and comprehension combined) in all areas. On the other hand, researchers like Prince and Barron (1998) have suggested that while there may be positive benefits to computerized reading programs and awards, there may be greater negative consequences to their use: “Studies suggest that use of the widely known Accelerated Reader Program alone cannot create better lifelong learners. Educators need to examine practices that have worked well in the past and work hard to establish sound principles that will produce able learners and readers” (abstract.) Betty Carter (1996) warns that although the computerized reading management programs increase library circulation and standardized test scores, they have drawbacks. She asserts that such programs “devalue reading, diminish motivation, limit title choice, restrict materials selection and collection development, discourage independent selection of books, emphasize testing rather than needs, and fail to make the best use of school resources.” (p.24). Roseneck et al (1996) surveyed fifth-grade students in three Lee County, Florida, schools. Two hundred twenty-two surveys were completed and the results tabulated. 23 Results indicated no relationship between the use of AR and frequency of library use or attitudes toward reading and the media center. Mathis (1996) presented the findings of a study of the use of AR to increase the reading comprehension scores on the Stanford Achievement Test (SAT) of sixth-grade students compared with the previous year when they did not use the program. Subjects for the study were 30 sixth-grade students from a rural farm community in north central Illinois. Results indicated that, after a year of exposure to the AR program, there was no statistically significant increase in reading comprehension scores from the fifth to the sixth grade. Field Reports from Schools and Districts Table 2 summarizes selected field reports from schools and districts across the country using AR. This information has been reported to the Institute for Academic Excellence (1999) and has not been verified independently. Summary of Chapter 2 Independent researchers have attempted to verify the effectiveness of AR in a number of studies of various sizes and types. Some of these studies have concluded that AR use is effective in improving reading comprehension, practice, and/or attitudes. A few investigations have pointed to a link between AR use and higher achievement in other subjects. Research conducted or sponsored by the School Renaissance Institute or its associated entities has invariably confirmed the efficacy of AR. There remain, however, a number of dissertations, theses, and research projects that have found no 24 Table 2: Field Reports from Schools and Districts – Summary Institute for Academic Excellence (1999) School or District Location Results Monroe County Key West, FL After AR implementation, reading achievement gap narrowed by 57% on Stanford Achievement Test, 9th Edition (SAT 9). Craven County New Bern, NC After Reading Renaissance (RR) implementation, pass rates on state proficiency test increased from 66% to 81% in reading, 62% to 79% in math. Shelby Oaks Elementary Memphis, TN RR implemented in 1997-98; 1998 mean gain for reading 95% above national average; math, 27.8%; language, 67.3%. Horizon Elementary Jerome, ID During two-year period of Model Classroom certification, reading growth increased 4.74 and 4.35 NCE’s, respectively. Harris Elementary Mesa, AR After implementation of Reading Renaissance®, 470 students in grades 2-6 gained an average 12.6% in reading in one year on SAT 9. Hobbton Elementary Newton Grove, NC After implementation of Reading Renaissance® (RR), 222 students in grades 3-5 led the district in reading and math on the state’s end-of-grade tests; reading performance achievement gap decreased by 54%-57%. Grant Elementary Muscatine, IA After school-wide RR implementation, library circulation increased 500%; Iowa Test of Basic Skills (ITBS) reading comprehension scores improved from the 40th to the 70th percentile; attendance increased from 92% to 96%. Pittsburg Middle Pittsburg, TX In first two years of RR implementation, all 500 students averaged 4.23 years’ growth on Stanford Diagnostic Reading Test. Buford Elementary Buford, GA With RR implementation, ITBS scores for 830 students indicated three years’ reading growth in a two-year period. Collins Elementary Collins, MS Percentage of 255 children in grades 2-4 reading below grade level dropped from 67% to 33% in one year. Miramonte Elementary El Monte, CA In a controlled study, AR was implemented with 80% of the school’s 3rd through sixth graders. After three months, the AR students achieved 28% higher reading scores on the SAT 9. 25 Table 2 (continued): Field Reports from Schools and Districts – Summary Institute for Academic Excellence (1999) Cottonwood-Oak Creek School District Cottonwood, AR After AR and RR implementation in two elementary and two middle schools, mean reading percentile scores on the SAT 9 increased seven points in one year. Bryan Independent School District Bryan, TX RR was implemented district-wide in 14 elementary schools and three middle schools with 10,000 students. Total growth in Texas Assessment of Academic Skills (TAAS) pass rates ranged from 12.3% in reading to 23.4% in math in two years. Heritage Middle Middlebury, IN After RR implementation, 125 sixth graders averaged reading growth of 1.5 years in one year, larger gains for students with initially lower abilities. McCamey Primary McCamey, TX In six months of using RR, 38 second graders experienced reading growth of 1.5 years, 6 percentile rankings, and 3.2 normal curve equivalent units (NCE’s). Coleridge Community School Coleridge, NE During one year of full RR implementation, 52 students, grades 3-6, averaged 2.1 years’ growth on Comprehensive Test of Basic Skills (CTBS). W. Alonzo Locke Elementary Memphis, TN AR was implemented school-wide in 1997-98. TVAAS reading gains for the 170 students in grades 3-5 were 47% higher than in previous year. 100% of Locke’s students are minority and qualify for free or reduced-price lunch, and the mobility rate is 32%. 26 connection between AR use and better readers or higher achievement in other academic subjects. The preponderance of the literature, published and unpublished, suggests that, under certain conditions, AR is an effective tool for increasing reading practice and comprehension among grade-school students. Some studies have pointed to a link between its use and higher achievement in other school subjects. A few studies have been critical of the program for its uncreative approach and low-level questions, casting doubt upon its usefulness in stimulating higher-order thinking in students. 27 CHAPTER 3 METHODOLOGY Introduction The purpose of this study was to determine whether statistically significant relationships exist between levels of AR/RR implementation and student achievement in Tennessee public schools that include grades one through five in reading, language, math, science, and social studies. In order to control for potential extraneous factors, this analysis also included the following school variables: (1) school enrollment; (2) percentage of students on free or reduced-price lunch; (3) racial composition of the student body; and (4) per-pupil expenditure of the school system. This chapter describes the subjects of the study, data collection procedures, statistical methods and procedures, and computer-assisted data analysis. Subjects The subjects of this study include all Tennessee public schools that contain grades 1-5. The schools identified for the study are taken from the directory of schools on the Web site of the Tennessee Department of Education. Data Collection Of the 2,262 active schools and educational institutions recognized and listed by the Tennessee Department of Education, 781 public schools serve students in grades 1-5. 28 Table 3: Schools in the Study by Grade Structure and Grades Tested Grades Contained # Schools Grades Tested PK, K, 1-5 373 3,4,5 PK, K, 1-6 163 4,5,6 PK, K, 1-8 or higher 245 6,7,8 Total 781 These schools are divided into three groups for purposes of this study, as outlined in Table 3. Since the skill of reading is usually taught by the time students complete the fifth grade, it is this group of schools that comprise the subject population for this study. This is the first of three major sources of data for the investigation. The second data stream comes from the purchase and implementation records maintained by School Renaissance Institute, Inc., Madison, WI, promulgators of AR/RR. The Institute has agreed to provide information concerning which Tennessee schools have purchased AR, when this purchase was effected, and what implementation level should apply to purchasing schools, as stated in Chapter 1 and according to Table 1 (page 5). This information is for statistical purposes only and has been treated as confidential, according to written agreement between the researcher and the Institute. The third source of data is the Tennessee 2001 Schools Report Card, available on- line at the Web site of the Nashville Tennessean. This site provides, for each school, the following items of information pertinent to this study: (1) percentage of students on free or reduced-price lunch, a measure of socioeconomic status; 29 (2) per-pupil expenditure for the district; (3) total pupil population in the school; (4) percentage of minority students; and (5) three-year-average value-added achievement gains in reading, math, language, science, and social studies. Methods and Procedures The statistical analysis involved in this study required the construction of a sizeable database including the above information about schools in Tennessee that include grades 1-5. In addition to the necessary directory-type information, the required fields were as follows: • percentage of students on free or reduced-price lunch; • per-pupil expenditure for the district; • total pupil enrollment; • percentage of minority students; • three-year-average value-added achievement scores in reading, math, language, science, and social studies; • level of AR/RR implementation. In addition to calculating the respective correlations between each variable and the value-added scores in each subject, multiple linear regression was used to calculate the effects of the selected variables (X1…6) upon the criterion variable (Y) (value-added score) for each of the five subjects: reading, language, math, science, social studies. The 30 level of AR/RR implementation was entered into the regression as a dummy variable based upon group membership, as described in Hinkle, Weirsma, and Jurs (1994). This procedure yielded a multiple R, estimating the relationship between the value-added scores and all the predictor variables; an R2, or the percentage of variation in the value- added score explained by variations in the predictor variables; an adjusted R2, reflecting the sample size; the standard error of the estimate; and coefficients and their significance for each of the predictor variables at the .05 level of significance. Interpretation of these results indicated the relative contribution of each of the predictor variables to the variation in value-added scores for each subject. In this way it was possible to estimate the effects of AR/RR implementation within the context of the other predictor variables. Finally, the means of the value-added scores in each of the five subjects were compared for all schools in the study based upon level of AR implementation. Analysis of variance was employed to determine whether differences in the means were significant at the .05 level. 31 CHAPTER 4 FINDINGS AND ANALYSIS OF DATA Introduction The Tennessee public schools which are the subjects in this study were initially divided into three groups: all schools including grades one through five, but ending at grade five (n = 373); all schools including grades one through five, and ending at grade six (n = 163); and all schools including grades one through five, and ending at grade eight or higher (n = 245). Thus, a total of 781 schools became the subject population for the study. Data for the study were derived from three sources. The collection process began, as stated in Chapter 3, with a list of all 1,965 active Tennessee public schools downloaded from the Tennessee Department of Education Web site. This list included school names, grade levels, identification codes, addresses, districts, and other demographic data. The school list was filtered to eliminate schools that did not include grades one through five, i.e. grades in which the subject of reading is traditionally given priority in the curriculum. Value-added achievement data, the second data source for the study, were obtained through the cooperation of, and from a spreadsheet created by, Dr. Benjamin Brown of the Tennessee Department of Education. The third source of data, including school enrollment, percentage minority population, district per-pupil expenditures, and percentage of students on free or reduced-price lunch (% F&RL), was the State’s online publication of the School Report Card, issued annually and available online at the Web site of the Nashville Tennessean. Value-added achievement data 32 and/or other variables (enrollment, percentage minority, or percentage on free or reduced- price lunch) were unavailable for 40 of the schools. Therefore, 741 schools remained in the study as valid cases. The next step in the process of data collection and analysis was the merging of value-added achievement scores into the spreadsheets with the other variables for the schools. The descriptive statistics for the data are presented in Table 4. The achievement scores were obtained, as noted previously, in electronic format from the Tennessee Department of Education. The statistic of interest in each subject was the cumulative three-year (1999, 2000, and 2001) average value-added gain as a percentage of the national norm gain. The scores ranged, as noted in Table 4, from a negative 19.2% for one school in language to 216.1% for another school in science. Next, the demographic variables for each school (per-pupil expenditure for the district, total school enrollment, percentage minority enrollment, and percentage of students eligible for free or reduced-price lunch) for the 2000-2001 school year were obtained from the Tennessee School Report Card, online at the Web site of the Nashville Tennessean. Table 4 also illustrates that per-pupil expenditure ranged from $4,281 per year in Smith County to $7,376 per year in Alcoa. School enrollment within the schools for this study ranged from 58 students at Shady Valley Elementary School in Johnson County to 1,603 students at Snowden School in Memphis. Both percentages of minority students and students eligible for free or reduced-price meals ranged from 0% to 100%. 33 743 -12.5 183.2 102.885 20.857 743 -19.2 199.4 91.330 24.937 743 56.9 176.8 106.228 18.877 743 32.6 216.1 110.998 18.836 743 2.0 193.0 109.636 21.039 763 4281 7376 5723.30 649.76 759 10 1603 488.71 240.18 759 0 100 23.59 33.02 759 0 100 51.86 26.33 741 Reading Language Math Science Soc. Studies $/Pupil Expend. Enrollment % Minority % F&RL Valid N (listwise) N Minimum Maximum Mean Std. Deviation Table 4: Descriptive Statistics – Tennessee Public Schools Including Grades One through Five – Value-Added Scores, Per Pupil Expenditures, School Enrollment, Percentage Minority Enrollment, and Percentage on Free- and Reduced-Price Lunch – Fall 2001 34 Finally, the factor of AR implementation level was coded for each school with data supplied by the research department of Renaissance Learning®, parent company of AR. Data were available on 764 of the subject schools in the study. Five mutually exclusive categories of AR implementation were identified and coded as to group membership. If the school had not purchased AR (“AR0”), then all four columns received “0’s.” The “AR1” column received a “1” if the school had purchased AR prior to August 1, 1999, but had no model classrooms. The “AR2” column received a “1” if the school had certified one or two model classrooms. The “AR3” column received a “1” if the school had certified three or more model classrooms. The “AR4” column received a “1” if the school had certified as a model school in AR implementation. Table 5 summarizes the various levels of AR implementation by school group. Figures 1 through 4 illustrate that, in all three groups of schools, the vast majority of the schools (89%) have purchased AR prior to August 1, 1999, but relatively few schools (10%) have documented more vigorous levels of implementation. Among schools ending at grade six, none of the schools have achieved “Model School” status. Data Analysis and Findings Excel® spreadsheets containing the data above were developed to analyze the data for each of the schools in the study: per-pupil expenditure in the district; total enrollment in the school; percentage of minority enrollment; percentage of students on free or reduced-price lunch as a measure of socioeconomic status; level of AR implementation; cumulative three-year average value-added gains as a percentage of the 35 Table 5: Numbers of Schools by Level of AR Implementation and School Group, Tennessee Public Schools including Grades 1-5, as of August 1, 1999 Group AR0 AR1 AR2 AR3 AR4 Total N 1-5 Schools 32 317 10 10 4 373 1-6 Schools 26 114 8 12 0 160 1-8 Schools 24 177 8 21 1 231 TOTAL 82 608 26 43 5 764 36 All Schools - Levels of AR Imlementation AR1 79% AR4 1% AR3 6% AR0 11% AR2 3% Figure 1: All Schools – Levels of AR Implementation 37 Chart 3: Schools with Grades 1-6 – Levels of AR Implementation Figure 2: Schools with Grades 1-5 – Levels of AR Implementation 1-5 Schools - Levels of AR Implementation AR1 84% AR3 3% AR4 1%AR2 3% AR0 9% 38 Chart 4: Schools with Grades 1-8 – Levels of AR Implementation Figure 3: 1-6 Schools – Levels of AR Implementation 1-6 Schools - Levels of AR Implementation AR0 16% AR1 71% AR2 5% AR3 8% AR4 0% 39 Figure 4: 1-8 Schools – Levels of AR Implementation 1-8 Schools - Levels of AR Implementation AR1 78% AR2 3% AR3 9% AR4 0% AR0 10% 40 national norm expected gain on the Terra Nova® achievement tests in reading, language, math, science, and social studies. Three spreadsheets were initially constructed, one for each of the types of schools identified: those schools including grades one through five but ending at grade five; those schools including grades one through six, but ending at grade six; and those schools including grades one through eight (regardless of whether they ended at grade eight.) Multiple regression analysis in the general form was first performed for each of the school groups, using SPSS® for Windows Student Version, to determine the relationships between the independent variables and the criterion variables of achievement gains for each of the five subject areas. The results of this analysis are presented in Tables 6 through 25 on pages 41 through 66, with interpretation of the results, beginning with the group of schools ending at grade five. In the analysis depicted in Table 6, the small adjusted R square, or coefficient of multiple correlation, (.021), the statistical significance of the regression ANOVA’s being greater than alpha (.052), and the small Beta weights of the predictor variables indicate that a very small amount of the variation in the criterion variable (reading scores) for this group of schools is explained by the variation in the combination of predictor variables (per-pupil expenditure, school enrollment, proportion of minority or socioeconomically disadvantaged students, or the school’s level of AR implementation.) As was the case when reading was examined, the language regression model (Table 7) did not reveal significant statistical relationships between language achievement and AR level of implementation. Only the variables of percentage minority 41 Table 6: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Reading Achievement as the Criterion Variable, Spring 2001 ANOVAb 8097.999 8 1012.250 1.951 .052a 180054.9 347 518.890 188152.9 355 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. Dependent Variable: Readingb. Coefficientsa 83.512 16.269 5.133 .000 51.514 115.509 4.891E-03 .002 .127 2.089 .037 .000 .009 3.857E-03 .006 .039 .611 .542 -.009 .016 -9.64E-02 .056 -.134 -1.731 .084 -.206 .013 -7.11E-02 .062 -.083 -1.144 .253 -.193 .051 -2.075 4.733 -.032 -.438 .661 -11.383 7.233 6.825 8.866 .047 .770 .442 -10.613 24.262 -8.045 8.615 -.058 -.934 .351 -24.990 8.900 -8.628 12.429 -.040 -.694 .488 -33.072 15.817 (Constant) $/Pupil Expend Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Lower BoundUpper Bound 5% Confidence Interval for B Dependent Variable: Readinga. Model Summary .207a .043 .021 22.779 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. 42 Table 7: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Language Achievement as the Criterion Variable, Spring 2001 Model Summary .328a .107 .087 20.676 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. ANOVAb 17831.312 8 2228.914 5.214 .000a 148334.6 347 427.477 166166.0 355 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. Dependent Variable: Languageb. Coefficientsa 79.640 14.766 5.393 .000 2.705E-03 .002 .075 1.273 .204 2.122E-03 .006 .023 .370 .711 -.148 .051 -.220 -2.934 .004 -.144 .056 -.179 -2.553 .011 -5.725 4.295 -.094 -1.333 .184 -5.243 8.047 -.038 -.652 .515 -2.304 7.820 -.018 -.295 .768 -.254 11.281 -.001 -.023 .982 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Languagea. 43 enrollment and proportion of students on free or reduced-price lunch had any significant association with language, and the relationship was slightly negative. In relation to math achievement (Table 8), as with language, AR implementation was not a significant contributor to the regression. Only percentage minority and socioeconomic make-up of the school make a significant, albeit negative, contribution to the model. None of the variables in the science achievement equation (Table 9) appear to have a significant relationship to science achievement within this group of schools, with the possible exception of the percentage of students on free or reduced-price meals. This factor actually seems to be associated with slightly higher science gains. The analysis of social studies achievement (Table 10) in this group of schools reveals a significant negative relationship with percentage of minority students, and with more intensive levels of AR implementation (AR3 and AR4.) Summary of Analysis of Schools Ending at Grade Five None of the foregoing regressions in this group of schools resulted in adjusted R square values that would yield a reliable prediction model of the selected variables with respect to value-added achievement in any of the five subjects. It is impossible to assert, based upon these data, that the variation in value-added-achievement among these schools does not occur by chance. Nevertheless, some interesting, if inexplicable, negative associations are evident. 44 Table 8: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Math Achievement as the Criterion Variable, Spring 2001 Model Summary .369a .136 .117 19.743 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. ANOVAb 21379.312 8 2672.414 6.856 .000a 135252.1 347 389.776 156631.4 355 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. Dependent Variable: Mathb. Coefficientsa 118.041 14.100 8.372 .000 6.395E-04 .002 .018 .315 .753 2.745E-03 .005 .030 .502 .616 -.157 .048 -.240 -3.252 .001 -.154 .054 -.197 -2.857 .005 -4.508 4.102 -.076 -1.099 .272 -6.426 7.684 -.048 -.836 .404 -2.939 7.467 -.023 -.394 .694 -8.347 10.772 -.042 -.775 .439 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Matha. 45 Table 9: Regression Analysis – TN Public Schools Ending at Grade 5 – Value-Added Science Achievement as the Criterion Variable, Spring 2001 Model Summary .214a .046 .024 20.426 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. ANOVAb 6971.902 8 871.488 2.089 .036a 144769.9 347 417.204 151741.8 355 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. Dependent Variable: Scienceb. Coefficientsa 107.229 14.588 7.351 .000 2.564E-04 .002 .007 .122 .903 -2.67E-03 .006 -.030 -.472 .637 -6.77E-02 .050 -.105 -1.355 .176 .165 .056 .214 2.955 .003 -3.364 4.244 -.058 -.793 .429 -5.974 7.950 -.045 -.752 .453 -8.863 7.725 -.071 -1.147 .252 -13.690 11.144 -.070 -1.228 .220 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Sciencea. 46 Table 10: Regression Analysis – TN Public Schools Ending at Grade 5 – Value- Added Social Studies Achievement as the Criterion Variable, Spring 2001 Model Summary .314a .099 .078 22.113 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. ANOVAb 18574.853 8 2321.857 4.748 .000a 169682.0 347 488.997 188256.8 355 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, $/Pupil Expend., Enrollment, AR1, % Minority a. Dependent Variable: Soc. Studiesb. Coefficientsa 93.552 15.793 5.924 .000 4.355E-03 .002 .113 1.916 .056 2.025E-03 .006 .020 .330 .741 -.244 .054 -.340 -4.522 .000 2.403E-02 .060 .028 .398 .691 -8.125 4.594 -.125 -1.769 .078 -9.959 8.607 -.068 -1.157 .248 -20.996 8.363 -.151 -2.510 .013 -26.747 12.065 -.123 -2.217 .027 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Soc. Studiesa. 47 Schools Ending at Grade Six Tables 11 through 15 display the regression analyses for value-added achievement in the five subject areas for schools ending at grade six. A brief interpretation of any significant results follows. None of the schools ending at grade six are classified as AR4 (“Model School”.) With respect to reading achievement (Table 11), no variable gave evidence of significant contribution to the prediction equation. Although the adjusted R square value indicates that only about 9% of the variability in language achievement (Table 12) is explained by the selected variables, there appears to be a positive correlation between language achievement and classification as an AR1 school. The factors examined seem to have a greater impact upon math achievement (Table 13) among the schools ending at grade six. Almost one third (.291) of the variation is accounted for in the model. Although AR use does not seem to be factor, school enrollment and especially racial composition of the school (standardized beta coefficient -.424) are negatively associated with math achievement for this group of K- or 1-6 schools. Science achievement (Table 14) is not significantly related to this combination of variables, but there appears to be a very weak negative relationship between minority enrollment and AR1 status and this subject. Only percentage of minority enrollment seems to be related significantly at the .05 level to social studies achievement (Table 15). 48 Table 11: Regression Analysis – TN Public Schools Ending at Grade 6 – Value- Added Reading Achievement as the Criterion Variable, Spring 2001 Model Summary .152a .023 -.022 18.994 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. ANOVAb 1291.984 7 184.569 .512 .825a 54479.255 151 360.790 55771.239 158 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. Dependent Variable: Readingb. Coefficientsa 89.874 18.454 4.870 .000 1.422E-03 .003 .052 .481 .631 -4.75E-03 .008 -.055 -.587 .558 -6.38E-02 .070 -.136 -.914 .362 -7.77E-03 .078 -.012 -.100 .920 1.012 4.570 .025 .221 .825 .805 8.053 .009 .100 .921 -1.527 7.068 -.022 -.216 .829 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Readinga. 49 Table 12: Regression Analysis – TN Public Schools Ending at Grade 6 – Value- Added Language Achievement as the Criterion Variable, Spring 2001 Model Summary .361a .131 .090 24.342 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. ANOVAb 13432.325 7 1918.904 3.238 .003a 89475.845 151 592.555 102908.2 158 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. Dependent Variable: Languageb. Coefficientsa 82.064 23.650 3.470 .001 3.121E-03 .004 .085 .824 .411 -2.88E-02 .010 -.245 -2.775 .006 -.107 .089 -.168 -1.201 .231 7.401E-02 .099 .082 .745 .458 14.259 5.856 .254 2.435 .016 16.111 10.321 .138 1.561 .121 16.777 9.057 .174 1.852 .066 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Languagea. 50 Table 13: Regression Analysis – TN Public Schools Ending at Grade 6 – Value- Added Math Achievement as the Criterion Variable, Spring 2001 Model Summary .568a .322 .291 14.208 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. ANOVAb 14494.052 7 2070.579 10.256 .000a 30483.918 151 201.880 44977.970 158 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. Dependent Variable: Mathb. Coefficientsa 138.003 13.804 9.997 .000 -2.42E-03 .002 -.099 -1.093 .276 -1.28E-02 .006 -.165 -2.116 .036 -.179 .052 -.424 -3.429 .001 -1.09E-02 .058 -.018 -.188 .851 -3.258 3.418 -.088 -.953 .342 -6.408 6.024 -.083 -1.064 .289 -2.601 5.287 -.041 -.492 .623 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Matha. 51 Table 14: Regression Analysis – TN Public Schools Ending at Grade 6 – Value- Added Science Achievement as the Criterion Variable, Spring 2001 Model Summary .315a .099 .057 20.136 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. ANOVAb 6739.684 7 962.812 2.375 .025a 61222.793 151 405.449 67962.477 158 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. Dependent Variable: Scienceb. Coefficientsa 136.416 19.563 6.973 .000 -2.22E-03 .003 -.074 -.708 .480 -4.95E-03 .009 -.052 -.576 .565 -.152 .074 -.292 -2.050 .042 .112 .082 .153 1.359 .176 -9.646 4.844 -.212 -1.991 .048 -14.942 8.537 -.158 -1.750 .082 -10.582 7.492 -.135 -1.412 .160 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Sciencea. 52 Table 15: Regression Analysis – TN Public Schools Ending at Grade 6 – Value- Added Social Studies Achievement as the Criterion Variable, Spring 2001 Model Summary .442a .195 .158 19.038 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. ANOVAb 13285.990 7 1897.999 5.237 .000a 54726.560 151 362.428 68012.551 158 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR3, Enrollment, AR2, % F&RL, $/Pupil Expend., AR1, % Minority a. Dependent Variable: Soc. Studiesb. Coefficientsa 120.133 18.496 6.495 .000 2.206E-03 .003 .074 .744 .458 -1.34E-02 .008 -.140 -1.651 .101 -.159 .070 -.306 -2.276 .024 -.106 .078 -.144 -1.359 .176 .658 4.580 .014 .144 .886 3.023 8.071 .032 .374 .709 -4.596 7.084 -.059 -.649 .517 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Soc. Studiesa. 53 Summary – Schools Ending at Grade Six Among Tennessee public schools including grades one through five, but ending at grade six, no regression model of the variables in this study explains even one third of the variation in value-added achievement in any of the five subjects examined. Thus, the null hypothesis is not rejected and it must be concluded that the multiple correlation among the variables in this group is not significantly different from 0. In other words, variation in the achievement levels at these schools could have occurred as easily by chance as from the influence of AR use, per-pupil expenditure, school enrollment, percentage of students on free or reduced-price meals, or percentage minority students in the school. In math, however, and to a lesser extent in science and social studies, total school enrollment and percentage minority enrollment are negatively associated with value- added achievement. By far the strongest statistical relationship found in this study, as measured by the size of standardized beta coefficients, regardless of grade levels in the school, was between minority enrollment and math achievement. In fact, the Pearson Correlation Coefficient between the math achievement and percentage minority at these 160 schools (-.535) is significant at the .01 level. In other words, for schools ending at grade six, the higher percentage of minority enrollment in the school, the lower is likely to be the mean value-added gain in math. Schools that Include Grades One through Eight Among schools that include grade eight, per-pupil expenditure appears to have a significant positive relationship with reading achievement (Table 16), as does AR4 54 Table 16: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Reading Achievement as the Criterion Variable, Spring 2001 Model Summary .307a .094 .061 15.387 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. ANOVAb 5343.595 8 667.949 2.821 .005a 51377.274 217 236.762 56720.869 225 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. Dependent Variable: Readingb. Coefficientsa 81.485 14.429 5.647 .000 6.366E-03 .003 .166 2.314 .022 -8.27E-03 .005 -.136 -1.795 .074 .102 .065 .112 1.568 .118 -.115 .058 -.148 -1.990 .048 -1.965 3.243 -.052 -.606 .545 .144 6.303 .002 .023 .982 -4.004 3.276 -.098 -1.222 .223 36.459 15.756 .153 2.314 .022 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Readinga. 55 (Model School) status. A negative correlation is observed with percentage free and reduced lunch. AR use does not seem to be related to language achievement (Table 17) in these schools, but a slight negative correlation is observable between school size and lower socioeconomic status. The adjusted R square of .000 indicates that there is no discernable predictive validity between the variables in this model and math achievement (Table 18) in these schools. There simply is no relationship among these variables and science achievement (Table 19) in these schools. Likewise, the combination of variables in this model appears to have no predictive value for social studies achievement (Table 20) in schools ending at grade eight or above. Summary – Schools Including Grades One through Eight Only the regression analysis of value-added reading achievement yielded any findings approaching statistical significance for schools in this group. It appears that in schools where more money is spent per-pupil, fewer students are on free or reduced-price meals, and AR is most vigorously implemented, there is a slightly higher probability of increased value-added reading achievement. Analyses of All Schools Combined (Regardless of Ending Grade Level) When there were no apparent relationships of significance at the school level between value-added achievement and the factors of AR implementation, per-pupil expenditures, percentage of students on free or reduced-price meals, and percentage of 56 Table 17: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Language Achievement as the Criterion Variable, Spring 2001 Model Summary .241a .058 .023 21.991 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. ANOVAb 6473.352 8 809.169 1.673 .106a 104941.0 217 483.599 111414.3 225 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. Dependent Variable: Languageb. Coefficientsa 101.641 20.622 4.929 .000 3.922E-03 .004 .073 .998 .319 -1.35E-02 .007 -.159 -2.045 .042 6.740E-02 .093 .053 .728 .467 -.209 .082 -.193 -2.546 .012 -3.132 4.635 -.059 -.676 .500 -2.101 9.008 -.017 -.233 .816 1.365 4.682 .024 .291 .771 35.813 22.519 .107 1.590 .113 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Languagea. 57 Table 18: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Math Achievement as the Criterion Variable, Spring 2001 Model Summary .189a .036 .000 16.554 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. ANOVAb 2192.440 8 274.055 1.000 .437a 59462.768 217 274.022 61655.208 225 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. Dependent Variable: Mathb. Coefficientsa 107.319 15.523 6.913 .000 1.694E-03 .003 .042 .572 .568 -1.03E-02 .005 -.163 -2.074 .039 1.927E-02 .070 .020 .277 .782 -.158 .062 -.196 -2.556 .011 1.781 3.489 .045 .510 .610 4.213 6.781 .047 .621 .535 2.409 3.524 .057 .684 .495 -2.288 16.951 -.009 -.135 .893 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Matha. 58 Table 19: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Science Achievement as the Criterion Variable, Spring 2001 Model Summary .197a .039 .003 13.672 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. ANOVAb 1640.943 8 205.118 1.097 .366a 40564.286 217 186.932 42205.229 225 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. Dependent Variable: Scienceb. Coefficientsa 119.491 12.821 9.320 .000 -5.31E-05 .002 -.002 -.022 .983 -8.05E-03 .004 -.154 -1.967 .050 9.069E-02 .058 .115 1.576 .117 -9.76E-02 .051 -.146 -1.909 .058 -6.04E-02 2.882 -.002 -.021 .983 4.635 5.600 .063 .828 .409 -.815 2.911 -.023 -.280 .780 13.383 14.001 .065 .956 .340 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Sciencea. 59 Table 20: Regression Analysis – TN Public Schools Ending at Grade 8 or Higher – Value-Added Social Studies Achievement as the Criterion Variable, Spring 2001 Model Summary .200a .040 .004 17.146 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. ANOVAb 2645.370 8 330.671 1.125 .348a 63791.154 217 293.968 66436.524 225 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR3, AR2, % Minority, $/Pupil Expend., Enrollment, AR1 a. Dependent Variable: Social Studies %b. Coefficientsa 110.912 16.078 6.898 .000 1.698E-03 .003 .041 .554 .580 -3.51E-03 .005 -.054 -.684 .494 .124 .072 .126 1.715 .088 -.124 .064 -.147 -1.926 .055 -2.070 3.614 -.051 -.573 .567 3.207 7.023 .035 .457 .648 -3.015 3.650 -.068 -.826 .410 7.961 17.557 .031 .453 .651 (Constant) $/Pupil Expend. Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Social Studies %a. 60 minority students, it was determined to combine the schools without regard to the ending grade level to learn whether any significant correspondences could be noted among the variables of interest. First, multiple regression analysis was performed on the combined school list, looking for any significant contribution of the independent variables to variation in achievement. In the absence of statistically significant relationships between AR implementation and value-added achievement in any subject or grade level, much less the emergence of any useful prediction models, it was of interest to see whether there was any observable, if not statistically significant, difference in achievement, by level of AR implementation. Thus one-way ANOVA’s were performed to determine whether differences in the means of the value-added achievement scores by category of AR implementation were statistically significant at the .05 level. None of the mean differences were significant. However, the means of the achievement scores were charted by category of AR implementation so that the differences can be visualized. Consistent with earlier findings when schools were disaggregated by groups according to grade levels served, no effective prediction model emerges from the variables included to explain variation in reading achievement (Table 21). It is noted that minority enrollment and socioeconomic status are negatively associated with reading achievement. Within the context of the weak adjusted R square (.041) that excludes this model from explaining the variation, it may be seen that minority enrollment is negatively correlated to language achievement (Table 22). Although only about 10% (adjusted R square = .103) of variation in math achievement (Table 23) can be explained by this 61 Table 21: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value-Added Reading Achievement as the Criterion Variable, Spring 2001 Model Summary .200a .040 .029 20.457 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, Enr., $/Pupil, AR1, % Minority a. ANOVAb 12738.282 8 1592.285 3.805 .000a 306347.5 732 418.507 319085.8 740 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, Enr., $/Pupil, AR1, % Minoritya. Dependent Variable: READINGb. Coefficientsa 93.470 9.174 10.188 .000 3.239E-03 .001 .102 2.253 .025 -1.86E-03 .004 -.021 -.511 .610 -8.68E-02 .034 -.139 -2.517 .012 -9.28E-02 .038 -.115 -2.464 .014 -1.363 2.578 -.026 -.529 .597 1.115 4.787 .010 .233 .816 -6.325 4.002 -.071 -1.580 .114 2.702 9.494 .011 .285 .776 (Constant) $/Pupil Enr. % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: READINGa. 62 Table 22: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value-Added Language Achievement as the Criterion Variable, Spring 2001 Model Summary .226a .051 .041 24.441 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, Enr., $/Pupil, AR1, % Minority a. ANOVAb 23542.464 8 2942.808 4.926 .000a 437278.5 732 597.375 460820.9 740 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, Enr., $/Pupil, AR1, % Minoritya. Dependent Variable: LANGb. Coefficientsa 107.696 10.961 9.826 .000 -1.21E-03 .002 -.032 -.705 .481 -1.44E-03 .004 -.014 -.332 .740 -.120 .041 -.160 -2.911 .004 -4.63E-02 .045 -.048 -1.029 .304 -4.677 3.080 -.075 -1.519 .129 .113 5.720 .001 .020 .984 4.227 4.781 .040 .884 .377 .332 11.343 .001 .029 .977 (Constant) $/Pupil Enr. % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: LANGa. 63 Table 23: Regression Analysis – All TN Public Schools Including Grades 1-5 – Value-Added Math Achievement as the Criterion Variable, Spring 2001 Model Summary .336a .113 .103 17.895 Model 1 R R Square Adjusted R Square Std. Error of the Estimate Predictors: (Constant), AR4, % F&RL, AR2, AR3, Enrollment, $/Pupil, AR1, % Minority a. ANOVAb 29907.377 8 3738.422 11.675 .000a 234401.8 732 320.221 264309.2 740 Regression Residual Total Model 1 Sum of Squares df Mean Square F Sig. Predictors: (Constant), AR4, % F&RL, AR2, AR3, Enrollment, $/Pupil, AR1, % Minority a. Dependent Variable: Mathb. Coefficientsa 113.724 8.025 14.171 .000 1.525E-03 .001 .053 1.213 .226 -5.88E-03 .003 -.074 -1.848 .065 -.110 .030 -.193 -3.635 .000 -.162 .033 -.221 -4.907 .000 -2.566 2.255 -.055 -1.138 .255 -2.685 4.188 -.026 -.641 .522 -2.057 3.500 -.025 -.588 .557 -6.296 8.305 -.027 -.758 .449 (Constant) $/Pupil Enrollment % Minority % F&RL AR1 AR2 AR3 AR4 Model 1 B Std. Error Unstandardized Coefficients Beta Standardi zed Coefficien ts t Sig. Dependent Variable: Matha. 64 combination of variables, clearly it is much more related to minority enrollment and socioeconomic status than to AR. Science achievement (Table 24) does not appear to be related in any significant way to any of the variables in this study. Although this model, like previous attempts, fails to explain a significant amount of the variation in social studies achievement (Table 25), it is worth noting that per-pupil expenditure and vigorous A