Education Consumers Foundation
Value-Added Achievement Awards

Introduction

Every citizen has a stake in school quality, especially parents. The purpose of these awards is to make school quality visible to all and to give those who are responsible for producing it the recognition they deserve.

All states collect, analyze, and report mountains of school performance data. However, most of this information is not easily accessed or understood by non-educators. Our aim is to strengthen the public’s understanding of school performance in Tennessee by distilling and presenting school achievement data in a clear and simple way. We want education’s consumers to see for themselves which schools are doing the best job of boosting student achievement, and our School Performance Charts allow consumers to do just that.

Schools teach many subjects, but no school can be called successful if it does a poor job in math and reading; for this reason, our reports and rankings are based exclusively on school performance in these two core subjects. And we’re happy to report that there are a number of schools that perform extremely well: schools like Amqui Elementary School, a Title I school in Nashville, are generating incredible results. We hope that the recognition earned by the leaders of exceptionally effective schools like Amqui Elementary will prompt other schools to imitate their success. Tennessee’s public education system could top the nation if all of its schools performed as well as our award winners.

The Education Consumers Foundation chose to inaugurate the Value-Added Achievement Awards in Tennessee because of the sophistication and maturity of Tennessee’s educational accountability system. Ten years before the federal No Child Left Behind Act (NCLB), Tennessee enacted TVAAS—the Tennessee Value Added Assessment System. As other states adopt similarly sophisticated data collection systems, the Foundation plans to extend its initiative to reward excellence wherever it is found.

Value-added assessment is a term taken from economics; it refers not to a student’s level of achievement but to the year-to-year increase in a student’s achievement. The schools with the highest value-added scores are the ones doing the best job of maximizing the talents and capabilities of their students.

The great virtue of TVAAS is that it accurately reports annual improvement regardless of students’ beginning test scores or social, economic, ethnic, or other background differences. By leveling the playing field, it permits users to compare the student achievement gains produced by different teachers, schools, and school districts regardless of the advantages and disadvantages had by their students. We were able to compare Tennessee’s schools and select the highest performers because of TVAAS’s unique statistical strengths.

TVAAS remains the most sophisticated educational accountability system in use today. It is currently being implemented in Ohio, New York, and Pennsylvania; and it is under consideration as a national model. For more information on the technical aspects of TVAAS, visit:
http://www.education-consumers.com/trustworthy_research.cfm#valueassessment

How the Award Winners Were Determined
By extracting and analyzing the data available via the Tennessee Department of Education's website, the Education Consumers Foundation identified the principals of the first, second, and third highest performing elementary and middle schools in Tennessee's East, Middle, and West Grand Divisions. Our analysis was based wholly on publicly available data. It is objective, transparent, and replicable.

Teachers and parents are the tip of the educational spear, but schoolwide achievement gains depend on teamwork led by a principal. Our purpose was to identify the principals whose schools excel at helping children to “be all they can be” regardless of their backgrounds or previous achievement levels. Only principals who have served in their present position for five or more years were eligible.

In Tennessee and most other states, a wide variety of grade configurations is found among elementary and middle schools. There are elementary schools that range from pre-kindergarten to grade 4 and others that cover grades K through 8. There are middle schools that go from grades 5 or 6 to grade 8. Some include only grades 7 and 8. Because of these differences, classification as a middle or elementary school is to some extent arbitrary.

In consultation with the architect of TVAAS, Dr. William Sanders, we concluded that combining schools into two groups—grades 4-6 (elementary schools) and grades 6-8 (middle schools)—would result in the fairest comparisons. Schools with grades in both groups (e.g., traditional K-8 schools) were included in the middle school group if they had two or more of grades 6-8.

We chose to focus on two of the four subject areas that are tracked by Tennessee's educational accountability system: Reading/Language Arts (i.e., reading and writing) and Math (i.e., arithmetic). Public schools aim to teach much more than basic skills, but basic skills are indispensable. Students who fail to learn them are handicapped for life. Schools that fail to teach these subjects cannot be described as effective regardless of their other strengths.

School Rankings: A Step-By-Step Guide
The data on which we relied is publicly available through the Tennessee Department of Education's (TNDOE) website: http://www.state.tn.us/education. Thanks to the work of Dr. Sanders and the farsighted support of Governor Ned Ray McWherter and the Tennessee General Assembly, Tennessee has been tracking value-added achievement gains by student, teacher, school, and school system for over 10 years. Only school and system data are public. The TNDOE also collects student and teacher-level data, but that information is available only to individual teachers and school system administrators.

The TNDOE provides a wealth of school-by-school data in addition to value-added reports. Much of it, however, is meaningful primarily to educators. For a more complete explanation of how Tennessee's Report Card may be interpreted, see TNDOE's Glossary at http://www.k-12.state.tn.us/rptcrd05/rptcrdterms.htm and the Help tab at the top of each page.

In the interest of transparency, the following steps outline the process used to gather and analyze the data on which the school performance charts and awards are based. Interested parties may replicate our results by repeating these steps.

Step 1
On the home page of the TNDOE website, see the navigation bar on the left and click on Report Card. A page offering access to the State's school report cards is opened. The full report card is a four-part document that primarily includes statistical analyses of test results and demographic data. The Report Card includes a summation of the TVAAS results provided by the Tennessee Department of Education. It may be viewed by using the menus in the center of the page, and can be used to review summary data for the entire state, individual districts, or individual schools.

The detailed TVAAS data on which the Value-Added Achievement Awards are based, however, can be accessed only by opening the TVAAS Public Site link at the bottom left of the page. Note: The detailed TVAAS data for a given year is available online only until the next year’s data is posted.

Step 2
The TVAAS Public Site button opens a page that contains a menu listing all Tennessee school systems. In order to view the data on which the Value-Added Achievement Awards are based, move the cursor to the Reports tab at the upper right and click on School Value Added. Next, move to the popup and select the System and the School of interest within the system.

The page that opens is titled: TVAAS School Report for X School in X County System, TCAP CRT Math. It contains both the value-added achievement data and the TCAP achievement test averages in math for your selected school. To see the value-added achievement data and TCAP results for the same school in reading/language arts, move the cursor to the Subjects tab at the top of the menu and click Reading/Language.

The TVAAS Restricted Site is available to teachers and administrators only. It provides the classroom and student level data necessary to pinpoint curriculum, teaching, and learning problems.

Step 3
In the Estimated School Mean NCE Gain section of the report (i.e., the value-added achievement gain section), see the row headed 3-Yr-Avg NCE Gain and select the relevant grades (scores available between grades 4-8) for math and then, by switching reports, for reading/language Arts.

These are the basic data on which the Value-Added Achievement Awards are based. They are the average gains in math and reading/language arts for the most recent 3 years. Note that schools lacking 3 full years of data for all grades offered by the school were not included in the present analysis.

To learn more about the terminology used in the TVAAS reports, see the Glossary of Terms and Abbreviations. It is provided in addition to the help files available through the TVAAS site and the information furnished by the TNDOE.

Step 4
The 3-Yr-Ave NCE Gain for each grade and each school was pulled from the individual school reports and incorporated into a distribution of like grades from all other schools (regardless of their middle or elementary school classification). The resulting distributions (i.e., 5 math and 5 reading distributions for each of grades 4-8) were “normalized” by dividing the difference between each 3-Yr-Ave NCE Gain and the distribution mean by the distribution’s standard deviation.

The product of this conversion process was 10 z score distributions (i.e., distributions with a mean of zero and a standard deviation of 1)–five in reading/language arts and five in math. The normalized scores in math and reading for each of the grades contained within a given school were then averaged into a single Composite Math/Reading Score on which schools were ranked.

It should be noted that normalization was statistically necessary to insure that each component would be weighted equally and that all schools be afforded equal treatment regardless of their grade configuration. It differs from the methodology used by the TNDOE to calculate its Mean NCE Gain over Grades Relative to [the] Growth Standard (see the second from right column of each school’s TVAAS School Report). The Mean NCE Gain over Grades is the statistic on which the TNDOE letter grades for school performance in each subject are based (see http://www.k-12.state.tn.us/rptcrd06/gradescale.htm).

The Composite Performance Grades and the Growth Standard were estimated from the Average Normalized Equivalents of the TNDOEs Mean NCE Gain over Grades in math and reading/language arts.  The percentile rank of each letter grade and that of the Growth Standard were used to set letter grade boundaries and the minimum expected achievement line on the chart.  The percentage of each letter grade tracks the aggregate percentages of the grades assigned by the TNDOE.

Step 5
The distribution of normalized scores for elementary schools and middle schools are displayed in two interactive charts. The charts permit users to view all schools in Tennessee ranked according to their effectiveness in producing student achievement gains. It also shows their performance referenced to Tennessee’s Growth Standard—i.e., the midline representing the minimum expectation for annual achievement growth that was set in 1998—and to Composite Performance Grades that are described above. The gap between the Growth Standard and the Current State Average reflects the increase in average student achievement gain that has taken place since 1998.

Local schools and school systems can be highlighted and all can be viewed in comparison to the Value Added Achievement Award winners. Customized wall charts with winners and local schools highlighted can be printed.