A "Second Opinion" Regarding Implementation of the Nebraska Partnership for Quality Education Project

September 9, 1999

George K. Cunningham, Ph.D.
College of Education
University of Louisville

J.E. Stone, Ed.D.
College of Education
East Tennessee State University

Martin Kozloff, Ph.D.
Watson School of Education
University of North Carolina - Wilmington

all of the Education Consumers Consultant Network

At the request of Ann Mactier--member of the Nebraska Board of Education--the Nebraska Partnership for Quality Education proposal was examined by the above members of the Education Consumers Consultant Network. The Education Consumers Consultants Network is a network of education professionals associated with the Education Consumers ClearingHouse <www.education-consumers.com> and the report is a product of its "Second Opinion" service--an independent consultation service that serves education's consumers exclusively.

The material reviewed was presented to the Board at an August 1999 meeting and action is to be taken at the Board's September 1999 meeting. These time limits posed substantial limitations on the nature and extent of the following analysis.

Overview

The commitment and effort that went into the writing of the NDE proposal is evident. The authors of the proposal are to be commended for successfully securing the participation of the most important organizations involved in teacher education in the state of Nebraska.

The proposal includes a series of strategies that have been developed to improve programs for educating teachers. It is the purpose of this report to consider those strategies with regard to their likely effectiveness and to make suggestions about possible alternatives or additions where appropriate.

This is a conventional proposal and the strategies that are recommended to improve teacher education are similar to those that might be proposed in other states. From that perspective, this is a safe document. If the Nebraska State Board of Education is fairly satisfied with the quality of its teacher education programs and the present level of student achievement, the proposed modifications to the Nebraska pre-service teacher education may result in a satisfactory level of small improvements. On the other hand, if the Board's primary aim is to significantly improve the ability of Nebraska teachers to bring about measured academic achievement, the proposed strategies are unlikely to be adequate and for the following reasons:

Findings

1. The parties who will implement the Nebraska Partnership for Quality Teacher Education Project (hereinafter called the Project) are the same institutional and organizational stakeholders who are responsible for the current state of teacher training in Nebraska. They are assessing themselves, they are proposing to reform themselves, and they plan to assess and report the outcome of their efforts. By contrast, it is standard practice within the business community for independent outside auditors and examiners to review and report the performance of business organizations. Too, outside leadership is typically required to produce any real change--especially in instances where the quality of organizational performance is in question.
   
2. One of the inherent limitations of an attempt by stakeholders to reform and improve themselves is that truly fresh ideas are very unlikely. Stakeholders have accustomed ways of conducting business. They have vested interests as well. Under the proposed plan, there is a very great likelihood that any changes will end up being quite similar to existing practice. For example, the new "performance based" NCATE Standards that the Partnership proposes to adopt are entirely consistent withthe current programs standards that now govern most teacher education programs in Nebraska. They are also entirely consistent with the pedagogical concepts now popular among teacher educators and, significantly, there is little evidence to suggest that teachers trained in programs under either the current or the new standards do a superior job of boosting student achievement. If the Board wants new ideas about how teachers should be trained, it would be well advised to open up teacher training educators from outside the Partnership.
   
   One way to open teacher training to fresh ideas would be to permit schools to train their own teachers. As schools become increasingly accountable for student achievement, they take an increasing interest in insuring that their teachers are equipped with demonstrably effective teaching methodologies. For example, the Chicago Public Schools found that schools facing the prospect of being reconstituted for lack of academic progress managed remarkable improvements with the help of result-oriented private sector consultants.
   
   Given that the Project has funding available for in-school mentoring programs, the Board might consider assisting local school districts in hiring consultants who could train and mentor individuals who have a bachelor's degree in the liberal arts. The Houston (TX) Independent School District, for example, has an alternative teacher certification program in which such individuals can enter teaching by completing a one year internship, passing certain basic exams, and taking 12 credit hours of college work. The same consultants would be able to provide professional development training to inservice teachers. Such a program would produce local personnel who are trained in locally used methods and trained for known local job openings--an attractive alternative to the present to programs that train numerous teachers in educational specialities for which there is an existing oversupply.
   
   Without competition from outside the Partnership, neither the Board nor local schools are likely to learn whether there are more effective forms of training. Neither are they likely to learn whether there are less time consuming and less costly forms of training.
   
   
3. The primary objective of the law under which the Project was funded--Title II of the Higher Education Act of 1998--is improved student achievement. The present Project calls for improving teacher training but it is not clear that the proposed improvements will increase student achievement. Rather the Project's "overall purpose" is simply said to be the improvement of teacher education. "Improved teacher education" could mean improved student achievement; but as a recent study by Public Agenda suggests, there may be very significant differences between the Board's understanding of improved teacher education and the teacher training community's view. Noted on page 31 of the Project proposal, Public Agenda's Different Drummers: How Teachers of Teachers View Public Education (Farkas & Johnson, 1997) not only found differences between teacher-educators and the public but "great disparities" between teacher-educators' views and those of K-12 teachers.
   
   The meaning of "improved teacher education" is not the only expression that might be understood very differently by the Board and by the teacher-education community. Two recent opinions by the Nebraska Attorney General found several of the terms used in the standards around which the Project is built--i.e., the standards of the National Council for the Accreditation of Teacher Education and those of the Interstate New Teacher Assessment and Support Consortium--to be "unconstitutionally vague." In light of these findings alone, the Board would be wise to insure that its understanding of the nature and aim of the present Project is shared by its various participants. For example, would participants agree that the primary objective of the Project is to strengthen the ability of newly trained teachers to increase objectively measured student achievement?
   
   
4. The differences in the vision of education had by teacher-educators and the public, and the vagueness of terms such as "improved teacher education" are indirect reasons why the Board should seek to clarify the aims of the proposed Project. The Project's emphasis on increasing "diversity," however, is a more direct indication that its top priority may not be improvements in measured academic achievement.
   
   The proposal approved by the U.S. Department of Education appears dedicated to the proposition that increased "cultural" and "ethnic" diversity is key to improved teacher quality. The Project asserts that "Many research studies suggest that students of color benefit significantly from teachers of their [own] cultural background" (page 6). This quote can easily be understood to mean that there is sound empirical evidence demonstrating a significant relationship between the ethnic or cultural background of a teacher and the academic success of "students of color." In fact, however, a review of the National Association of State Boards of Education document on which this claim is based reveals that the expected benefits are outcomes such as closer ties with local racial/ethnic communities and similar matters, not increased academic achievement. In other words, the premise on which so much of the Project appears to be based--i.e., that improved teacher diversity will substantially benefit students--turns out to be a reference to non-academic benefits.
   
   The non-academic benefits that might be derived from greater teacher diversity may have substantial merit but they are not reflective of the public's (or likely the Board's) primary expectation with regard to schooling. Rather, as repeatedly affirmed by public opinion studies, schooling that fails to improve student achievement is not considered acceptable regardless of whatever else it is said to produce.
   
   
5. Given that there may be a discrepancy between the Board's aims and those intended by the Project, the Board would be well advised to carefully define the manner in which the Project's expected results are measured. By doing so, the Board would have some assurance that whatever the vagaries of interpretation and implementation, outcomes would be consistent with Title II's intended purposes.
   
   Presuming that its top priority is improved student achievement, the Board may want to consider assessing the effectiveness of newly trained teachers by measuring the achievement gains of their students during their first year of teaching. A methodology for assessing such gains called the Tennessee Value Added Assessment System (TVAAS) has been used in Tennessee since 1993. Other states and districts are either considering it for use or have adopted it on a limited scale. TVAAS provides teachers, principals, and superintendents annual reports of the increases in student achievement exhibited by the students for which a teacher was responsible during the preceding year. TVAAS uses a sophisticated statistical methodology to calculate an estimate of teacher effectiveness that is much like the estimates teachers informally make of themselves, i.e., it compares the increase in achievement made by students in the current year to the record of increases made by the same students in previous years. In essence, TVAAS defines excellent teachers as those who help their students to improve the most. By contrast, standards-based systems of teacher and school assessment compare year-end student achievement levels to a fixed standard regardless of how much students had to gain in order to reach the standard. With fixed standards some students may make important gains without reaching the standard and others may reach the standard without much gain.
   
   Both fixed standards and value-added gain scores are necessary to a complete assessment of teacher and school performance. Fixed standards tell policymakers about the quality of the finished product. They are appropriately used to judge whether a student should be promoted to the next grade or graduated. Value-added scores tell them how well teachers and schools are working to produce a high quality product. Of particular importance with respect to the present Project, the value-added scores of novice teachers are telling indicators of how well they have been trained to produce achievement.
   
   The use of value-added assessment as the prime indicator of a novice teacher's classroom effectiveness would eliminate many of the uncertainties associated with the proposed Project. No matter how pedagogical terms might be interpreted or programs implemented, those teachers who were successful in increasing achievement would be identified as effective and those who were unsuccessful would not be so identified--no matter how many courses taken or no matter what their scores on tests of pedagogical knowledge.
   
   In Tennessee, value-added assessment system is used in all public schools. It is a significant (but not unreasonable) expenditure but it provides statewide public school accountability. Value-added assessment employed only for Nebraska's novice teachers would necessitate analysis of existing student achievement test data but only for those students taught by novice teachers--a very modest expense. The advantage of such a program would be that policymakers and school administrators would not have to rely on measures of teacher preparedness such as classroom observations by teacher-educators or scores on tests that have an uncertain relationship to student achievement. Instead they would be able to assess teacher preparedness for themselves.
   
   
6. If the Board were to decide in favor of using value-added assessment instead of developing or adopting a new test of pedagogy--as urged by the NDE proposal--the overall cost of assessing teacher preparedness would likely be in the range contemplated by the present proposal. Also, if the Board were to use an off-the-shelf examination of teacher content knowledge--the PRAXIS, for example--a net reduction in the cost of teacher assessment might be achieved. Both of these possibilities would need to be explored beyond the current report.
   
   Information regarding value-added assessment may be obtained from Dr. William Sanders of the Value-Added Research and Assessment Center at the University of Tennessee. Information regarding the PRAXIS examination may be obtained from Educational Testing Service.

Summary

The Nebraska Partnership for Quality Education has assembled a conventional proposal for NDE Grant Proposal. The Education Consumers Consultant Network was asked to render a second opinion about this project. The perspective taken in this second opinion was to evaluate this proposal from the viewpoint of the "consumer" rather than the teacher education community. In general, the NDE proposal places a great deal of emphasis on the importance of modifying the process of teacher education. Less emphasis is placed on the outcome. Since a primary role of teachers Nebraska is to increase the achievement of the Nebraska Academic Content Standards, it would seem that a focus on outcomes rather than process would be a better choice for evaluating the effectiveness of teacher preparation in this state. Value-added methodology was presented as an objective and demonstrably effective way of accomplishing this purpose.

The adoption of a value-added strategy for measuring teacher preparedness would entail expense not anticipated by the proposal but any added expense would be offset by the lessened need for the development of new tests. Off-the-shelf tests of the content are also available. The cost of developing new tests both in actual expenditures and in the difficulty of achieving a consensus regarding the content of these tests may render them unworthy of the effort. If the focus of the Project were shifted to student achievement gains, the impact of the proposed reforms would become far more visible to the Board and to the public than is otherwise likely.

 

J. E. Stone, Ed.D.
Education Consumers ClearingHouseTM
P.O. Box 4411
Johnson City, TN 37602
phone & fax (423) 282-6832
professor@education-consumers.com