Do value-added methods level the playing field for teachers?

Highlights

Value-added measures partially level the playing field by controlling for many student characteristics. But if they don’t fully adjust for all the factors that influence achievement and that consistently differ among classrooms, they may be distorted, or “confounded”.
Simple value-added models that control for just a few tests scores (or only one score) and no other variables produce measures that underestimate teachers with low-achieving students and overestimate teachers with high-achieving students.
The evidence, while inconclusive, generally suggests that confounding is weak. But it would not be prudent to conclude that confounding is not a problem for all teachers. In particular, the evidence on comparing teachers across schools is limited.
Studies assess general patterns of confounding. They do not examine confounding for individual teachers, and they can’t rule out the possibility that some teachers consistently teach students who are distinct enough to cause confounding.
Value-added models often control for variables such as average prior achievement for a classroom or school, but this practice could introduce errors into value-added estimates.
Confounding might lead school systems to draw erroneous conclusions about their teachers – conclu sions that carry heavy costs to both teachers and society.

Introduction

Value-added models have caught the interest of policymakers because, unlike using student tests scores for other means of accountability, they purport to “level the playing field”. That is, they supposedly reflect only a teacher’s effectiveness, not whether she teaches high- or low-income students, for instance, or students in accelerated or standard classes. Yet many people are concerned that teacher effects from value-added measures will be sensitive to the characteristics of her students. More specifically, they believe that teachers of low-income, minority, or special education students will have lower value-added scores than Comment [LQ1]: Note: Won’t have any of these at the time of the website launch, so this page will be commented out Comment [LQ2]: Link downloads PDF copy of the knowledge brief Comment [LQ3]: Include link to webinar page for this knowledge brief

equally effective teachers who are teaching students outside these populations. Other people worry that the opposite might be true — that some value-added models might cause teachers of low-income, minority, or special education students to have higher value-added scores than equally effective teachers who work with higher-achieving, less risky populations.

It is very challenging to differentiate between a teacher’s contribution to student achievement and the influences of the school, the classroom, and a student’s peers.

Practical Implications: How Does This Issue Impact District Decision-Making?

Because the evidence on confounding by student background variables is mixed, those who make decisions about teachers should allow for its potential to occur. Confounding can lead to serious errors with heavy consequences for teachers and society: ineffective teachers may be deemed effective and vice-versa. So, districts need accurate measures of these errors, and they must consider them if they plan to use value-added measures for high-stakes decisions.

Works Cited

Anthony S. Bryk, Penny Bender Sebring , Elaine Allensworth , Stuart Luppescu , and John Q. Easton, Organizing Schools for Improvement: Lessons from Chicago, (Chicago: University of Chicago Press, 2010).