Alarm System Monitored

Is there a place for performance-based teacher compensation in our public school system?

Performance-based pay will and should play a more prominent role in our public schools. But the form of performance-based pay envisioned by its proponents'merit pay which explicitly links individual teachers compensation to standardized tests scores'is a wrong-headed, technically infeasible proposal that will take our schools in the wrong direction.

Other Knol Debates

Closing Argument

Here are our responses to the assertions put forth by Mr. Lips in his rebuttal argument.

1. It is true that standardized tests are an important tool for assessing student learning - however, their use in teacher compensation is more tricky due to the concerns we raised in our earlier posts. These include, but are not limited to, the difficulty of separating teacher value-added from family effects, inconsistency of these value added effects from year to year (high degree of noise'), and the problems caused by non-random assignment of students to teachers by principals. It is imperative that we have a better understanding of these issues before we rush headlong into a standardized test score based merit pay regime. Note that these same problems are likely to persist if the bonuses are based on performance in AP tests rather than performance in state tests, as Mr. Lips suggests. In fact, any merit pay regime that bases teacher salaries on student performance in some form of standardized tests, be it district, state or national tests, raises the same concerns.
2. As we pointed out earlier, school-wide bonuses might reduce the extent of competition among teachers belonging to the same school, as they are not competing with each other for the limited awards. However, problems of free-riding - where some teachers decide to piggyback on the efforts of their colleagues in the school instead of expending more effort themselves - are likely to be more severe for awards at the school level rather than the individual level. Further, because some schools might be better due to extraneous factors - for example, for being located in a high-income neighborhood where parents are more educated and more motivated - a system of merit pay based on overall school performance is likely to mistakenly reward more affluent schools rather than more effective schools. (Recent research by Jesse Rothstein at Princeton suggests that school effectiveness is not a primary determinant of parental choices, though the reason can be either that parents cannot distinguish effective from ineffective schools, or that parents prefer other neighborhood or school attributes to effectiveness, or that the particular variation in effectiveness, as defined in his study, is not an important determinant of student performance.)
3. While principals should have some discretion over the staff they oversee, we should be concerned about the potential for favoritism and abuse of power in such cases (e.g. a teacher critical of her principal being assigned to a difficult class which adversely affects her value added estimates). It is interesting to note that one of the main rationale behind the institution of a single salary schedule was the desire to address such concerns about abuse of discretionary power.

Everyone agrees that there should be some real change in the way we pay our teachers. Teachers have been at a relative disadvantage in terms of salaries, compared to similar professions, for years now and have still been losing ground (see here and here). However, tinkering with salaries of a select few at the top - based on untested and volatile criteria which are incomprehensible to most teachers - will not solve the problem. As Richard Rothstein, Rebecca Jacobsen and Tamara Wilder point out in their important new book, Grading Education, ill-thought-out accountability programs has led to unintended consequences - narrowing the curriculum, misidentifying both failing and successful schools and establishing irresponsible expectations for what schools can accomplish. An ill-thought-out merit pay program for teachers will have similar unintended consequences and unlikely to improve the educational attainment of our poor and minority children.

Rebuttal Argument

Resist the Urge

In his opening argument, Dan Lips acknowledges-rightly-that the evidence on performance-based pay for teachers is slim. He encourages greater experimentation with alternative pay strategies, and on this we can agree. We are, however, much less optimistic than Lips about their likely effects on our system of public education.

To be sure, teacher pay linked to test scores has a deceptively simple, common-sense appeal. Test scores are an indicator of student learning, so why shouldn't we reward teachers for improving these test scores? The reality is much more complex, for reasons we highlighted in our own argument:

(1) isolating individual teachers' role in improving (or not improving) test scores is exceptionally difficult, even on narrowly-defined outcomes

(2) standardized tests represent only a small fraction of what we expect our students to know and do, and

(3) few agree on what the "output" of our education system should be.

Lips is careful not to over-hype the as-yet-unknown benefits of performance pay for teachers. Yet like many advocates of education reform, he is more than willing to attribute low public school test scores to the lack of performance pay: "what is clear now is that the current system of teacher compensation used in most school districts isn't producing excellence in student learning."

If we know little about how teacher compensation systems relate to teacher quality and student achievement, how can we assume they are the source of the problem? Variations on the "single salary" schedule exist in nearly 15, 000 local school districts, and in a preponderance of private schools. Some of these districts are high performing, some very low. Many have unionized teachers, many do not. Without more evidence of their role in student learning, teacher pay systems are simply another educational scapegoat, another blunt tool for reform.

In the coming years, we will know more about how performance pay plans impact students, teachers, and schools. Countless "experiments" are already underway around the country: Texas, Florida, and Minnesota have operated merit pay programs for several years. New York City announced in 2007 it would implement a $20 million teacher bonus program-the largest in the nation-while in Washington D.C., Chancellor Michelle Rhee recently offered merit-based raises to teachers in exchange for job security.

These experiments will surely improve our knowledge of performance pay systems in education. But policymakers and the public need to have an honest conversation about how and when we should consider these experiments a "success." We would be surprised if paying teachers for test score improvements did not raise scores on those tests. But we would also be surprised if schools and teachers accomplish this goal without diverting time and resources from other valued activities that are not on the test. Advocates of performance pay systems (and accountability in general) reply to this concern with "just offer a test worth teaching to." But as psychologist Dan Koretz shows in his excellent book Measuring Up, this is much easier said than done.

Let's assume for a moment that we do have a reliable "test worth teaching to" that captures all of our goals for public education. Even in this ideal world, it is exceptionally difficult to isolate a teacher's unique contribution toward these goals (and relying on "value-added" will not solve the problem). As we had pointed out in our original post, these include, but are not limited to, issues with robustness and non-transparency, the difficulty of separating out teacher and school contributions from family contributions, the imprecision of estimated teacher effects and the effects of non-random assignment of students to teachers by principals. For a nice discussion of many of these issues, see this blog post by eduwonkette.

Lips claims that a merit pay system just needs to be "well-designed" to avoid potentially damaging competition among teachers. However, this is easier said than done. While school level bonuses, as opposed to individual level bonuses, can mitigate concerns about fraternal competition, they also should dilute the incentives that teachers face and that supposedly leads teachers to exert extra effort in the first place. This is because with school level bonuses free-riding becomes an issue, unless administrators and teachers are spying on each other.

Finally, Lips' claim that "in many professions, tying an employee's compensation to his or her performance is a fact of life" is certainly true to some degree. But the kind of pay system that directly ties salaries to individual output (such as classroom test scores) is only widely found in occupations like sales or manufacturing, where output is easily measured and can easily be traced back to the individual. Stock options for CEOs are designed to link compensation with performance, but even there-with millions of dollars at stake-little to no attempt is made to isolate an executive's unique contribution to profitability.

Until more is known about the consequences of merit-based pay, Congress, school districts and the states should resist implementing these plans on a wider scale.

Original Post: Is there a place for performance-based teacher compensation in our public school system?

Performance-based pay will and should play a more prominent role in our public schools. But the form of performance-based pay envisioned by its proponents-merit pay which explicitly links individual teacher's compensation to standardized tests scores-is a wrong-headed, technically infeasible proposal that will take our schools in the wrong direction.

The recent emphasis in Washington and our state capitals on attracting and retaining high-quality teachers into the profession is well-founded. Recent research has found that teachers are the most important resource schools contribute to their students' educational outcomes, and public policymakers have responded appropriately by seeking new ways to keep talent in the classroom. The latest such proposal is the use of performance-based, or "merit" pay for teachers. These plans come in many flavors, but the one most prominently being advanced is the tying of an individual teacher's salary to changes in her/his students' standardized test performance. Commentators argue that such a link will not only incentivize teachers to focus on student outcomes, but will also radically transform the teacher labor market by drawing more talent into teaching. In our view, they are wrong, for the following reasons:

--Performance pay systems work best when they are transparent and tied to metrics that are easily understood by teachers and parents. Standardized "bubble-sheet" math tests may meet this criterion, but most educational priorities are not so easily distilled into single metrics.

Most would agree that competency in reading and mathematics are critical outcomes of the educational process, but that writing ability and knowledge of science, history, economics, and civics are equally important. Parents expect schools to foster creativity, increase understanding of good health practices, and form an appreciation for art, music, and literature.

-- Even if consensus were reached on a set of test scores that adequately capture goals for the education system, it is exceptionally difficult to isolate teachers' individual contributions toward those goals. Student performance is determined by a long list of factors: families, schools, communities, peers, teachers, and the students themselves. A proper performance pay system would not reward or punish teachers for forces outside of their control, but rather recognize their contribution net of these factors (their "value-added"). Social scientists and statisticians have developed sophisticated methods for estimating value-added, but these methods are complex, non-transparent, and highly imprecise. For example, it is difficult if not impossible to separate teacher value-added from family effects, or the effects of non-random assignment of students to teachers by principals. In practice it has often been the case that the statistical confidence interval around a teacher's value-added score is very large, rendering the information of very limited use: that is, the estimate of a teacher's contribution could not distinguish whether she/he was way above average or considerably below average.

-- Schools also lack the basic data to support merit pay: most teachers are not in subjects or grades where tests are available; tests are not given at the beginning and end of the year so as to measure a teacher's performance; and many schools do not have data that follows students over time, an essential ingredient. So, it is curious to see merit pay discussed as if it can be implemented anytime soon.

-- Though supporters claim that performance-based pay would radically transform the teacher labor market by attracting and retaining the most effective teachers, this is unlikely to be the case in practice. Existing teacher salaries are already quite low relative to those in other professions, and relative salaries have been slipping over the last two decades. Under these circumstances, tinkering with salaries at the top by offering a select few the opportunity to earn extra income is unlikely to significantly affect the overall talent pool in teaching.

Putting aside the relatively low salaries in teaching, proponents still argue that the "single salary" schedule in teaching discourages excellence and skilled workers who are better rewarded in other sectors of the economy. But in fact, an analysis of merit pay in the private sector by Scott Adams and John Heywood of the University of Wisconsin - Milwaukee found that few private sector workers are paid in a direct formulaic way according to output. Such pay systems are most prevalent among production and sales workers and in other cases where a relatively well-defined measure of individual output exists.

-- Finally, the heavy reliance on narrow quantitative indicators in performance pay systems almost inevitably undermines its original intentions. In 1979, famed social scientist Donald Campbell framed what he called a "law" of performance measurement: The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. In recent work, Richard Rothstein has examined other sectors' experiences with accountability systems and finds widespread examples of such corruption, including goal distortion and the subversion of intrinsic motivation. Performance pay at the individual level may significantly compromise the collegial atmosphere at schools, with teachers competing with their colleagues for a limited set of rewards.

Proponents of No Child Left Behind argued that with strict accountability, "failing" schools would feel tremendous pressure to expend more effort and student achievement would dramatically improve. However, subsequent events have belied that optimism - the growth in test scores in national NAEP tests has been no higher in the post-NCLB period than in the previous decade. Ratcheting up accountability by mandating merit pay systems-supposedly without the consent of teachers or their representatives- will not fare any better.

Opposing Argument