VAM and the law of unintended consequences

Results-oriented teacher evaluations are creating controversy over the publicly releasing names of teachers who generate high as well as low value-added test score gains. Join an education expert who argues that the high value-added error rate makes sharing these results irresponsible and dangerous.

David Leonhardt’s New York Times Magazine piece on teacher accountability offers a reasoned view of the recent ruckus over publicly releasing names of teachers who generate high as well as low value-added test score gains. I could not agree more with Leonhardt’s assessment that it is time for results-oriented teacher evaluation. The big questions, of course, are (1) just what do we mean by “results,” and (2) how are we going to be sure we’re measuring for the outcomes we really want. Because we can be sure that the measurements we choose will heavily influence those outcomes.

Leonhardt rightfully notes that “schools generally do not allow parents to see any part of a teacher’s past evaluations, for instance” and that “for the most part (parents) have to hope for the best.” But while he points to several of the serious technical flaws in using even the most sophisticated value-added models to judge teachers, he ultimately shrugs them off.

VAM methods falsely identify teachers as ineffective (as well as effective) at very high rate. However, drawing on the viewpoints of several school reform advocates (including Kati Haycock) who seem comfortable with the VAM error rate, Mr. Leonhardt suggests that it’s far better to make mistakes in judging who is an effective teacher than not using value-added tools at all.

Leonhardt, like many of the reformers he quotes, seems to think there are only two choices to pursue when it comes to teacher accountability: Use test-based results, or not. While the “either-or” scenario painted by many journalists fuels the hot debates between reformers and union leaders, it does little to actually improve public education and ensure that all students will get the effective teachers they deserve.

With value-added error rates at 25% (and higher), it’s morally irresponsible for VAM to receive such overwhelming emphasis — and it’s educationally irresponsible to use it to fire teachers as called for by many self-proclaimed reformers. Millions of students will be harmed when suitable replacements cannot be found — and many students will also lose effective and caring teachers who were falsely identified as ineffective. With the accountability pressure turned up even higher (now careers are at stake), many teachers will feel compelled to just teach to whatever is on the current standardized test. Forget all about addressing the “creativity crisis” ballyhooed on a recent Newsweek cover. America can settle in for another generation of rote learning and remediation.

None of this means that value-added data should not be used as one piece of information in a comprehensive system of teacher accountability that all of us, including Mr. Leonhardt, want for our students and their families. But failing to recognize the downside of using flawed data to make life-changing decisions may lead to rewarding the wrong teachers — a development that will undermine parents even more as they seek to secure the best education for their children (and hopefully for everyone else’s kids as well).

Related categories: