Even More "One Study Does Not a Policy Make"

Matt DiCarlo weighs in on the latest teacher evaluation working paper:

It’s also critical to note that this analysis does not account for the (very real) possibility that high-stakes implementation of policies using VA estimates might alter teachers’ behavior, or the supply of candidates into the profession. If, for example, increasing the stakes compels teachers to teach to the test, then the connection between teacher effects (and test results in general) and future outcomes such as earnings may be compromised, perhaps severely. 

Interestingly, the authors themselves raise some of these concerns – specifically, high-stakes implementation and whether the cost of errors from using VA outweighs the benefits – as “important issues” that “must be resolvedbefore one can determine whether VA should be used to evaluate teachers.” [emphasis mine] 

This appropriately cautious conclusion stands in stark contrast with the fact that most states have already decided to do so. It also indicates that those using the results of this paper to argue forcefully for specific policies are drawing unsupported conclusions from otherwise very important empirical findings.

Asking for caution from those who are hell-bent on using tests to evaluate teachers is like asking an 8-year-old at the sundae bar to go easy on the whipped cream. Which is why the authors of this paper - if they really believe what they write - should exercise more caution in their interviews. Bruce Baker addresses some of their quotes in the New York Times:

These two quotes by authors of the study were unnecessary and inappropriate. Perhaps it’s just how NYT spun it… or simply what the reporter latched on to. I’ve been there. But these quotes in my view undermine a study that has a lot of interesting stuff and cool data embedded within.

These quotes are unfortunately illustrative of the most egregiously simpleminded, technocratic, dehumanizing and disturbing thinking about how to “fix” teacher quality.

And it leads to the type of stupidity found here in the New York Daily News:

The research also documented that kids whose instructors produce the biggest gains in standardized test scores will eventually reap the benefit in the form of substantially elevated incomes. 

Those would be the same standardized test scores that United Federation of Teachers President Michael Mulgrew claims are worthless on their own in evaluating whether an instructor is effective in the classroom. 

Mulgrew is fighting to prevent school districts from using scores to identify and weed out the weakest teachers. He and other union leaders have placed protecting the jobs of incompetents over the future financial well-being of children. 

Shameful.

Nice. Sorry, professors - this is on you. We may have once lived in a world where your research would not have precipitated this kind of inanity, but no longer. You have a responsibility to understand the atmosphere your research enters and adjust your part in the discourse accordingly. Leo Casey adds:

One last word on the timing of Friday’s release of CFR. The choice of this particular moment, with the NYC DoE walking out of negotiations with the UFT over the teacher evaluation system for the 33 PLA schools, State Education Commissioner King’s disallowance of the teacher evaluation plans of all ten major urban school districts and Governor Cuomo’s announcement in the State of the State address that he would take up the matter of teacher evaluation, is telling. Combined with the assertion by the CFR authors that their study supported the mass firings of teachers identified as low-performing by value-added measures,  this moment points to the political nature of the CFR authors’ abandonment of scholarly ‘peer reviewed’ norms for the publication of research. Their decision serves well their partisan political purpose. It is just the quality of debate and decision-making around important public policy choices that suffers.

Now, some of you who read me regularly are going to be shocked, but I don't think it's fair to question the authors' motives here; that is, unless someone can show me a pattern in their work. I see no indication as to whether they are "partisan" or not, and I'll give them the benefit of the doubt until proven otherwise.

(Yes, I have dealt, and will continue to deal with, the motivations of many of the corporate reformers. The difference is I can back up what I say. Michelle Rhee grossly exaggerated the success of her teaching career; Rupert Murdoch admitted he wants to replace teachers with computers; Chris Cerf is reneging on his promise to release charter school data as "fast as is humanly possible." All of this - and a lot more - is well-documented and germane to a discussion of why these people want what they want and do what they do.)

But I don't excuse the authors from releasing this report into a highly charged and political environment the way they chose to; Casey clearly spells out the consequences of doing just that.

I know some are not happy to hear this, but the ivory towers have all crumbled. This is a political age in America; everything is steeped in it. If you wade into policy without understanding that, don't be surprised when your academic caveats are thrown to the winds.