Don't Believe Everything You Read

One of the hallmarks of our modern discourse is that there are those who hold a particular position so dear (usually on the right, but not always) that they not only block out masses of contradictory evidence; they jump on the first bit of "research" that comes along that comports with their world view, no matter how shoddy it may be.

Case in point: B4K is jumping for joy that a study "proves" money doesn't matter in our schools:

A NEW STUDY: MORE RESOURCES DO NOT MEAN BETTER SCHOOLS

Getting Beneath the Veil of Effective Schools

12/12/11

Amid the debate on education reform in New Jersey, we often hear assertions that more resources are better: more teachers for smaller class sizes, more overall spending on schools, continued pay increases for teachers for advanced degrees, and so on.  Support for such policies is claimed to be “research-based” and thus apparently beyond question.  However, new, credible research sheds some doubt on these longstanding assertions.  As New Jersey embarks on devising new education reform legislation, our citizens and politicians should take notice. 

A recent working paper by two Harvard professors, Will Dobbie and Roland Fryer, is appropriately titled, “Getting Beneath the Veil of Effective Schools: Evidence from New York City” (National Bureau of Economic Research, December 2011) (here). While not yet an official NBER publication, their exhaustive study collected “unparalleled data” on the inner workings of 35 New York City charter schools employing a wide range of educational strategies to determine what separated effective schools from the others.   Their findings are eye-opening.

The authors conclude that traditional measures long held out to be determinative of “good” schools class size, per pupil expenditure, the fraction of teachers with no certification, and the fraction of teachers with advanced degrees – are not correlated with school effectiveness.   In fact, the data suggest that increasing such resource-based inputs may actually lower school effectiveness.

I knew it! I knew that money couldn't possibly make a difference in our schools! I mean, 30 years of conservative propaganda primed me to believe it, so it must be true!

Oh, wait...

But I’ll be honest here. I find this Dobbie/Fryer piece infuriating on many levels, some of which are simply entirely inexcusable (and, as noted below, this is the 3rd in a row, so my patience is running thin).  The basic structure of their study, as far as I can tell from the disturbingly sparse documentation in their working paper,  is that they conducted a survey of NYC charter schools to gather information on practices (the no excuses stuff) and on expenditures and class size. Then, they evaluated the correlations between individual factors (and an aggregate index of them) among traditional and no excuses measures, and alternative forms of their charter effect estimates.

Let’s be really clear here – simply testing the correlation between spending and an outcome measure – comparing higher and lower spending schools and their outcomes to see if the higher spending schools have higher effectiveness measures – WOULD TELL US LITTLE OR NOTHING, EVEN IF THE DATA WERE ACCURATE, PRECISE AND WELL DOCUMENTED. Which, by the way, they are not.

Oh, dear - you mean an incomplete and poorly documented working paper about 35 charter schools in NYC may not be applicable to the general student population? And that maybe we shouldn't make radical changes to education based on this research?  Who ever would have guessed...

I've made the parallel between global warming research and ed reforminess before, and I think it really is apt. Common sense alone would suggest that you can't pour billions of tons of CO2 into the atmosphere without something bad happening. The research overwhelmingly confirms this. But every now and then,  someone with good credentials says something to the contrary, and BAM! Fox News is off and running. Meanwhile, the planet burns up.

In the same way: I don't know how anyone could look you in the face and tell you the amount of money you spend on schools really doesn't matter, and could in fact make things worse. We just had a case in Colorado where the judge effectively told an "expert" making this claim that he was flying in the face of both the overwhelming evidence against him and simple reason.

But the folks who have their hearts set on reforminess will believe what they want to believe. And they will always be able to find "evidence" to back up their views; that's what think tanks are for, apparently.

But not all "evidence" is the same. Caveat lector.