Schools Need Real Research

Yesterday, I pointed out that Newark superintendent Cami Anderson's piece in the Star-Ledger referred to all sorts of "research" without giving us any citations. I can't say for sure whether that's the fault of Anderson or the S-L; I can say that Anderson is making some very controversial assertions, and she should back them up with actual citations from the literature.

A good role model for both her and the S-L is Matt DiCarlo of shankerblog.org. Matt is a real researcher who reaches his conclusions through careful study of high-quality studies, and not through ideology. Matt pointed me to some real research that contradicts some of Anderson's claims. For example:

Anderson: "Some research shows that we lose our best teachers to charter schools and other professions because they feel they are not growing and they become disheartened seeing students in ineffective classrooms."

That's a very audacious claim: "our best" teachers are going to charters? Really?

Real research:

I find that the North Carolina teachers who leave the mainstream public school sector for charter schools are less qualified and less effective than other mobile teachers, even in the presence of controls for sending and receiving school environments. These results suggest that charters have lower demand for these teacher qualities, or that charters have insufficient resources to outbid competing mainstream schools, or both. The relative risk of charter mobility increased with nonwhite student shares in mainstream schools, so choice schools may exacerbate higher turnover in high-minority schools. It is important to note, however, that charters could reduce overall teacher turnover by offering a viable alternative to non-teaching careers. Charter movers resembled teachers leaving North Carolina public schools more so than other mobile teachers, but charter movers taught for another 3.24 years on average. Low-performing or high-minority mainstream schools do not lose substantially more effective or more qualified teachers to the charter sector, but among recipient charters, better teachers gravitate to better schools, schools with fewer nonwhite students, and schools in less urban areas. These patterns will reinforce sub-par achievement in North Carolina’s charter schools. [emphasis mine]

Sure, it's a limited sample, but it's a real study that uses rigorous methods, and it directly contradicts Anderson's claim. Obviously, we should compare this study to the one Anderson cites... whoops! Looks like Anderson and the S-L forgot to mention the research she's relying on. It might have been helpful to see that research before we make policy decisions, don't you think?

Anderson: "We need to retain our most talented teachers, but many successful teachers leave because their contract doesn’t recognize and reward good performance."


Anderson couches her words very carefully here: "...many successful teachers leave..." How many is "many"? In spite of her careful wording, the implication of her sentence is that we are losing "good" teachers because "bad" teachers are treated the same as the "good" ones. But is it true? Are "bad" teachers staying while "good" teachers are leaving?


Real research:

While we find little evidence of differential attrition by the effectiveness of teachers with more than one year of teaching experience, elementary teachers and middle-school math teachers who leave teaching in New York prior to their second year are responsible for lower achievement gains for their students, on average, than are their colleagues who remain, especially for those teaching in schools where student achievement is lowest. In other words, the achievement scores of many students will likely increase as a result of the attrition of some teachers. This may be a reasonable response to a poor initial career choice and may reflect “counseling out” by school officials. [emphasis mine]

So there's at least some evidence that attrition occurs when teachers are bad at their jobs. Again, I'd very much like to see Anderson's research so we can square her study with this one; it's quite possible the two are not contradictory. Too bad she and the S-L won't tell us what she's citing.

This is just laziness, and there's no excuse for it. Anderson is, by her own admission, talking about radically changing the terms of teacher employment in Newark. Why can't we see the basis on which she makes her policy prescriptions?

I would never advocate making wide-scale changes to our educational system based solely on the studies Matt pointed out; that would be very foolish. And the burden of proof remains on those who want to change the system; they are the ones who have to justify their plans. The very least they owe us is a look at the "evidence" they rely on.