Joel Klein: As Excellent As He Says He Is? Part IV

Joel Klein is one of America's foremost proponents of corporate education "reform." He asserts that public education is in such a crisis that it constitutes a national security threat - a threat that can be dealt with by buying technology from the firm he runs for Rupert Murdoch, Amplify. He also believes that the "reforms" he and Mayor Michael Bloomberg instituted in New York City - including mayoral control, school closings, charter expansion, and test-based teacher evaluations - should be replicated across the country, based on what he terms the "compelling" improvements made under his tenure.

But does the record support his claims? What is the legacy of Joel Klein in New York City? I'm looking at the evidence to see if the facts support Klein's boasts. Here's the series so far:

Part I: Joel Klein has no problem twisting the facts to suit his ends. Has he done the same thing when crowing about his "success" in New York?

Part II: When you break down national test scores by student subgroups, Klein's "success" in New York isn't very impressive; in fact, it's downright disappointing.

Part III: A look at national test scores as a whole, comparing New York to other cities during Klein's reign. The results? Meh.

When we last left Joel Klein, his record as New York City's chancellor left me saying, "meh." Sure, there were some gains on the National Assessment of Educational Progress (NAEP), known as the "Nation's Report Card." But NYC's growth was anemic compared to other large cities across the nation. There is certainly no evidence that Gotham, under Klein and Mayor Bloomberg, was setting the pace for the rest of the nation.

But Klein doesn't stake his reputation on progress on the NAEP alone. Once again, here is his magnum opus from the NY Daily News:

Similarly strong improvements were achieved on state tests, although the story here is harder to explain because the state changed the testing requirements several years ago, and so the number of kids passing (i.e., achieving the proficiency or advanced level) dropped because the questions were intentionally made harder and it took more correct answers to pass. 

Nevertheless, if you compare New York City’s performance under Bloomberg to two other groups (the other so-called “big four” cities in New York State — Rochester, Buffalo, Syracuse and Yonkers — and the rest of the state), each of which took precisely the same tests as the city, the results are compelling. 

In a nutshell, in 2002, when Bloomberg started, on all four tests, New York was much closer to the big four cities and far away from the rest of the state. 

Today, it’s the other way around, showing that the city moved significantly forward when compared to the other two relevant groups. Indeed, despite the fact that the exams were more difficult and required more correct answers to pass, New York City increased its pass rate on all four tests; the other two groups didn’t come close.

We'll get to those changes in state tests later in this series. Let's focus, for right now, on Klein's premise: that we should judge the improvement of New York City on state test proficiency rates compared to the other "big" cities in New York State.

Setting aside the obvious fact that NYC is much, much larger than any other city in New York - which makes the comparison specious all by itself - is this a good way to judge NYC's success? Are Rochester, Buffalo, and Syracuse legitimate comparisons?

Well...

I have to admit something: I know Upstate New York pretty well, but even I was shocked by these numbers. Child poverty has skyrocketed in the next three largest cities in the Empire State, while remaining relatively flat in NYC*.

It makes sense if you know the region: manufacturing has died, and so, to a large extent, has agriculture. The Rochester area has some high-tech growing these days, but the big drivers of the upstate economy - Kodak, GE, Xerox, Carrier - have downsized, fled, or died. The last time I was in Utica (admittedly, a few years ago), it was a ghost town. The majority of Schenectady's children live in poverty. It's a trend over most of the region...

But not in New York City, where the change in the child poverty rate has been rising at a considerably slower rate, and was far less to begin with.

The relationship between poverty and academic performance is beyond dispute. Does it make any sense at all to compare New York City's academic gains with those of regions that are toiling under poverty rates that are growing faster?

Of course, if we could find a way to account for poverty in comparing test scores, maybe the comparison would be more apt. That's exactly what a clever study Klein cites attempted to do; more in a bit.

Rochester could turn things around if they buy my software!

* Technical note: the Census Bureau reports these figures with both a percentage of children in poverty and a margin of error (in 2005 and 2011), which I didn't include. As you may imagine, the MOE is somewhat higher for the upstate cities. I don't think it's that big of an issue, but I am no demographer or sociologist, so caveat lector. If someone knows more about this, I'm always happy to learn.

Also: I had to pull out each year's table separately, and I wasn't about to do that for every year. So there's got to be some year-to-year variation I missed. If you think that's a big deal given the point I'm making, let me know in comments - but I don't think it is.