What Is a "Bad" School?

My GI tract decided to get into a fight with a virus this week, which left me with some extra time to  read as I tried to keep down Gatorade and saltines. One item I found especially interesting was a series of reports in the Asbury Park Press about the Asbury Park school district. Reporter Nicquel Terry lays out a lot of facts, even if the conclusions aren't exactly obvious:

The soaring price of failure, abysmal test scores, constant turnover in administration and infighting among school board members in Asbury Park has left many students without a solid education and parents like Ling worried about their child’s future.

The state has spent the last decade trying to fix the district, funneling nearly $550 million in aid, investing in support programs and sending five different state monitors to oversee operations.

Still, nothing appears to be working.

The K-12 school district, of 1,926 students and five schools, consistently ranks lower than its counterparts and below state averages in standardized test scores. A recent state report card said Asbury Park’s academic performance “significantly lags” in comparison with schools across the state. [emphasis mine]

Let's be clear: the state's "report cards" aren't worth the paper they're printed on. But let's explore the question anyway: is Asbury Park "lagging" in relation to its "peers"? Do schools that serve students demographically similar to those in Asbury Park show better outcomes? Terry offers some proof:

Only 2 percent of the students who took the SAT achieved scores of 1550, according to the state’s school-by-school performance reports. The report indicated Asbury Park High School had a composite SAT score of 962, well below the state average of 1512.

SAT scores are not really the best indicator of a school's efficiency, for many reasons (#1: it's a voluntary test). And we know that SATs correlate to strongly to a student's economic circumstances... but what if we controlled for that? How does Asbury Park do when we take into account the number of students it serves that are living with economic disadvantage?

This is why I love scatterplots. Here we've got every high school in the state, with its position plotted based on two variables: the percentage of students eligible for free lunch (which puts their families' incomes at 130% of the poverty line or below), and the average total combined SAT score for each school's student population.

The "r-squared" is geek speak: nearly 70% of a school's SAT score can be statistically "explained" by its FL percentage, a very strong correlation. The line running through the plots is a trendline -- a "prediction" made based on kinda-sorta "averaging" all of the points on the graph, and figuring out a line that fits best between them all. Basically, if a school is on the line, its SAT score is what we would expect, given its FL population.

And there's Asbury Park High in red, with a total SAT score of 962. Prediction, based just on the percentage of free lunch eligible students (81%), says Asbury Park should have scored about a 1093. Certainly, there are few high schools in New Jersey that score as low on the SATs as Asbury Park, no matter the amount of student poverty. And we have to acknowledge that FL percentage doesn't explain everything. But, when accounting for poverty, and when looking at prediction based on the results of other schools, Asbury Park is not getting the scores it should be.

But wait a minute...

Yes, there are schools, like Asbury Park, that are below prediction. But there are also schools that are above:

New Brunswick High has an 81% FL percentage, but its SAT scores average 1237. So...

(And here is where it always comes off the rails...)

So maybe there's something we should look at here -- maybe we should look at what New Brunswick is doing right, and get Asbury Park to do the same thing. Right? I mean, New Brunswick proves that poverty isn't destiny, and your zip code doesn't mean your status quo is left behind, and if certain unnamed parties (you know who you are!) would just start thinking about the kids first for once, and let's all be fiercely working hard and no more excuses! and blah, blah, blah...

Would that life were this simple, folks.

Look, I'll be the first to say that there is probably something going on in Asbury Park's schools that needs addressing. The state monitor is at war with the school board, and that can't be helping. Maybe there are governance issues in Asbury Park; maybe they should be doing better.

And there are certainly demographic differences between Asbury Park and New Brunswick, so it's not as if the two districts are completely similar. But perhaps New Brunswick has programs worth imitating. I'm all for best practices -- bring them on.

But if you think the story of the graph above is the distance between the two red diamonds, you are missing the entire point:

The predicted SAT score for a school with no FL eligible students is 1650. That's better than Asbury Park, better than New Brunswick, and better than almost any school with more than 20 percent of its children living with that level of economic disadvantage.

That's the important point. That's what we ought to be talking about. That's what needs to be fixed.

Our reformy conversation about schools keeps centering on the "miracle" schools (they usually aren't) that "beat the odds" (they almost never really do) despite serving children in poverty. What no one seems to want to acknowledge, however, is the much greater and more blindingly obvious truth: poverty matters. If you want better student outcomes, you've got to fix poverty -- first, foremost, and yes, right away.

Does the difference between Asbury Park and New Brunswick matter? Yes, it does. Ought we try to find ways to make schools that underperform, based on student characteristics, get better results? Yes, we should. Are schools a critical part of an overall strategy of combating poverty? Unquestionably, yes.

But we continue to distract ourselves when we focus our energies on "fixing" public education -- especially when the answer to the problem of unequal student outcomes is right under our damn noses.