Latest "Chiefs For Change" Fail: Bowen in ME #2

I just posted about Maine "Chief For Change," Stephen Bowen, and how he's had to back off on plans - written by his patron, Jeb! Bush - to fast-track for-profit virtual charter schools into the state.

Before we move on to another state and another CFC Fail, however, let's take a moment to enjoy the twisted logic of Commissioner Bowen:

Second, the Press Herald seems to feel that the new A-F grading system suffers from a fatal flaw because it does not discriminate between schools in more affluent areas of the state and those in less affluent areas. The paper seems to suggest that the department should have developed a grading system with one set of performance standards for wealthy areas and a second, presumably lower, set of standards for poorer areas.

Developing such a two-tier system was never a consideration for the administration, however, for the simple reason that we don't share the Press Herald's view that students in less affluent communities are necessarily destined, by virtue of where they live, to struggle in school.

There is, of course, a correlation, well supported in the research and illustrated by the grading system, between the income and education levels of families and the academic achievement of their children. There are also plenty of examples, however, in Maine and across the nation, of schools in very poor communities achieving extraordinary results for the students they serve.

All the negative press about how the new grading system is unfair to poor communities ignores the fact that a number of schools in poor communities across Maine earned an A or B in our grading system. Nearly 80 percent of the students attending Phillips Elementary School, for instance, qualify for free or reduced-price school lunch, yet the school earned an A for its students' high achievement. [emphasis mine]

There are layers and layers of reformy illogic to dig through here:

- If the correlation between income and education levels is "illustrated by the grading system,"how can you say for sure you're measuring school effectiveness, and not poverty levels?

Here's a tool that shows the correlation between student poverty and Maine's school grading system. The high school results:

And the elementaries (annotation mine):

It appears that Maine's school grades are strongly correlated to poverty levels for those schools.

We've seen this before:

There is a reason why over 97 percent of Florida’s lowest-poverty schools receive A or B grades, and virtually every one of the schools receiving a D or F have poverty rates above the median. It’s because schools are judged largely by absolute performance, and students from higher-income families tend to score higher on tests.

Same thing in Ohio. And Indiana (outstanding work here, as always, by Matt DiCarlo). 

I suppose Bowen would argue that, because the Maine school evaluation system uses both "growth" and "absolute performance" to judge schools, it wouldn't have as strong a correlation between poverty and performance. We'll leave aside the fact that there is substantial evidence that there is a correlation between poverty and "growth" (contrary to the statements of Bowen's fellow CFC, New Jersey's Chris Cerf); the problem remains the same:

If you aren't judging schools on absolute performance to begin with, aren't you in effect admitting high-poverty schools should be judged differently than low-poverty schools?

- Outliers are not proof that causation doesn't exist. This is one of the greatest hits of reformy types like the CFCs, and it needs to be put down once and for all: the existence of an outlier does not disprove a causal effect between two variables. For example:

• There is a correlation between between smoking and cancer rates.
• But there are some people who smoke and don't get cancer.
• Therefore, smoking doesn't "cause" cancer.

Yes, we hear it all the time from the reformy side: "correlation is not causation." But you can't just throw away mounds of inferential evidence because you found a few outliers; that's irresponsible and irrational. Even if Phillips Elementary appears to "beat the odds," that's no reason to believe that Bowen's A-F school grading system isn't biased.

Further, if you're going to investigate why an outlier outperforms its expectations, you have to be rigorous; you can't just say, "They're awesome!" and be done with it. Phillips Elementary does, indeed, do better than the school grading trend-line - the question is "how?" Is it really a superior school? Or is this the result of statistical noise or bias? Is the data used for grading not accurate or precise enough?

Bowen doesn't want to venture a guess, but maybe he should take a look at the school's demographic profile:

Special Education: 13.64%
(State Average Demographics, Grades K-8 Special Education: 13.9%)  

Free/Reduced-Price Lunch: 70.7%
(State Average Demographics, Grades K-8 Free/Reduced-Price Lunch: 48.72%)  

Limited English Proficiency: 0%
(State Average Demographics, Grades K-8 Limited English Proficiency: 2.15%) 

Location: Phillips, Maine
Enrollment: 154

So, yes, Phillips has more kids in poverty than the average for Maine, but no kids who are LEP. And it's a tiny school - 154 kids for grades K through 8. A population that small - a school that had 13 Eighth Grade students in 2010-11 - could easily be subject to significant swings in test scores year-to-year.

Again: how did Phillips get a good grade in spite of it's large FRPL population? What can be learned here that can be broadly applied across the state? Unless and until Bowen is prepared to give a serious answer, he ought not to be holding up any outlier as proof that his school grading system is fair.

- There is very little evidence that an A-F grading system will help "failing" schools improve substantially. Earlier this week, I blogged about Matthew Ladner - writing for Jeb!'s FEE website - and his contention that research shows Florida's A-F school grading system has improved low-performing schools. I found the evidence Ladner presented, based on the NAEP, to be very weak, but he also included a reference to a paper from the Urban Institute that purports to show real gains in test scores when a school is hit with a grade of "F."

However, as one of my commenters pointed out:

Am I reading this wrong or does the Urban Institute report at best, in math, give the grading system credit for 38% of a 6 to 14% of a SD gain? So... a 2% to 5% SD gain? They're hanging their hat on that?

Go to pages 4 and 5 of the report, and you'll see my commenter is correct about the reported gains and how they were attributed to the A-F accountability system. This is very weak tea, and it's certainly not indicative of any real, practical improvements in instruction.


So, no, there's no reason to think Stephen Bowen's school grading method is fair toward schools with higher levels of poverty. There's no reason to think it will help improve Maine's schools. And there's no reason to think chanting "poverty is not destiny" over and over again is going to make the correlation between income and test scores go away.

And if there's no reason to think any of this in Maine, there's no reason to think it in any of the other little educational fiefdom's Jeb!'s Chiefs For Change have set up for themselves.

Data gives me a headache...