Jeb!'s Florida Education Meh-racle

After watching Jeb! Bush talk about his Florida education "miracle" at Campbell's Brown's little union-bashing festival, I thought it would be instructive to make a quick graph:

I've standardized Florida's scores on the National Assessment of Educational Progress over time as a way of comparing the state to the other 49 plus DC. The NAEP tests reading and math in Grades 4 and 8. Upward bars are above average scores; downward bars are below average.

You'll notice that Jeb! is always bragging on Florida's Grade 4 Reading scores. What he rarely mentions, however, is how his state does on Grade 8 tests, or on math tests. This graph makes clear why: Florida has been either at or below the (weighted) average on Grade 4 math and on Grade 8 reading and math for the last two administrations of the test. Yes, there was a bit of a bump on Grade 4 math in the 2000s, but the scores came back to Earth.

Does this look like a miracle to you?

Normally, I think any politician who gets up in front of people and brags on how his policies turned around his state's schools practically overnight is being ridiculous. There are just too many factors other than policy changes involved in affecting test scores: economic changes, demographic changes, cohort effects of other sorts, measurement errors, and so on.

In this case, however, I do think we have to give Jeb! some "credit" for those Grade 4 reading scores. Because, according to this analysis by Walter Haney, Florida did, in fact, implement a policy that changed the trajectory of Grade 4 scores (p.6):

But particularly notable regarding how Florida boosted NAEP grade 4 results in 2005 are the grade 3 and 4 results for 200-3-04. What these results indicate is that in the 2003-04 school year Florida started flunking far more children – on the order of 10-12% overall – to repeat grade 3. Hence it is clear what caused the dramatic jump in grade 4 NAEP results for 2005. Florida had started flunking more children before they reached grade 4.

What caused the dramatic decrease in the race gap in NAEP results in Florida? Grade transition analyses of enrollment data make the answer abundantly clear. I will not present detailed results in the short time available here. But what I can say by way of summary is that analyses of grade enrollments in Florida by race (Black, Hispanic and White) make it clear that when Florida started in 2003-04 to flunk more children to repeat grade 3, these were disproportionately more Black and Hispanic children (15-20% of whom were flunked) than White ones (about 4-6%% of whom were flunked in grade 3). Thus it is clear that the NAEP grade 4 results for 2005 reflected not any dramatic improvements in elementary education in the state. Rather they were an indirect reflection of Florida policy that resulted in two to three times larger percentages of minority than White children being flunked to repeat grade 3.

This is, regrettably, a tragedy in the making. Research now makes it abundantly clearly that flunking children to repeat grades in school is not only ineffective in boosting their achievement, but also dramatically increases the probability that they will leave school before high school graduation (see, for example, Shepard & Smith, 1989; Heubert & Hauser, 1999; Jimerson, 2001; Jimerson, Anderson & Whipple, 2002). I will not try to summarize here the abundant evidence on these two points, save to note that considerable research has found that among children who are overage for grade in grade 9 (regardless of whether they were flunked in grade 9 or earlier grades), 65-90% will not persist in high school to graduation. [emphasis mine]

Florida has since rethought its policy of mandatory Grade 3 retention. Good for them.

Much as I don't want to, I do have to agree with Matt DiCarlo: there is some evidence that some of Jeb! Bush's policies, particularly the use of accountability pressures, did indeed raise test scores modestly. Those of us who have serious reservations about the overuse of standardized tests should acknowledge that there may, in fact, be benefits to their use in accountability systems (although I would say the way Bush used them probably caused more harm than good - more later).

But Jeb! and his acolytes have so oversold his education "successes" that it's nearly impossible to have a serious conversation about what may have gone right in Florida -- let alone what has definitely gone wrong.

There was no Florida education "miracle." Anyone who says otherwise is selling a bag of magic beans.

But if I can't own the Florida "miracle," what do I have left?