Milwaukee's Best?

Those of you stopping by from outside the Garden State may not be aware that we have had a big battle brewing over school vouchers, in the form of the Opportunity Scholarship Act. There's talk that a scaled-down version of OSA is going to try to get through the legislature again, even though it's missing a key previous supporter, Senator Ray Lesniak (whose pet school, St. Pat's in Elizabeth, is shutting down this year - tells you something, doesn't it?).

It's worth noting that OSA is going to disproportionately affect one community in New Jersey: Lakewood. There are over 17,000 private school students in Lakewood, most attending yeshivas; but I doubt very much the pro-voucher forces will be discussing this fact.

No, I'm guessing we'll be hearing much more about this:

The Milwaukee Parental Choice Program is a success story in virtually every measurable way.
  
     A new study released on the nation’s oldest private school voucher program shows that participants are more likely to graduate and go on to college, improve in reading and science, and represent a more diverse student population than their counterparts at traditional Milwaukee public schools.
  
     “Although we have examined virtually every possible way that school choice could systematically affect people, schools, and neighborhoods in Milwaukee, we have found no evidence of any harmful effects of choice,” wrote Patrick Wolf, professor at the University of Arkansas and lead author of the study.
     
     The five-year evaluation (2006-11) of the Milwaukee voucher program was conducted by Wolf, along with University of Wisconsin professor John Witte and Furman University professor David Fleming, as part of theUniversity of Arkansas’ School Choice Demonstration Project.
  
     The research shows that the growing number of students in the voucher program and area charter schools not only “produced better student outcomes than those experienced by similar students in (Milwaukee Public Schools),” competition from those schools raised achievement in local public schools.
  
     In dollars and cents, Milwaukee’s voucher program saved the state $52 million last fiscal year, because the $6,500 vouchers cost far less than the per-pupil state funding sent to the city’s public schools.

I want to get this on the record right now because I'm starting to understand how these people think. I am certain the NJ voucher cheerleaders are going to go on about how Milwaukee is a shining example of voucher success.

Don't believe them. Let's start with the actual evaluation itself:

When similar MPCP and MPS students are matched and tracked over four years, the achievement growth of MPCP students compared to MPS students is higher in reading but similar in math. The MPCP achievement advantage in reading is only conclusive in 2010-11, the year a high-stakes testing policy was added to the MPCP (Report #29).

• When a snapshot of all MPCP students who took the state accountability test is compared to a snapshot of the performance of MPS students with similar income disadvantages, the MPCP students are performing at higher levels in the upper grades in reading and science but at lower levels in math at all grade levels examined and in reading and science in 4th grade (Report #32).

Hardly a ringing endorsement. But let's go further:

To the evaluators of the Milwaukee Parental Choice Program, vouchers are like a vaccine. Once students are "exposed" to the voucher program - even if they subsequently leave - that "exposure" somehow accounts for any good things that happen later on. 

And leave they did - a whopping 75% of them. 

Here are the details: The evaluators began by following 801 ninth-grade voucher recipients. By 12th grade, only about 200 of these students were still using vouchers to attend private school. Three of every four students had left the program. 

Given this attrition, the researchers had to estimate graduation rates (as well as college attendance rates and persistence in college) by comparing Milwaukee Public Schools students to students who had been "exposed" to the voucher program - even though most of those students appear to have actually graduated from an MPS school. 

Yet the summary report from the evaluators has no mention of the 75% attrition rate. What readers were told was, "Enrolling in (read as "being exposed to") a private high school through MPCP increases the likelihood of a student graduating from high school, enrolling in a four-year college and persisting in college by 4-7 percentage points." 

That sounds positive, and voucher advocates have trumpeted this statement. But a more defensible statement is that there are no findings of benefits that are statistically distinguishable from zero. 

Here's why: After controlling both for students' prior measured achievement and for differences in the level of parents' formal education, to ensure that comparable students were being compared, none of the benefits showcased by the evaluators are statistically significant using conventional significance criteria. [emphasis mine]

It's also worth noting that any gains in student achievement in the voucher schools is confounded by the fact that the schools got the inevitable bump that comes from subsequent administrations of standardized tests; in other words, the schools get "used to" the tests and do better. This is not a real indicator of student achievement.

As to the claim that the Milwaukee vouchers saved taxpayers money:

Finally, there is no reference to any literature that has raised issues about the earlier studies. For example, the report claiming that the MPCP saves taxpayers’ money was reviewed by this author two years ago.14 The existence of any savings depends heavily on two assumptions: how many students would have attended public school if the voucher option was not available, and how much is saved as each student transfers out of the public school system. Any savings that might arise are very sensitive to the first assumption about take-up. Plus, the savings are assumed as if the voucher students would have cost the same as other students; this assumption is not tested and is most likely not valid. Students who leave for voucher schools are likely to cost less to educate than the students who remain. Neither of these issues is addressed in this report. [emphasis mine]

In other words: you can't judge the savings until you look at how much it would cost to educate the different students. Why reformy types continue to ignore this critical part of the debate is beyond me.

Here's the conclusion of NEPC's analysis:

Over the past two decades, the injustices done to MPS students have multiplied. The student body has become more desperately poor and the number of students with costly special educational needs has increased. Art, music and physical education have been drastically cut, and the teacher mentoring program scaled back. 

Over that same period, the Milwaukee voucher program has not delivered. As this evaluation confirms, students who receive vouchers probably don't do any worse or any better when they move to private schools. 

But evidence doesn't seem to matter. The "choice" system has become the broken status quo - choice as an end in itself - vigilantly guarded and professionally promoted by a well-funded, well-compensated and well-placed phalanx of advocates while the real needs of Milwaukee's schoolchildren remain largely unaddressed. [emphasis mine]

 Gosh, that's sounds so familiar...

New Jersey and other states confronting vouchers, you need to ask yourself one thing:

Twenty years from now, do you want to look like Milwaukee?