I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Saturday, January 16, 2016

Only You Can Prevent Charter Cheerleading

UPDATE: Whitmire calls me out on Twitter, where apparently all education policy is now debated:

Few data points? So when NJ releases parcc and Alexander scores right, will look for a retraction.

Um, no. As you will read below, unless NJDOE changes how it does business, we'll never see what Alexander's score on the PARCC was in the official state data, because North Star's results are published in the aggregate: we only get results for all of North Star's schools together, and not the individual schools in their system. 

I actually wonder how North Star was able to separate Alexander from its other schools; does NJDOE have school-level data for charter chains? If yes, why don't they release those figures publicly? And release the special education and student demographic and suspension rates and all the other breakdowns by school as well?

More and better data -- please.

When I want some shallow charter school cheerleading, I go straight to Campbell Brown's The 74 Million. With the possible exception of Education Post, you'll not find a more reliable source of credulous, unconditional charter love; take, for example, this piece by the reliably reformy Richard Whitmire:
But a steady drip of recent data points to a very different story line: Not only did the reforms of traditional Newark Public Schools produce some real benefits, but the relatively small portion of the gift invested in Newark charter schools paid off big. Real big. 
Whitmire gives three citations to back up his claim that Newark's schools produced "real gains." Of course, all are from -- surprise! -- The 74: one from a spokesman for KIPP, the largest chain chain in Newark; one from current State Superintendent Chris Cerf, who, when he was the state education commissioner, was former State Superintendent Cami Anderson's biggest supporter; and one from Anderson herself.

All trot out data points to make their case; unfortunately, none account for what quantitative researchers often call secular effects: changes that affect the entire system being studied. In other words, if Newark's test scores rose as part of a statewide rise in scores, that really doesn't make the case that Newark's reforms had any direct, causal effect.

Bruce Baker and I looked at Newark's test scores -- both charters and NPS -- over the period of "reform" in the city's schools. We found no evidence that Newark has seen any positive changes that couldn't be explained by overall, statewide trends (I'll have a similar analysis of graduation rates out soon).

So when Whitmire says NPS reforms produced "some real benefits," understand he is making a claim that is supported only be the weakest of evidence, and is based on data points from directly interested parties. Does anyone think that's enough to make a suggestion like this?
The gains are so striking, in fact, that they raise a key question: Why didn’t the Newark reforms emphasize charters from the beginning? If you look across the Hudson River where former New York City schools chancellor Joel Klein produced striking gains by pulling in the region’s top charters with offers of $1 a year rentals to use existing buildings, it’s reasonable to ask (with the admitted benefit of hindsight): Why didn’t Newark do the same?
First of all, as Leonie Haimson among others has shown, the Klein era was hardly an undeniable success. In addition, it's worth pointing out that there is a good argument to be made that NPS would have been better off not selling off properties like 18th Avenue School to private owners. Not only did NPS arguably not get fair market value for the property; as (again) Bruce Baker and Gary Miron point out, the taxpayers are paying to buy a property they already own and turn it over to private ownership.

But let's get to Whitmire's central claim: that the evidence shows us that Newark would have been better off had it turned more of its buildings over to charters. Do we have enough proof to say that's a reasonable argument? Whitmire hangs his hat on the "success" of one -- yes, one -- charter school conversion:
The latest and most dramatic example comes from test results released to The Seventy Four by the charter network Uncommon Schools for its North Star Academy Alexander Street School: Based on the tough new PARCC tests, in just a single year Uncommon was able to erase years of education malpractice. [emphasis mine]
OK, hold on -- Richard, you're going to base your case on data that is not publicly available, cannot be verified, and comes straight from an interested party? That's your evidence?

I've complained about this before: time and again, charter operators are relying on their own, proprietary data to make claims about their "success" -- and "journalists" like Whitmire just swallow them whole. I suppose it's to be expected that charters would do this; public school districts, to be fair, will also crow about their proficiency rates before all the data is made available to the public. But you would think Whitmire would prefer to wait until he could confirm the data...

Because you can't make the claim that test scores show any school is "succeeding" without accounting for differences in its student populations.

Whitmire thinks it's enough to simply show the state average proficiency rates for "non-economically disadvantaged" students and compare them to North Star/Alexander's. But that's wholly inadequate: what about differences in the populations of special education and Limited English Proficient students? What about the differences in the types of learning disabilities? What about the differences between free lunch-eligible and reduced price lunch-eligible students, which I've shown can significantly affect test scores?

Let's look at North Star's student population a little more closely (click to enlarge):

North Star has a relatively small FL population, even compared to many other Newark charter chains.

From my posts on Teachers Village: North Star enrolls far fewer special education students proportionally than NPS.

Also from the Teachers Village series: North Star enrolls not even one LEP student.

When I testified last year before the NJ Legislature on Newark's "reforms," I showed this graph:

The methodology was (again) developed by Bruce Baker to control for differences in student characteristics, spending, class sizes, and other school variations, and then compare adjusted growth percentiles. Let's be fair: North Star is one of the more "efficient" schools in Newark. But several other schools, including NPS schools, are equally or more efficient. Further, many Newark charters are relatively inefficient. Where is there any proof a mass conversion of NPS schools to charters will substantively change student outcomes for the better?

But we're not done yet, folks. Because Whitmire doesn't even tell us the most important fact we need to know to fully evaluate the conversion of Alexander School into a charter:
Alexander was a school so depressing and so low performing that former Newark Superintendent Cami Anderson told me she used to cry when she visited. Uncommon agreed to assume responsibility for the building and its K-4 students, almost all of whom returned. [emphasis mine]
Yes, Uncommon took the K-4 students; but they weren't the only grades at Alexander. This memo comes straight from NPS:
Alexander Street: North Star Academy will operate grades K-4 beginning fall 2014. Alexander families with K-4th grade students must submit a One Newark Enrolls application. Families will have preference to North Star Alexander Elementary School or their other top-choice schools through One Newark Enrolls. Alexander families with 5th through 8th graders must submit a One Newark Enrolls application and will have preference to their top-ranked district or charter schools. [emphasis mine]
When North Star took over Alexander, they cut the Grade 5 through Grade 8 students loose.  I ask all of you who thunder that we cannot tolerate any delays in charter school conversions: if it was so important for Alexander's students to be enrolled in a charter like North Star, why didn't Cami Anderson force the charter to take all of the students enrolled there?

Dropping the four middle school grades to focus on younger students is a luxury NPS district schools can't afford. And there's good reason to believe that it matters:

Again, from my testimony before the Joint Committee on the Public Schools. Here's how the Class of 2018 shrank during their Grade 5 through Grade 8 years at North Star -- all while their scale scores on the NJASK went up. Did Whitmire ever bother to ask about this?

Unless and until the state-run NPS district releases publicly available and vetted data, we will never know how many students actually stayed when Alexander converted. We will never know if the proportion of FL or special education students changed, because NJDOE only requires charters to report student demographics aggregated across their entire network.

And, perhaps most importantly: unless and until someone actually goes into North Star and carefully vets the school's practices, we will never really know if there are unobserved differences between North Star students and NPS students.

I get the sense sometimes that people think charter schools are a recent phenomenon. But North Star has been around since 1997; it's hardly new. According to many charter advocates, students and families learn from each other how charters differ not only from the public schools, but from other charters. I agree; I think the word most certainly has gone out about what kind of school North Star is.

In 2013-14, North Star had the highest suspension rate of any "popular" school in Newark. Not only that:

North Star's certificated staff is one of the least experienced faculties in the city. Yes, I do think many parents know these facts, and I do think they can influence their choices. I think many parents hear that North Star is a "no excuses" charter with fewer highly experienced educators than NPS, and they think: "Is this school going to be good fit for my child?" And then they act accordingly. 

I am always amazed at charter advocates who can't follow through this rather simple line of reasoning -- one based on premises they all accept. If parents are going to "choose" their child's school, why would we be at all surprised that the children who enroll in a school like North Star differ from those who enroll in an NPS school -- even in ways that do not show up in state data? 

Why would we think the children in "no excuses" charters are just like the children who are in other schools? Again, a "choice" system is predicated on families being informed consumers; of being able to access more information than the poorly-constructed measures of school effectiveness put out by clearly biased governmental entities. 

If we believe this, than of course the students at North Star will take to a "no excuses" model of schooling better than the entire student population. Of course many students at North Star will leave after their families discover the school isn't the right "fit." I honestly do not understand why any charter advocate would ever try to argue otherwise.

Again, let me be fair: North Star is a school that generally outperforms on test-based measures given its student population. Good for them; they should be proud of that. But there is no evidence the "successes" of North Star can be significantly scaled up. North Star may work for the students who stay there, but there is no evidence it will work for all students; in my opinion, there's no evidence it will work for many of Newark's students, particularly give Uncommon's rather questionable pedagogical methods.

In addition: we are seeing more and more evidence that Newark's school funding system must change, or charter school proliferation will become an increasingly destructive force for the many students who, by choice, remain enrolled in NPS schools. That cannot be allowed to continue; it's completely unfair and it will rip the city apart. Full funding of all Newark schools is a necessary precondition for the expansion of charters.

These issues, however, seem to be lost on Richard Whitmire. Like any good ideologue, he substitutes spin and hype for facts and reason:
The awkward bottom line is that Newark traditional schools can’t compete with the top charters — a fact Anderson acknowledges — for really simple reasons: The charters can recruit promising talent and then lavish them with extensive training.  Plus, the charters can take advantage of their slim headquarters staff to push more resources to the classroom. [emphasis mine]
That claim is simply not supported by the evidence. 

I don't care what Dale Russakoff's book says, there is no evidence Newark's charters are lean and mean while NPS is full of bloat. I have repeatedly backed up my position on this with a slew of data; where's Whitmire's?

Further, as (again) Baker & Miron note:
These excess costs can be difficult to track since education management organizations do not report relevant, detailed, comprehensive expenditures in the same format or with comparable documentation as public districts or the charter schools themselves. One example comes from the IRS 990 form for the Uncommon Schools network, which operates North Star Academy in Newark, NJ. It reported 2012 compensation for its systemwide CEO approaching $270,000, for its CFO at $207,000, for its Senior Director of Real Estate at $130,000, and for the Newark managing director at $213,900; in addition, the network maintains school-level administrative staffs. These EMO salaries, presumably subsidized  by management fees, are not accounted for in state professional staffing reports or in the schools’ own expense reports, and they may not be fully accounted for by the management fees listed in financial reports, where they are typically shown as central administrative interacted service expenses. Administrative costs for the academy are also borne by the Newark Public School district, which has a separate districtwide administration. Thus, it is no easy task to determine exactly what the various administrative expenses for the North Start Academy actually total. [emphasis mine]
"Slim headquarters," huh?

 A couple more thoughts before I end:

- Yes, charter parents deserve a voice in how schools are structured. But so do the parents and students who advocate for NPS schools -- families whose voices are repeatedly marginalized or flat-out ignored.

- Market-based school "choice" is not the same as democratic self-governance.

Newark parents should not have to settle for "voting with their feet"; like suburban parents, they should be able to vote with their vote.

- If the Newark community wants to have schools with differing discipline codes, so be it. But parents should not have to enroll their children in schools that abrogate their rights simply to get them into schools that aren't crumbling and dangerous.

Richard Whitmire thinks that selectively citing data points is "fishy":
What says: . When someone cites only hs grad stats, as in / NYT op-ed, u know something fishy.
I'll be the first to agree that Kirp's argument was missing some key context. But Whitmire's propaganda here -- yes, let's call it what it is -- is, in my opinion, even worse. A few carefully selected data points that can't be verified and that come from an interested party do not add up to evidence for the radical remaking of Newark's schools that Richard Whitmire desires.

It's fine for charter schools to be proud of their work. But ideological charter cheerleading does not help anyone. Enough, please.

Gimme a "C"!

ADDING: Even the staff of Uncommon Schools agrees that finding the right "fit" is extremely important when choosing a school; just ask one of their college counselors:
Can you address the best way for students to research colleges -- resources, criteria, or do's and don'ts?
Visit in person whenever possible! Be intentional about your visits: use a checklist and take notes on your thoughts and impressions.  Don’t fixate on “name brands” or college rankings - keep an open mind to what may be a good fit for you and your family.  I like Collegeboard.orgNaviance and College-insight.org as online resources.  Make a folder for every college you are researching to stay organized. [emphasis mine]
Well, if that's true for college, isn't it just as true for K-12 schools? Isn't that the entire point of the school "choice" movement: that some schools aren't always a good "fit" for everyone?

And if that's true, why should we believe North Star's methods will work for all children?

As always: Bruce Baker is my advisor in the PhD program at Rutgers GSE.


gfb9+2/3 said...

With your graphs, I had to look very hard to find where the North Star charter school is. In some cases, with my weak eyes, I couldn't find it. Could you please go back and use color and arrows or something to show where that North Star school can be found? My eyes can't be the only weak ones out there.

Duke said...

gfb9+2/3: I recycled these graphs from other work. If I had to make brand new graphs every time I wrote a post, I wouldn't write even half as much as I do. I'm sorry, I wish I could do more, but I have to live my life as well as blog.

Thank you for reading.

teachingeconomist said...

An off topic question.

Peter Greene and I are having a discussion about the number of stand alone charter schools. He linked to the pie chart you posted (http://jerseyjazzman.blogspot.com/2015/10/charter-schools-exchange-part-ii.html). I think that the data set underlying that graph is the set of charters that are associated with an EMO of some sort, not all charter schools. Peter believes the data set underlying that graph is the set of all charter schools. Can you resolve our disagreement?

Duke said...

Hey, TE:

That pie chart is Brice Baker's, but he made it from a dataset I constructed (with his constant advice). Bruce's original post is here:


Check out the maps, which I found fascinating.

The dataset comes from 2011-12 NCES data, aka the Common Core of Data. No, not THAT Common Core; this one:


I linked this to a dataset from Gary Miron and Charisse Gulosino for a report they did for NEPC:


What I basically did was take Gary and Charisse's dataset and link it to the CCD (it was a royal pain in the ass, believe me). I then went to every charter school tagged in the CCD that wasn't linked to Miron/Gulosino and either: 1) linked it to an CMO in Miron's dataset, thinking they missed it; 2) created a new code for a CMO (offhand, I know Mavericks in FL wasn't in Gary's data, so I added that among others), or; 3) said the charter was unaffiliated.

So Peter's right: the total dataset is ALL charters. I would say we captured the large majority of them, although you can never be completely sure with data like this.

Other represents all the mom and pop operations out there. There are many, but I am just as concerned about all the CMOs who HAVEN'T been studied by Mathematica: Academica, White Hat, Noble, UNO, CS-USA, Imagine, NHA, and so on.

We basically know NOTHING about these guys, yet they are allowed to occupy large segments of the charter sector. For the life of me, I don;t understand how this has been allowed.

teachingeconomist said...

So you added all the charter schools that do not have a contract with an EMO to the data set of charter schools with contracts to EMO constructed in Miron and Gulosino?

Miron and Gulosino only had a little over 2,000 charter schools in the 2011-12 data set, but the NCES estimate is that there were over 5,600 charter schools in 2011-12. Did you nearly triple the size of Miron and Gulosino's data set?

Duke said...

I don't know the exact proportions but we certainly added many. NJ, for example, only had two charters in Miron/Gulosino inked to an EMO (TEAM/KIPP and North Star/Uncommon). Any schools marked as charters in the NCES data would have been added. We also merged to CRDC school-level data.

Read Miron/Gulosino for their description of their methods (Appendix E).

Our dataset has 5,130 charters and 88,309 district schools. I'd have to go back and see if we used the CRDC or NCES marker for charter status (I think NCES). We identified 4,717 "non-EMO" charters, based on Miron/Gulosino. Of course, Bruce's chart is not based on number of schools but on enrollment.

Formal description of how we made this dataset is forthcoming...