Charter School Propaganda: A Case Study, Part II

All this week, I've been debunking the Star-Ledger's big, fat, wet kiss to TEAM Academy Charter School, the Newark branch of the national charter chain KIPP.

First, I showed how the author of this piece, Julie O'Connor, displayed what can only be described as willing ignorance about how "successful" charter schools get the gains that they do. Bruce Baker and I patiently (OK, not always patiently...) explained to O'Connor the realities of charter school demographics, funding and outcomes -- but O'Connor either wouldn't or couldn't understand.

Next, I showed how there are dozens of schools -- both public and charter -- that, by O'Connor's own criteria, should also be considered "jewels." Yet O'Connor and her boss, Tom Moran, have shown no interest in any of them; maybe because those schools don't have full-time communications directors, like TEAM/KIPP does, who can spoon feed newspaper op-ed writers the masticated pablum they crave.

I then explained how the "facts" TEAM/KIPP obviously fed O'Connor are, at best, completely lacking in context and, at worst, just flat-out wrong. Because O'Connor clearly didn't do any of her own research, she simply accepted TEAM/KIPP's spin as gospel truth, never challenging herself to think critically about what she was being told.

I don't want to beat this to death. But I also don't want a couple of other assertions in this piece to go unchallenged:

- One single study does not make or break an argument -- especially if that study is irrelevant. In her piece, O'Connor cites a study by the research group, Mathematica:

Traditionalists also don't like the idea of breaking off from the main public school system. They argue KIPP's success is too good to be true, a product of statistical quirks that can't be replicated with an entire district of students. 

But these critics seemed blinded by ideology, a misplaced loyalty to educational tradition. Mathematica, a respected non-partisan research institute, studied KIPP schools nationally and found they did better than traditional schools, even with similarly disadvantaged kids. 

When readers pushed back in the comments section at nj.com, O'Connor repeatedly cited this study as evidence of TEAM/KIPP's sparkle:

@Joseph Addison Repeating my comment on attrition below, in case you missed it: Here's how KIPP responds to the criticism that it has higher student attrition for black males: http://blog.kippnj.org/attrition. The national Mathematica study, by a nonpartisan institute, matched at the student level to control for demographics and also looked at student attrition. http://educationnext.org/student-attrition-explain-kipps-success/ 

@NJParents1 You're right that all charters aren't better than district schools - some are worse. It really depends on the charter. Here's how KIPP responds to the criticism that it has higher student attrition for black males: http://blog.kippnj.org/attrition. The national Mathematica study, by a nonpartisan institute, matched at the student level to control for demographics and also looked at student attrition. http://educationnext.org/student-attrition-explain-kipps-success/

@JuliaSassRubin I understand your position, Julia. But the two schools most requested by parents in Newark under universal enrollment were North Star and KIPP. When 10,000 parents line up on a charter school waiting list, it speaks volumes about their preferences. I don’t think anyone could argue that KIPP is an unwelcome interloper being imposed upon this community.
I think Professor Baker raises good questions in his research, but his model used school level data. The Mathematica study, by a nonpartisan institute, matched at the student level to control for demographics, and also looked at student attrition. http://educationnext.org/student-attrition-explain-kipps-success/Although it was a national study, I didn’t think Baker made the case for why his extrapolations are a better measurement. And KIPP sends 95 percent of its students to college. That in itself is impressive.

Now, I have some problems with the Mathematica study; or, more precisely, I have problems with how it has been interpreted. I'll leave it to Bruce Baker to spell out his concerns (I have some others that I will try to get to later). But there's an important bit of information O'Connor is withholding from her readers about this study:

The study group for the Mathematica reports on KIPP charter schools does not include TEAM Academy -- the very school O'Connor is hyping. 

This was confirmed on Twitter by TEAM/KIPP's own data analyst. Go to page xvi of Mathematica's 2013 report on KIPP and you'll see that New Jersey is excluded from their study. TEAM/KIPP's analyst confirms this is the same study group used in the report O'Connor cites. 

Why does this matter? The Mathematica report compared students at KIPP schools around the country to students in "feeder" district schools. But New Jersey is one of the top-performing state systems in the nation: what if the gap between Newark Public School's (NPS) students and TEAM's isn't as great as in other parts of the country because New Jersey's public school students perform relatively better?*

At the very least, O'Connor should have told her readers TEAM/KIPP was not part of the study she cites repeatedly. This, of course, assumes that she even knew.

- Anecdotes do not supersede data -- and the data says that TEAM/KIPP does not "do more with less."

O'Connor tells a story about air conditioners to "prove" that TEAM/KIPP knows how to spend a buck much better than NPS or the Camden City Public Schools (CCPS). She says the charter school is free from the "wasteful central bureaucracies of district schools," but she never bothers to look at universal, publicly reported data to back up her claim.

There is only one source of state-level data for comparing district and charter school spending: the Taxpayers' Guide to Education Spending, a product of the NJDOE. I have data from the 2014 guide (the 2015 guide was recently released, but I haven't had time yet to prepare it for analysis), and I use it here to show how TEAM/KIPP, NPS, and CCPS compare in per pupil spending.

On the advice of the NJDOE itself, let's look at the differences in Budgetary Costs Per Pupil. As the state says:

The Budgetary Per Pupil Cost (BPP Cost) section contains the Budgetary Per Pupil Cost and its subcomponents as they are reported for districts’ User Friendly Budgets (required by N.J.S.A.18A:22-8.a). While these costs do not provide an exhaustive picture of the cost for educating all students, they do allow school administrators and citizens to compare specific measures of school district spending. Generally, the BPP measures the annual costs incurred for students educated within district schools, using local taxes and state aid. These costs are considered to be more comparable among districts, and may be useful for budget considerations. Examples of costs that are not included in the BPP are: expenditures funded by restricted grants, Teachers’ Pension and Annuity Fund (TPAF), tuition payments to other districts and private schools, debt service expenditures, and principal and interest payments for the lease purchase of land and buildings. Consistent with the exclusion of tuition expenditures, the measure excludes the enrollment for students sent out of district (Indicators 1 through 13, and 15). It should also be noted that budgetary costs for non-operating districts, Educational Services Commissions, Regional Day Schools, and Jointures are not included in this document. [emphasis mine]

This is an important point that must be understood if we're going to compare TEAM/KIPP to district school systems. NPS and CCPS have expenses they have to cover that TEAM/KIPP does not. For example, the schools have to transport all students in their borders, including the charter students. TEAM/KIPP relies on NPS and CCPS to pay those transportation costs for their students; it makes no sense, then to simply compare the overall budgets of district schools and charter schools.

In addition:

NPS educates far more children with a special education need proportionately than TEAM/KIPP. It costs more to educate these students, and NPS has more of them. Did O'Connor ever think about this?

All that said, let's see how TEAM/KIPP, NPS, and CCPS stack up in their spending:

Click to enlarge. Let's start with the leftmost columns: yes, KIPP/TEAM does spend less overall per pupil than either NPS or CCPS. But how does this break down?

The next set of columns shows instructional spending: the amount of money that goes into the classroom, largely to the salaries of staff. Again, NPS and CCPS clearly spend more per pupil than TEAM/KIPP. The next set of bars shows that the charter spends less on those teacher salaries. Of course, their staffs are less experienced, so that makes sense...

But the next set of bars tells the real story. Here's how NJDOE describes "Student Support Services":

This indicator includes expenditures considered student support services under the National Center for Education Statistics (NCES) definition - services supplemental to the teaching process that are designed to assess and improve students’ well-being. It also includes expenditures for activities associated with assisting the instructional staff with the content and process of providing learning experiences. Attendance, social work, health and guidance services, educational media/school library services and child study team services are student support services under the NCES definition. This area also includes the costs associated with physical and mental health services that are not direct instruction, but are nevertheless provided to students, such as supervision of health services, health appraisal (including screening for vision, communicable diseases, and hearing deficiencies), screening for psychiatric services, periodic health examinations, emergency injury and illness care, dental services, nursing services and communications with parents and medical officials. The expenditures of the guidance office includes counseling, record maintenance, and placement services. The costs for the child study team include salaries and benefits for members related to the development and evaluation of student individualized education programs (IEPs). Services provided as a result of IEPs are considered instructional costs and are included in the appropriate classroom instruction indicators. The school library services include books repairs, audiovisual services, educational television services, and computer assisted instruction services. The actual provision of computer assisted instruction is considered classroom instruction. [emphasis mine]

So this category is much of the spending needed for children with special education needs: guidance counselors, child study teams, health services, and so on. Not that these aren't important for general education students as well; however, it's reasonable to expect a district with more special needs children will spend more on these services.

Camden's and Newark's district schools far outspend TEAM/KIPP on student support services.  It's possible that TEAM/KIPP reports these expenditures under different lines, and that's fine... except they spend less on instruction as well. If they're spending on support services, where are they reporting it?

One more thing: O'Connor told her readers TEAM/KIPP doesn't have a "wasteful central bureaucracy" like CCPS and NPS. But the fifth and sixth sets of columns above show the opposite: TEAM/KIPP spends far more on administration than Camden's and Newark's district schools, and much of that additional spending is on administrator salaries. 

Notice, also, that TEAM/KIPP spends far more on its physical plant than NPS or CCPS -- but not on plant salaries.

So who, exactly, is being "wasteful" here?

It's time to move on to other topics, but I think I've made my point: when it comes to education policy, the Star-Ledger's editorial board -- like much of our punditocracy -- is in the tank. It's quite clear to me that O'Connor didn't do her own research, relying instead on pre-chewed talking points from an interested party that was happy to use her to pump itself up.

May I make a confession? I find this work to be both exasperating and exhausting. Is it really too much to ask the op-ed page of the state's largest newspaper to be a little less credulous and a little more demanding? Is it too much to ask they check things out for themselves, rather than simply dictating whatever they're being fed by whomever happens currently to be in favor? Doesn't the Star-Ledger's editorial staff have any pride in their work anymore?

I know regular readers will think I'm being sarcastic here, but I'm really not -- I still hold out hope for O'Connor and Moran. I hope that they are so chastened by this episode that, even if they hold on to their reformy opinions, at least they won't ever again blindly accept whatever fluff is found in the latest press release from the big national charter chains. I hope they care half as much about the credibility of their profession as the hard-working, dedicated public school teachers of New Jersey care about theirs -- if they do, they'll never be such pawns of the charter industry again.

And I hope they learn that it's always better to chew your own data.

Saturady Night Live - Pre-Chew Charlie's... by ClassicPL

* This is one of the methodological flaws in the Mathematica report, by the way: they should have broken down the differences by state, rather than mashing them all together. There is reporting on the variation across the study sample, but nothing to indicate as to whether the variation may have something to do with differences within the entire comparison group.