I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Monday, January 16, 2017

How To Correctly Compare Charter and Public District Schools

Why do states collect education data if they won't use it properly?

I found myself asking this question once again this week as I read through a "Comment/Response Form" put together by the New Jersey Department of Education and released earlier this month. The form was in response to the state Board of Education, which is evaluating a series of changes in charter school regulations. Those changes, as I wrote in my last post, include loosening certification requirements for charter teachers. 

The rationale for this, according to the state's charter cheerleaders, is that charters "do more with less"; in other words, they get better results and spend less money than public district schools. Chris Christie has repeatedly made this case in his push for the disastrous "Fairness Formula," which would rob urban districts, serving many more disadvantaged children, of necessary state aid. Christie points to the alleged efficiency of charter schools to justify these cuts: if they can "do more with less," why can't the district schools?

In their form, the NJDOE included this argument in response to a question from a member of the state BOE (p. 8):
12. COMMENT: The commenter asked for the per pupil cost for charter school students based on a random sampling of 10 percent of charter schools and their districts of residence. The commenter also asked if the proposed amendments to N.J.A.C. 6A:26-7.5 will affect the per pupil cost. (C) 
RESPONSE: The proposed amendments to N.J.A.C. 6A:26-7.5 will not affect per pupil costs for students attending charter schools. 
The 2014-2015 Taxpayers’ Guide to Education Spending was used to determine per pupil costs in charter schools and in districts of residence. The table below provides information on per pupil spending for eight charter schools that were selected randomly. Charter schools are ordered by the size of the difference in per pupil spending between the main sending district and the charter school. Per pupil spending in 2014-2015 in the eight charter schools ranged from $12,845 (compared to $23,466 in the main sending district) to $18,541 (compared to $22,013 in the main sending district). (emphasis mine)
This response is immediately followed by this chart:

Wow, look at that -- see how lean and mean the charter schools are compared to their hosting public district schools? Chris Christie must be right; the charters "do more with less"! We should be letting them open up all over the state!


Jersey Jazzman (artist's conception)

NJDOE analysts, please get out your pencils and take notes, as I present...


1) When comparing the spending of two school authorities, do not include services one provides for the other, or for the community at large.

The figures in the table above come from the NJDOE's Taxpayers' Guide to Education Spending. Yes, that's right, this is the state's own data. The figure used in this comparison is "Total Spending per Pupil, 2014-15." How does this state define this figure?
Total Spending Per Pupil was first developed in FY 2011 to provide a more comprehensive representation of district expenditures, since the former per pupil measures excluded some significant cost categories. This variable uses a larger enrollment number, including all students for which the district is financially responsible. The Total Spending measure adds the following items to the costs already included in the Budgetary Cost (Indicator 1):  
  1. pensions and social security payments made by the state on behalf of districts;
  2. transportation costs (including students transported to nonpublic and charter schools);
  3. judgments against the school district;
  4. all food services expenditures (including those covered by school lunch fees);
  5. capital outlay budgeted in the general fund (facilities and equipment);
  6. special revenues supported by local, state, and federal revenues (such as preschool, IDEA, and Title I);
  7. payments by the district to other private and public school districts for the provision of regular, special, and preschool education services (charter school students and their associated costs are only included in the charter school in which they are being educated).
  8. debt service for school debt; and
  9. an estimate of the district's share of the debt service that the state is paying for school construction bonds issued for school construction grants and School Development Authority projects.
The number of students sent to other entities (except charter schools) is added to the district's average daily enrollment in order to calculate the per pupil expenditure.  It should be noted that sent students and their associated costs are included in the per pupil cost of both the sending district as well as the school where the student is actually being educated.  Therefore, it is not appropriate to sum all districts' total expenditures, as this would overstate the aggregate cost.  This variable is calculated using audited (actual) data since some of the additional categories are not available in districts' budgets.  Two years of data are provided for comparison. (Emphasis mine)
This simple explanation makes clear that school districts provide many services that charter schools do not; it is, therefore, inappropriate to compare their costs. Take transportation: the total spending figure NJDOE uses for host districts includes the transportation costs for students attending charter schools. But why should a district have that counted in their comparative budget, but not the charter school whose students use that service?

The district pays certain costs to students who attend private schools; again, charter schools don't have to worry about that cost. The school has to pay legacy debt costs, meaning its budget includes students who have already graduated and left the district. Why should that be included in a comparison of current expenses? For that matter: if the community uses school buildings for a variety of reasons outside of school, why should the cost of maintaining that building only be put on the host district?

Charter schools also tend to enroll students in the lower grades, and many don't enroll pre-K students. How can we then compare the spending patterns of charter and district schools when grade level expenses aren't necessarily the same?

The thing that's really incredible about this comparison is that the NJDOE knows there is another figure -- Budgetary Per Pupil Costs -- that the department itself says is better for this purpose:
The Budgetary Per Pupil Cost(BPP Cost) section contains the Budgetary Per Pupil Cost and its subcomponents as they are reported for districts' User Friendly Budgets (required by N.J.S.A.18A:22-8.a).  While these costs do not provide an exhaustive picture of the cost for educating all students, they do allow school administrators and citizens to compare specific measures of school district spending.  Generally, the BPP measures the annual costs incurred for students educated within district schools, using local taxes and state aid. These costs are considered to be more comparable among districts, and may be useful for budget considerations. Examples of costs that are not included in the BPP are: expenditures funded by restricted grants, Teachers' Pension and Annuity Fund (TPAF), tuition payments to other districts and private schools, debt service expenditures, and principal and interest payments for the lease purchase of land and buildings.  Consistent with the exclusion of tuition expenditures, the measure excludes the enrollment for students sent out of district (Indicators 1 through 13, and 15). It should also be noted that budgetary costs for non-operating districts, Educational Services Commissions, Regional Day Schools, and Jointures are not included in this document. (emphasis mine)
BPP isn't perfect (for example, it doesn't address the grade level issue), but it's much better than Total Spending per Pupil. How do the charters and the districts compare on this measure?

Yes, the districts are still spending more, but it's much closer than the figures NJDOE gave in its memo. Still, why would the charter be sending less? Could it be...

2) When comparing the spending of two school authorities, you must take into account differences between their student populations.

I can't believe I still have to explain this to anybody, let alone the NJDOE, which should know better. Take, for example, special education:

As I've pointed out literally dozens of times: Charter schools, on average, do not serve nearly as many special education students as public district schools. Everyone acknowledges it costs more to educate a child with a learning disability; even Chris Christie doesn't argue that point! And yet NJDOE made their little table without even acknowledging this glaring problem with their comparison. Of course, the special education gap varies from city to city.

The data for Freedom Academy in Camden are clearly very noisy, but the overall trend is clear: charters don't serve nearly as many special education students as public district schools. In Hudson County, however, the story is more complex.

At first glance, you would assume some of the charters, like Elysian in Hoboken and University Academy in Jersey City, were picking up their fair shares of classified students. But there's a caveat...

3) Always acknowledge the limitations of crude data.

In this case, the problem is that special education classification is binary in the data; in other words, a students is either classified or isn't. But not all classifications are the same:

A "specific learning disability" is a lower-cost classification, unlike, say, autism or a visual impairment or a traumatic brain injury. For both Elysian and University Academy, the majority of their special education students are SLD; that's not true for their host districts.* So the public district schools are enrolling more of the students with costly disabilities compared to the charters. That explains a good part of the cost differential, as we'll see below.

Here's another difference that shows up in the data on charters in Hudson County:

To be fair: Jersey City Public Schools has a free lunch-eligible (FL) rate equivalent to University Academy; Soaring Heights' rate, however, is much lower. FL is a crude proxy measure for economic disadvantage, but it's the best one we've got. In Hoboken, the difference between the public district schools and Elysian is very large.

Again: schools are supposed to get more state aid when they enroll more FL students, because everyone acknowledges it costs more to equalize educational opportunity for disadvantaged children. Of course, if the charter cheerleaders don't agree, their beloved charters should stop taking more money for enrolling more FL students. Think that'll happen?

One more difference that matters:

Across the state, the education of children who do not speak English at home has been left to the public district schools; the charters have taken a pass. Once again: schools are supposed to get more state aid when they enroll LEP students, because it costs more to educate them. Ignoring this reality when comparing charter and public district school spending leads to a flawed analysis.

4) Remember that education is a human capital-intensive enterprise.

Think about those differences in special education rates while pondering this:

Support services include many functions, like child study teams, that are necessary for schools with special education students. It's obviously a significant part of a public school district's budget -- but not a charter school's. Those zeroed out column above aren't missing data; they're showing charter schools who simply don't report any spending on support services. I've always urged caution when interpreting these figures, because there may be reporting differences and data error.


At this point, given year after year of data, there is just no question about it: Charter schools spend, on average, much more on administration than public district schools. It just makes sense: as Bruce Baker points out in his latest report (which, sadly, has been completely misinterpreted by the usual suspects), small charter schools can't leverage economies of scale, and that manifests, to a large extent, in administrative costs.

And yet charter spending, overall, is still less. Again, part of that is the low amount charters spend on support services, a function of enrolling fewer classified and LEP students. But there's another factor...

5) Remember: it's easier to keep costs low when you have an inexperienced teaching staff.

One of the truly foolish things I hear from charter cheerleaders is that charters are taking advantage of  millennials' alleged desire for temporary careers. First of all: we know experience matters, especially in the first few years of a teacher's career. Why, then, would it be a good thing to have charter schools where the average teacher's experience is less than 2 years, like Bergen A & S and Newark Legacy?

Second, do millennials really want to start at the bottom of the pay scale every time they change jobs? Because that seems to be what's happening with many charters:

On average, charter school teachers make considerably less than their public district school counterparts. As I've noted before, this difference holds even when accounting for differences in experience.

Look, if charter cheerleaders want to brag on "doing more with less," then fine: acknowledge you're doing that, in part, by paying your teachers less. Then explain to the rest of us how that's a good thing for the profession.

6) Remember: it's easier to keep costs low when you offer less expansive educational programs. As a music teacher, I think all students should have the opportunity to make music. But that's hard to do if you don't have the personnel to make a program:

This is a technique I've used before: looking at the staffing files to determine the extent of programs in areas like music. The first thing to look for is whether schools actually have teachers in these specialist areas: in the case of Newark Legacy, Soaring Heights, and Freedom Academy, the staffing files suggest music just isn't a part of those charters' curricula.

The next thing to watch is how many students each specialist teacher has for their "load." At North Star -- which compares itself to the most affluent suburban districts in the state -- the music teacher has a much greater student load than music faculty in the Newark Public Schools. That means it's much less likely North Star has the bands and choruses and orchestras NPS can offer their students -- they just don't have enough teachers to make it happen.

That said, it's a mixed bag. Some of the charters do quite well on music; how do they do in other areas?

According to the staffing files, Barack Obama CS in Plainfield had a music teacher, but not a health/PE teacher. And that's understandable, if not acceptable; it's hard for a small charter school to offer everything a public district school can. And maybe that's the takeaway here...

As I have said, many, many times on this blog: I really don't have a problem with charter schools per se. I started my career in a charter. I have seen first-hand that some kids just don't thrive in a "regular" public school, and might do better if given a "choice." We can and we should try to innovate in our schools.

But let's not fool ourselves about how and why charter schools "do more with less." Advantages in student populations; advantages in staffing costs; unequal curricular programming and support services: these are the reasons for the differences in costs between charters and district public schools.

Keeping this in mind, let's step back a bit and think about how I did this analysis. This is all based on data collected by the NJDOE. The reason they collect the data, supposedly, is that they can then analyze it to present to policy makers -- like the state BOE -- so they can make good decisions.

But that is most certainly not what happened here. Instead, when a member of the board asked a reasonable question,** the NJDOE gave a facile, cursory answer that matched Christie's ideological predilections.

That is a very bad way to make policy. It's a disservice to the many students and families in this state who are looking for better education and better schools. It's an abdication of the duty a state department of education has to the citizens of its state.

There are some really good people at NJDOE, as there are at all the state departments of education. But too many ideologues are at the top, and they are failing in their jobs. 

Step up, folks. Do the work.

ADDING: One thing I didn't get to -- and I will get to this one day in a comprehensive way, I promise -- is how charters like Elysian benefit from substantial philanthropic giving.

This is Elysian's campus, in just about the most affluent section of Hoboken. Public money is in no way the only revenue that drives programming and provides facilities at this school. More to come...

* There's a lot of suppression in the special education data, ostensibly to protect the privacy rights of classified students. It's so prevalent that I really couldn't make useable graphs for speech disabilities, another lower-cost classification. SLD is more important for this discussion, however, because charter schools get more funds for enrolling non-SPL classified students -- but there isn't a distinction between SLD and other higher-costs classifications when the aid calculations are made. This is a  complicated topic I get into more here.

** Well, somewhat reasonable: why ask for just a sample of charters? Why not analyze all of them? It's not impossible -- I did it. In fact, why not ask someone like me, who's already done the research, to answer the question?

Unless you don't really want to hear the answer...

No comments: