Don't Believe The Hype: The Facts About Graduation Rates in Newark and Camden, NJ (UPDATED)

This week, the NJ Education Policy Forum published a brief of mine on graduation rates in Newark and Camden. Not more than a couple of days later, however, the NJDOE published new graduation rate data for 2017.

So this post has two purposes: to update the data, and to explain what my analysis shows. Let me give some background first:

Over the years, two cities in New Jersey have been held up as national models for education "reform": Newark and Camden. Both are urban districts with severe economic challenges, both have many children living in poverty, and both have large populations of English Language Learners.

In addition, Newark and Camden have school districts that have been under state control for years: Newark since 1995, and Camden since 2013 (although Camden was subject to regulation by a state monitor well before that). Under state control, both cities implemented similar policies for their districts, including large expansions of charter schools.

Both cities have been run for the past several years by state superintendents who were appointed by former Governor Chris Christie. Cami Anderson was superintendent in Newark from 2011 to 2015, followed Chris Cerf, the former state Commissioner of Education, who just recently resigned. Paymon Rouhanifard was appointed by Christie in 2013 and remains in his role. All three cut their teeth in the New York City DOE during the time when Joel Klein, a master of self-promotion, was the chancellor of schools. None had ever run a school district before taking their positions.

In the past year, there's been a lot of talk about whether or not Camden and Newark have seen improvements during Chris Christie's two terms. As I explain in the brief, graduation rates are often brought up by supporters of Anderson, Cerf, and Rouhanifard as proof that their policies and leadership got positive results.

Just this past week, for example, the Star-Ledger gave a glowing review of Cerf's tenure: "The graduation rate has jumped to 77 percent, up by more than 20 points."* And last month the NY Times suggested Rouhanifard would make a fine choice to return to NYC and take over as chancellor: "In Camden, which was seen as one of the most troubled school systems in the country, Mr. Rouhanifard has improved graduation rates and lowered suspension rates."

The problem with claims like these is they lack context. It's great that graduation rates have climbed in Newark and Camden... but what if they've been climbing everywhere? What if the rises in graduation rates are part of an overall trend, and have little to nothing to do with these superintendents' policies or leadership? If that's the case, there's no reason to believe state control has led to better graduation outcomes.

So that's what I set out to explore: Did Newark's and Camden's graduation rates rise faster than other districts? Or is there a universal (what researchers often call a secular) trend that explains the rise? It sounds like a simple enough analysis, right? All we have to do is look at whether these cities saw their rates increase faster than the rest of the state...

Except there's a problem.

If you compare Newark's or Camden's graduation rate changes to all the other districts in New Jersey (which is what one author does here), you're going to have to deal with something known as a ceiling effect. I've blogged about this before:

You can't have a graduation rate over 100 percent, right? So there's a "ceiling" effect: because so many suburban districts were already graduating almost all of their students, they really couldn't improve. But more disadvantaged districts, with lower graduation rates, could and did improve.

It makes no sense to judge Newark or Camden against districts that have graduation rates near 100 percent and therefore can't get any higher. What we should do instead is judge their graduation rate growth against similar districts -- which is what I've done in the brief, and now updated here with 2017 data.

To determine which districts we should compare to Newark and Camden, I use the state's District Factor Groups (DFGs). Districts in the same DFG have similar socio-economic profiles; both Camden and Newark are DFG-A districts. The state began a new way of reporting graduation rates in 2011, so that's the baseline year. You can find all the rest of the technical information in the brief.

So, what do the data show?

The rise in graduation rates for Newark and Camden simply mirror a trend for all similar districts in New Jersey.

You can see this from the green line: all other DFG-A district saw their rates rise 11 percentage points over the past seven years. Camden's rose a bit less; Newark's somewhat more. But there's no doubt an overall trend in rising graduation rates for DFG-A districts explains a great deal of the trend in Newark and Camden.

Now, one of the arguments for state control is that they've allowed for charter school growth, and that's helped improve outcomes in Newark and Camden. As Bruce Baker and I explain in great detail, there isn't much evidence to support that contention. But let's go ahead and add charter schools into the mix -- how do Newark and Camden compare when we add in local charter school graduation rates?**

Again, it's clear that the statewide rise in graduation rates explains much of the growth in rates in Newark and Camden -- even when we add in charter schools. Yes, Newark does close the gap somewhat with other DFG-A districts. But I did some econometric modeling, and the change is not statistically significant. The updated regression outputs, using 2017 data, are below.

In addition: Baker and I found in our review that there are differences in outcome growth that appear to be correlated to regional differences; in other words, test score outcomes change in different ways in different parts of the state. So let's just look at the changes in DFG-A graduation rates for districts that are close to Newark -- in this case, within the same labor market.

Newark's graduation rates grew at almost the same rate as all the other DFG-A districts in its immediate area. And Camden?

There's only one other DFG-A district in Camden's labor market (Paulsboro). The data are noisy, but it's clear that Camden is not showing outsized growth in its graduation rate; if anything, the district is slipping.

I can only conjecture at this point, but I suspect the overall rise in graduation rates may be due to a policy called credit recovery. Newark pushed it hard in 2012, which might explain the initial bump in its graduation rate. Credit recovery includes a variety of programs -- include online learning, summer school, and so on -- that give students an opportunity to make up high school credits in subjects they previously failed.

Credit recovery is controversial: there's a lot of speculation that many programs used in credit recovery (especially on-line learning programs) are low in quality. There hasn't been a lot of study of credit recovery, and I don't know of any work specific to New Jersey on the issue.

In any case, no matter why the rates are rising, it's clear that they are rising in all DFG-A districts. Which means the claim that Newark's and Camden's graduation rate growth is due to the leadership of Anderson, Cerf, or Rouhanifard is little better than hype.

Look, I don't think it's wrong for any superintendent to promote the successes of their school district. But Newark and Camden have repeatedly been held up as exemplars, and reformy folks have pointed to their graduation rates as proof that charter school expansion and state control and "great school leaders" get results.

That's just not the case -- and it's unfair to districts like Paulsboro and Elizabeth and Orange and Irvington and Dover and East Orange, which have seen their rates rise just as much, if not more. Most of these districts haven't had large growth in their charter sector, and they remain under local control. Many of their superintendents are lifelong educators, and took the traditional route to district leadership positions: classroom, building principal, central office, with a stop along the way at a traditional university or college where they earned an advanced degree in educational administration.

Where is the applause for these leaders? Why aren't newspapers interviewing them? Where are the calls from editorial boards to bring them on board in the biggest urban districts? Why aren't they proof that local control of schools can yield good results?

I've noted this before: there are a lot of good schools and good school districts out there, serving many children living in economic disadvantage, that never get their due. They may not have leaders who relentlessly promote themselves, but that doesn't mean we should ignore them. And we certainly shouldn't foist policies on them when those policies come from districts that don't perform any better.

Here are the updated regression tables:

* The piece never spells out exactly what the baseline year was for comparison.

** Aggregating the rates of charter and public district schools is tricky stuff. See the brief for how I did it.