Cerf's Problems #3: Data Abuse

Here again are the three problems I have with ACTING NJ Education Commissioner Chris Cerf:

1. He plays fast and loose with standards of ethics and transparency.
2. He has accrued too much power to himself.
3. He misrepresents research, data, and facts to push his policies.

Time to take a look at #3.

Tomorrow, the day of Cerf's long-awaited confirmation hearing, it will have been 507 days since Cerf promised a report on charter schools "as quickly as is humanly possible."

Let's remember exactly why Cerf promised this report. Over and over again, Cerf has touted the successes of charter schools. Yet he fails to acknowledge a simple truth: "successful" charter schools in New Jersey are less likely to educate children in poverty, children who are Limited English Proficient, and children with special needs than neighboring public schools.

Dr. Bruce Baker, a professor of school finance at Rutgers, has taken issue with Cerf and the NJDOE on this issue:

"Charter schools are dramatically outstripping traditional public schools in Newark," Cerf said.

Cerf also promised to "increase transparency" and expand the quantity of data available online.

The report is available at www.state.nj.us/education/chartsch/research/interim.pdf

But the data released did not go one step further, to link specific performance on test scores to children at the different income level — showing, for example whether schools with a higher percentage of children receiving reduced-price lunch, compared to a free lunch, achieve higher test scores.

Cerf said New Jersey’s state aid formula, and federal accountability programs, do not distinguish between the two levels of poverty. "Most charters in New Jersey cross a threshold of concentrated poverty that makes these distinctions meaningless," the report read.

Critics disagree, however.

"They have no basis for arguing that this is a trivial distinction. This is spin," said Bruce Baker, an associate professor at the Rutgers Graduate School of Education. He had commented widely on the January data and was, in fact, mentioned by Cerf at the meeting.

"The data just aren’t precise enough to make any reasonable conclusions about relative performance of charters versus district schools," Baker said. "Some of this is better than previous information, but there are certainly mis-statements and spin."

Baker said charter schools still serve many fewer special education students than public schools, and that the data on special ed does not break out how severe the children’s disabilities are.

This is the key to understanding Cerf's claims about charter "success": he refuses to acknowledge that these charters serve different populations of students. Baker has documented this difference extensively based on the data that is publicly available. He has also, unfortunately for Cerf, documented how mendacious the NJDOE can be when making these claims of charter success:

The Star Ledger reporters, among others, were essentially reiterating the information provided them by the New Jersey Department of Education. Here’s their story.

http://www.state.nj.us/education/news/2011/0118chart.htm

And here’s a choice quote from the press release:

“These charter schools are living proof that a firm dedication to students and a commitment to best education practices will result in high student achievement in some of New Jersey’s lowest-income areas,” said Carlos Perez, chief executive officer of the New Jersey Charter School Association. He pointed to NJASK data for third grade Language Arts, where more than half the charters outperformed the schools in their home districts, and of those, more than 75 percent were located in former Abbott districts.

No spin there. Right? Just a balanced summary of achievement data, with thoughtful interpretation of what they might actually mean. Not really.

There are many, many reasons why the comparisons released yesterday are deeply problematic, and well, quite honestly, pretty darn meaningless. I could not have said it better than Matt DiCarlo of Shanker Blogdid here:

“Unfortunately, however, the analysis could barely pass muster if submitted by a student in one of the state’s high school math classes (charter or regular public).” [emphases mine]

Baker's objective look at the charter school data led him to arrive at this conclusion:

On average, this statewide picture is actually pretty ugly. It would certainly be very hard to argue that charter school expansion across New Jersey has led to any substantive overall improvement of educational opportunities. Numerous charter schools are substantial underperformers. And overall, as the regression model indicates, the net performance is bread even. [emphasis mine]

Baker is also highly critical of the controversial report on Newark's schools produced by Cerf's old firm, Global Education Advisors:

Let’s be really blunt/honest here. This stuff is hack junk, and whoever is responsible for producing it really has no business providing recommendations on anything relating to schools/education, or for that matter the basic use/presentation of data.

But Cerf has never appreciated critics like Baker, who are well-versed in education policy and can't be snowed by data abuse. And he's responded in nasty and personal terms:

6:00: "Dr. [Bruce] Baker has never seen a reform he likes, so at least he's consistent on that point. He's against charter schools, against using data in any way, shape or form to evaluate teachers. I don't think he's been for any kind of accountability system when it comes to differentiating between excellence and mediocrity."

You'll notice Cerf has no substantive response to Baker; he merely questions his motives. Similarly, Cerf twisted Albert Shanker's words to support his position on "choice," even though it's clear, according to Shanker's family, that he would never support Cerf's initiatives. This, unfortunately, seems to be the new modus operandi at the NJDOE: questionable data and analysis, followed by personal invective against anyone who sees through the "hack junk."

This data abuse also extends to other areas of education policy. Here's Baker again:

I just couldn’t pass this one up. This is a graph for the ages, and it comes from a presentation by the New Jersey Commissioner of Education given at the NJASA Commissioner’s Convocation in Jackson, NJ on Feb 29. State of NJ Schools presentation 2-29-2012

Please turn to Slide #24:

The title conveys the intended point of the graph – that if you look hard enough across New Jersey – you can find not only some, but MANY higher poverty schools that perform better than lower poverty schools.

This is a bizarre graph to say the least. It’s set up as a scatter plot of proficiency rates with respect to free/reduced lunch rates, but then it only includes those schools/dots that fall in these otherwise unlikely positions. At least put the others there faintly in the background, so we can see where these fit into the overall pattern. The suggestion here is that there is not pattern. [emphasis mine]

Matt DiCarlo has also challenged Cerf's strange notions about poverty and student achievement:

 

This table shows that, in three out of four NAEP tests, both low- and higher-income cohorts’ scores have increased substantially, at roughly similar rates. In fourth grade math, students eligible for free/reduced-price lunch scored six points higher in 2011 compared with 2005, the equivalent of roughly half a “year of learning,” compared with a similar, statistically discernible five point increase among non-eligible students. The results for eighth grade math and fourth grade reading are more noteworthy – on both tests, eligible students in NJ scored 12 points higher in 2011 than in 2005, while the 2011 cohorts of non-eligible students were higher by roughly similar margins. 

In other words, achievement gaps in NJ didn’t narrow during these years because both the eligible and non-eligible cohorts scored higher in 2011 versus 2005. Viewed in isolation, the persistence of the resulting gaps might seem like a policy failure. But, while nobody can be satisfied with these differences and addressing them must be a focus going forward, the stability of the gaps actually masks notable success among both groups of students (at least to the degree that these changes reflect “real” progress rather than compositional changes). [emphasis mine]

This has been another theme of Cerf's: that New Jersey's schools, arguably the best in the nation, are really not very good at all, because they don't serve poor, urban children well. Yet many researchers have shown that New Jersey's poor and minority students have made substantial progress - mostly because of the state's commitment to funding equity. According to Linda Darling-Hammond of Stanford University:

I also describe how states like New Jersey, now arguably now the highest-achieving state in the U.S. if student demographics are taken into account, raised overall achievement and cut the achievement gap in half after being pushed by 30 years of school finance reform litigation to substantially increase spending in its poor urban districts. New Jersey – serving 45% minority students and a large and growing number of new immigrants – ranks in the top 5 states on NAEP on every measure and is first in the nation in writing, having invested in quality preschool for all children and quality pedagogy, with a focus on early literacy now expanding to other subject areas. [emphasis mine]

That commitment looks like it is in jeopardy under Christie and Cerf. Their justification is that New Jersey already spends too much on schools, has little to show for it, and must instead embark on an untested "reform" agenda that has never been shown to work as well as funding equity. Leave aside the irony that Christie wants to cut funding at the same time he crows about increasing funds to public schools (a bizarre statement, considering that Christie's initial cuts to schools weren't fully replaced). Leave aside the hypocrisy of Christie choosing to send his own children to elite private schools that spend far more per pupil than public schools that serve the most difficult and expensive children to educate.

What Cerf and Christie won't acknowledge is that funding equity is a necessary precondition for educational equity, and New Jersey has shown substantial progress under this premise. Their reform agenda - vouchers, charters, teacher evaluations based on testing - will not work, and cannot replace meaningfully implementing full funding equity.

It's this willingness to ignore issues of equity that leads to bizarre policies like holding charters accountable for educational outcomes but not diversity outcomes. It leads to an obsession with changing the way we count poor children in schools when there is little evidence that this is a serious problem. It leads to an astonishing denial that poverty actually matters in the lives of children:

The report, however, does not stop with a call to re-evaluate how poor students are counted.

Cerf also challenges the long-held assumption that poverty puts students at a disadvantage in the classroom.

His report recommends the state study "whether a poor student should be presumed to be educationally at-risk, or whether there is a more precise way to define at-risk students."

This is an incredible statement from a man who claims to be so "data-driven." The association between poverty and education outcomes is simply undeniable. As the ACTING Commissioner, Cerf should be demanding the state address childhood poverty; instead, he actually downplays its importance.

Another area where Cerf makes policy that flies in the face of accepted research is teacher evaluation. The NJDOE plans to use Student Growth Percentiles as part of teacher ratings; these SGPs will be based on standardized tests that have questionable grading practices and odd questions. There are serious questions as to whether these metrics can be applied fairly throughout the teaching corps, but perhaps the most troubling part of using SGPs is that they do not even attempt to isolate a teacher's effect on her students' scores. Baker again:

Put very simply, on its face, SGP is entirely inappropriate as a basis for determining teacher “ineffectiveness” leading to teacher dismissal.*** By contrast, VAM is, on its face appropriate, but in application, fails to provide sufficient protections against wrongful dismissal.

There are important implications for pending state policies and current and future pilot programs regarding teacher evaluation in New Jersey and other SGP states like Colorado. First, regarding legislation, it would be entirely inappropriate and a recipe for disaster to mandate that soon-to-be available SGP data be used in any way tied to high stakes personnel decisions like de-tenuring or dismissal. That is, SGPs should neither be explicitly or implicitly suggested as a basis for determining teacher effectiveness. Second, local school administrators would be wise to consider carefully how they choose to use these measures, if they choose to use them at all. [emphasis mine]

Cerf's insistence on using SGPs is potentially setting up the state for a major lawsuit when the first high-stakes decisions are made using them; and yet, he seems not to care.


Why is Cerf so reluctant to acknowledge high-quality research on charter schools, teacher evaluation, the effects of poverty, and other areas of education policy? The answer seems clear: Cerf is an ideologue. He doesn't adjust his views on the basis of the evidence; he adjusts the evidence on the basis of his views.

That's not the sort of leadership we need in the NJDOE right now. The K-12 public school system is the crown jewel of New Jersey's government; we can't risk its demise just because the Education Commissioner won't face some simple truths.