I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Tuesday, August 23, 2016

Chris Cerf's Late Conversion On School Funding and Testing

UPDATE: Looks like Chris Cerf went a little too far this time:
On Monday, Christie’s former education commissioner Christopher Cerf, who is now running the Newark school district, told NJ.com that the governor’s funding plan would be “catastrophic” for the city. Christie said he had a “spirited discussion” with Cerf last night.
"Spirited," huh?

I wonder how much more "spirit" Cerf can take...

* * * 

Chris Christie's "Fairness Formula" is so blatantly unfair and illogical that even Chris Cerf -- Christie's former Commissioner of Education and current State Superintendent of Newark Public Schools -- can't support it:
The superintendent appointed by Gov. Chris Christie to run Newark's state-controlled school district said a 60-percent cut in aid projected under a new funding formula proposed by the governor would be "catastrophic" for the district.  
"I don't mind saying explicitly that a reduction in our budget of 60 percent would be catastrophic," said Superintendent Christopher Cerf, a former state education commissioner under Christie, who appointed Cerf to run the state's largest district last year.
Cerf's comment was in response to a reporter's question about the impact on the district of a projected 60 percent cut in aid under the new Fairness Formula proposed by Christie.
The formula, unveiled in June, would dole out precisely the same amount of aid per pupil — $6,599 — to all New Jersey school districts, regardless of affluence, resulting in a savings for 85 percent of the state's property taxpayers, while translating into dramatic cuts in aid to poor districts that rely on the state to fund a much higher proportion of their school budgets. 
An analysis by NJ Advance Media found that aid to Newark and some other poor districts would be cut by 60 percent or more under the governor's proposed formula. [emphasis mine]
As I pointed out last October, this is quite a shift in thinking for Cerf. When he was in Christie's administration, he used to go around the state saying NJ's urban districts were getting way too much money:
Pumping more money into our worst-performing districts has provided us with moral cover, persuading us that we have met our obligation to the students in those districts while allowing us to under serve them.More money has permitted past governors and legislatures to avoid the politically difficult reforms – like implementation of an educator evaluation system, tenure reform, and ending the pernicious “last in, first out” policy – so critical to turning around our lowest-performing schools. And more money has likewise allowed the Department of Education to be satisfied with a role as district compliance-monitor rather than district partner, collaborator, and, where necessary, instigator of seismic reform. [emphasis mine]
Funny how actually running a district causes you to think about this stuff a little differently, isn't it? This, among other reasons, is why I said years ago that Cerf wasn't qualified to be the Commissioner: he had never run a public school district, let alone one with many students living in poverty, so he had no idea of the challenges faced by district leaders who saw their state aid plummet under Christie.

But Cerf's flip-flopping isn't limited to school funding. The State Superintendent was interviewed last week on NJTV, where he laid out his terms for what should constitute "success" when judging the Newark schools:
Williams: Speaking of QSAC, every district in the state is governed, is tested by, is accessed by QCAS — you won’t be. Newark is going to be accessed by growth. That’s pretty broad. 
Cerf: One of those five domains I was mentioning is called instruction in program, and that one we applied for what’s called — I’m sorry for the technical terms — an equivalency waiver. Essentially what that is is measure us by student outcomes, but measure them predominately by outcomes that are based on growth rather than absolute proficiency. [emphasis mine]
Wait -- is 2016 Chris Cerf saying it's unfair to ask a district with many students in poverty to be judged by the same standards as a district with few of those students?

Because 2012 Chris Cerf was tired of those "excuses":
Some would say we should not put such effort into these schools. They will say these schools are low-performing because of poverty, and that until we fix community and family issues, we can’t do much better in the schools.  
Of course, poverty matters. And of course, it affects a child’s experience in schools. But rather than allowing these circumstances to be used as an excuse for inaction, we should redouble our efforts as educators to make sure we do everything in our power to provide great options for these students. We have too many examples of schools and great teachers overcoming the constraints of poverty to believe we can’t do any better. 
I refuse to work within a system that accepts that the circumstance into which a child is born should determine his outcome in life. And I refuse to believe that great public school teachers cannot make a difference in a child’s life. 
Let’s work together over the next several years to give all students in New Jersey equal opportunities for success, and let’s hope that the support of expert educators in our RACs will help to turn around low-performing schools. [emphasis mine]
In fact, 2012 Cerf "proved" that poverty could be overcome with his magical graph that showed poverty doesn't matter! Bruce Baker called it out in real time:
Now, it’s one thing when and under-informed tech CEO goes all TED-style on us with big screens, gadgets, bells and whistles and info-graphics that just don’t mean crap anyway. But, it’s yet another when a State Commissioner of Education presents something not only equally ridiculous… but arguably far more ridiculous, disingenuous, unethical and downright WRONG.
This is a graph for the ages, and it comes from a presentation by the New Jersey Commissioner of Education given at the NJASA Commissioner’s Convocation in Jackson, NJ on Feb 29. State of NJ Schools presentation 2-29-2012
Slide4
The title conveys the intended point of the graph – that if you look hard enough across New Jersey – you can find not only some, but MANY higher poverty schools that perform better than lower poverty schools.
This is a bizarre graph to say the least. It’s set up as a scatter plot of proficiency rates with respect to free/reduced lunch rates, but then it only includes those schools/dots that fall in these otherwise unlikely positions. At least put the others there faintly in the background, so we can see where these fit into the overall pattern. The suggestion here is that there is not pattern.
Note: this graph may not even be the worst one in the presentation. You decide!
The apparent inference here? Either poverty itself really isn’t that important a factor in determining student success rates on state assessments, or, alternatively, free and reduced lunch simply isn’t a very good measure of poverty even if poverty is a good predictor. Either way, something’s clearly amiss if we have so many higher poverty schools outperforming lower poverty ones. In fact, the only dots included in the graph are high poverty districts outperforming lower poverty ones. There can’t be much of a pattern between these two variables at all, can there? If anything, the trendline must be sloped uphill? (that is, higher poverty leads to higher outcomes!)
Note that the graph doesn’t even tell us which or how many dots/schools are in each group and/or what percent of all schools these represent. Are they the norm? or the outliers?
Well, here’s what the pattern really looks like with all schools included:
Slide5
Hmmm… looks a little different when you put it that way. Yeah, it’s a scatter, not a perfectly straight line of dots. And yes, there are some dots to the right hand side that land above the 65 line and some dots to the left that land below it.
2012 Cerf says those red diamonds prove that schools can "overcome the restraints of poverty." Well, if that's true, 2016 Cerf, why don't you think NPS should be judged on absolute measures of student proficiency? Why should "growth" be how your schools are judged four years after you told us all "we have too many examples" of schools beating the odds?

I really can't think of a better example of why state and federal education policy leaders should have significant classroom and/or administrative experience before they are nominated. Not that experience guarantees they will promote good policies; however, a state commissioner who has actually been down in the trenches and understands how difficult it is to improve public schools is much less likely to indulge in fantasies about how schools will perform in the face of chronic poverty, inequality, racism, and declining funding.

If Chris Cerf ever rises to another leadership position in education policy, I hope he remembers just how tough it is to effect change in a district like Newark. I hope he won't make excuses for politicians like Chris Christie who won't give schools what the state's own law says they need. I hope he won't tut-tut at teachers and administrators who point out that while schools can and should improve, they certainly can't overcome, on their own, a host of problems they never created.

Accountability begins at home.

Monday, August 22, 2016

John Oliver on Charter Schools: A Supplement

John Oliver's piece last night on charter schools is getting a good bit of attention in the education policy bloggosphere:



I may have some more to say about it later, but I thought some of you might enjoy some further reading about some of the issues Oliver brought up.

- Pitbull and Charter Schools:

Charter Schools, Pitbull, & Money, Money, Money

Make Big Money Building Charter Schools: Joe Bruno Shows You How!

Pitbull Speech at Ntl Charter Schools Convention - The Remix (this remains a personal favorite if only because it was one time I let my snark run truly free)

- KIPP and its "Success" in Context:

The KIPP Propaganda Machine and Its Willing Saps In the Media

The Public School "Jewels" The Media Ignores

Charter Schools, An Exchange: Part II (skip to the "KIPP" section)

And it's worth pointing out once again that KIPP, no matter what you think about their "successes," is a tiny part of the entire charter sector:


- The CREDO Studies (which Oliver cites to claim charters and public schools are about even in performance):

Charter Schools, An Exchange: Part II (skip to "CREDO" section). This is how charter effects, as reported in the CREDO studies, compare to the effects of economic inequality:



- General Charter School Myth Busting: 


- Florida Charters:

Miami-Dade's Charters Don't Serve the Same Students

Are Miami-Dade's Charter Schools Really "Impressive"?

Does DFER Support Pitbull-Zulueta Values? (Click through the link in the post for the Miami Herald's terrific series "Cashing In On Kids")

-Why Pennsylvania's Charter System Is Such a Mess:

The Selling Out of Camden's Schools: Part I (yes, this does concern PA -- trust me, and read Part II)

- Why John Kasich's "Pizza Shoppe" Analogy Makes No Sense:

Democracy, Markets, & Charter Schools: Part I


Again, I may have some more to say about Oliver's piece, which I thought was good if necessarily incomplete. Until then, see you at the pizza shoppe.

More pepperoni, please.

Sunday, August 21, 2016

The Last Thing Atlantic City's Students Need Is School Vouchers

Atlantic City is in big fiscal trouble, and that includes its schools. So what do its leaders think the state should do? An infusion of aid? A new approach to curriculum? A full-frontal assault on childhood poverty?

Nope -- what the kids really need is "choice"!
Atlantic City’s MUA debacle (part of the larger state takeover debate) overshadowed a major development at Wednesday night’s council meeting, Save Jerseyans. 
This November, when voters in A.C. head to the polls, they’ll pass judgment not only on the state-wide casino gaming referendum but also a pair of city-specific school choice ballot questions. 
The questions will read as follows:

Shall the State of New Jersey designate the City to begin offering vouchers to families with children ages 6-16 so they can select the school they want their children to attend?
 
and

Shall the State of New Jersey designate the City of Atlantic City to begin offering property tax credits to families with children ages 6-16 who choose to home school?
 
The resolution (see attached here) was passed unanimously by the Democrat-dominated body and was filed with the Atlantic County Clerk by the August 19th deadline for submitting a non-binding referendum in time for the November ballot. It’s the brainchild of our friend, freshman GOP Councilman Jesse Kurtz, who is himself an NJEA member at Atlantic Cape Community College.
Yes, they are completely serious:
Besides the obvious benefit to students in one of the state’s largest struggling education districts, Councilman Kurtz sees massive near-term and long-term gains for taxpayers.
“It both gives poor families the means to receive the education of their dreams and would save taxpayers between $12,000-to-$15,000 per student who leaves the public school system and opts for a voucher — $5,000 for elementary, $8,000 for high school — not to mention the opportunity for educational innovation with the small private schools that would pop-up to offer different educational options and compete for the vouchers.”
Councilman, can I point out something you probably should have considered before you put this cockamamie idea before the public?

Last year, Atlantic City Public Schools enrolled 7,130 students. By contrast, the two private schools in AC that were listed in the NCES Private School Universe Survey enrolled a total of 168 K-12 students. Not one was in high school.

If, magically, these two schools were able to double in size, they would still only be able to enroll about 2 percent of Atlantic City's students -- again, not one of them would be in high school.

Our Lady Star of the Sea, the far larger of the two, is a Catholic school. Oh, your family's not Catholic?

Too bad.

Of course, we could open up all of Atlantic County to the high school kids. Problem is there are only three schools in the whole county that enroll more than 30 students in a class (all are Catholic). All require admissions applications; one explicitly states it receives more applications than it can accept (and has a tuition of $16,900 a year, not including transportation). What do you think the odds are that these schools can make room for all of ACPS's 1,948 high school students? Especially if they're only getting $8,000 a student in tuition?

The idea that "small private schools would pop-up to offer different educational options and compete for the vouchers" is a libertarian fantasy that has no basis in reality whatsoever. Atlantic City's public school students need well-funded schools right now -- not when some dopey Ayn Rand-ish dream comes to pass, but today.

The last thing Atlantic City should do is divert already scarce funds into a voucher program that would, at best, move a few children into religious schools that would need extra (and costly) oversight after having received public monies.

Someone should tell Councilman Kurtz and the rest of the city council to stop with these goofy schemes and get serious about properly funding their city's schools.

Didn't work out so well, did it?


ADDING: Here's the state's education commissioner on what it would be like to move AC's students to other districts:
New Jersey Education Commissioner David Hespe said Tuesday he is charged with ensuring that the school district receives the local tax levy revenue the city collects on its behalf.
"That money is the school district's money, and, without that, I will have to close the schools of Atlantic City," he said. "Not sure what that moment in time is, but I can tell you I have already started that process by not paying certain vendors of the school district."
If the schools close, Hespe said, moving students to other districts is doubtful.
"There is not much capacity in that area for moving large numbers of children, and that would be very disruptive to their education, and exactly the reason that we went to court because there aren't any solutions," he said. [emphasis mine]
But somehow Kurtz thinks he can find a way to move lots of kids to private schools, and that won't be disruptive. Totally makes sense...

Jersey Jazzman (artist's conception)


ADDING MORE: You might wonder why Councilman Kurtz wants to include homeschoolers in his voucher scheme. After all, current homeschoolers aren't a burden at all on the city's finances; why give them extra funding at a time when the city is in crisis?

Yes, it's exactly as bad as you think: Kurtz is homeschooling his own children. I've confirmed this from two sources.

I have to wonder what the good people of Atlantic City think about this.



When It Comes To Schools, Money DOES Matter -- Even in Michigan

I've become something of a connoisseur of a particular genre of opinion writing: the "We can't just throw money at schools!" op-ed. These pieces have a style all their own: they use the same talking points, the same context-free data points, and the same appeals to the same authorities. 
The goal of these pieces isn't to give a nuanced view of the role of funding in public education. Instead, they exist to place just enough doubt into the reader's mind about the need for equitable and adequate school funding so the status quo of public schools with mushrooms growing on the walls seems almost acceptable -- or, at least, better than the alternative.
Here's a fine example of the style -- Ingrid Jacques in the Detroit News:
Michigan is at an education crossroads. As its public schools continue to plummet in performance, state leaders can either demand proven accountability measures and smart investments — or they can take the easy way out.
In this case the easy way is to call for more money. And that’s exactly what school unions and administrators are doing. The State Board of Education is also singing that tune.
Jacques undoubtedly knows that calling for increased funding for any government function these days is hardly "easy." Decades of conservative rhetoric (transmitted through outlets like Jacques' editorial page) have made it nearly impossible for even the most liberal politicians to advocate for significant tax hikes to support public programs, especially education.

The "easy" way to justify the horrible conditions found in Detroit's (and Michigan's other disadvantaged cities') schools is to pretend that all sorts of vaguely described "accountability" systems must be put in place before necessary funds are allowed to flow to public education. Because, as everyone knows, we spend so very much on our schools:
Austin is right about how Michigan students are falling behind. Studies have shown how the state’s students are often in the bottom 10 for performance in key subjects on the National Assessment of Educational Progress. And that includes all students — regardless of race or income.
But he’s wrong when he tries to pin most of the blame on money — and a Republican governor and Legislature. This year’s School Aid Fund budget has increased to over $14 billion; students across the state are getting a boost.
OMG! $14 billion! That's obviously insane!!!

$14 BILLION dollars!

Rather than citing utterly useless a-contextual figures, Jacques should have gone to the School Funding Fairness Report Card, and seen for herself how Michigan actually compares with other states in its effort to fund schools:
The National Report Card 4th Edition

State 
Distribution 
Effort 
Funding Level 
Coverage 





MichiganCB2522







Michigan is right in the middle, when judged by figures appropriately adjusted for regional differences. And yet the state is quite mediocre when it comes to distributing funds to more disadvantaged communities where it's needed the most.

These op-ed pieces also inevitably bemoan the poor performance of the author's "high spending" state:
It’s helpful to look at how Michigan compares with other states that are getting good returns on investment.
According to the latest data from the U.S. Census Bureau, in 2014 Michigan came in 20th with $12,856 in total K-12 per-pupil revenue (state, local and federal funding). Massachusetts ranked eighth with $17,896 per student. Yet Tennessee, another state often upheld for the academic progress it’s making, is 46th at $9,284.
When a state’s personal income data is thrown into the mix, Michigan lands at 23rd — and Massachusetts falls to 29th. Tennessee is 47th.
Both Massachusetts and Tennessee (along with the majority of other states) outperform Michigan on national tests.
Let's first acknowledge that cherry-picking Massachusetts and Tennessee for a comparison is suspect on its own. Further, it's quite a stretch to say Tennessee is beating the pants off of Michigan:


The way Jacques wrote her piece might lead a reader to think MA and TN are in the same category when it comes to student achievement. But clearly they are not: TN is actually much closer to MI when it comes to national test scores. In fact, while TN edges out MI in Grade 4 scores, the two states are indistinguishable in Grade 8.

Of course, simply comparing scores without controlling for student characteristics is wholly unwarranted. When making a simple adjustment for poverty, MI is about where we'd expect on its test score outcomes (see p.16 here). Could MI do better? Of course, but it's hardly the outlier you would think it was from reading Jacques' op-ed.

In any case, Jacques would have us believe that her highly selective data points contradict a large and growing body of empirical evidence that supports the claim that funding can and does significantly affect student outcomes. Each year, more evidence piles up that schools can and do improve when they have adequate resources. It's become increasingly untenable to deny this -- so Jacques, like all writers in this style, tries to muddy the issue by saying how schools spend is more important than how much they spend:
State Superintendent Brian Whiston also got on board with the adequacy study, calling it a good start, but he included caveats, suggesting longer school years and more professional development for educators.
“We can’t just pour more money into the current way of doing things,” Whiston said in a statement.
Massachusetts didn’t rise to the highest-achieving state overnight. It started two decades ago crafting strong accountability measures and a detailed budget plan — spearheaded by state business leaders. This group, which still exists, created budget proposals for a wide range of districts. While the state did increase funding as part of these reforms, it targeted the money to the districts that needed it most — accounting for poverty and special education.
OK... so what?

If Jacques is saying that money needs to be targeted where it's needed the most, you'll get no argument from me, or anyone else who studies this stuff. But you can't spend money you don't have! Jacques herself notes Massachusetts increased school funding. Granted, it takes less effort for a high-income state to increase its school spending; still, Massachusetts did increase its effort over the years to get more funding for education.

No one thinks we should just throw money at schools. But school funding doubters take this obvious point and twist it to justify inaction on reforming school funding. How much "accountability" does Jacques need before she's willing to stand up and call for Michigan's urban districts to finally get the funds they need to do their jobs?

Jacques' piece ends with what all opinings like this seem to require: a quote from "The Merchant of Doubt," economist Erik Hanushek:
Eric Hanushek, a senior fellow at Stanford University’s Hoover Institution, warned against Michigan’s adequacy study. As an expert in the economics of education, he has shown that it takes much more than increased funding to influence performance.
Yet he highlights Massachusetts as a state that was able to reap rewards from its larger school investment.
“Massachusetts combined strong standards, assessment, and accountability with increased funding,” Hanushek has found. “The basic problem with most school finance systems, both those in existence and those proposed, is that funding is separated from education policy.”
Again, the evidence that funding does matter has piled up so high that not even Hanushek can argue against it. The current tack, instead, seems to be making the case that we need more "accountability" before we open up the purse strings.

But just how much "accountability" do folks like Jacques and Hanushek need before they call for more money for Michigan's disadvantaged schools? How "strong" do the standards have to be? How many more biased, noisy tests do the kids have to take?

Lord knows Michigan's school system could use more oversight, starting with its disastrous charter school sector (which is such a mess even charter cheerleaders say it needs a huge overhaul). But even a rigorous accountability system requires adequate funds. I mean, you have to pay the overseers, right? So where's the call for more funding for that?

For Jacques, and the others who opine about "throwing money at schools," there will never be enough accountability -- because accountability is nothing more than an excuse. Like the obsessions with teacher tenure and charter schools and merit pay and all sorts of reformy reforms, increased "accountability" is just another way to put off some hard truths: Schools need money, disadvantaged schools usually don't have enough, and states are going to have to raise more revenues to get those schools what they need to educate their students.

Ingrid Jacques and her fellow travelers can fret and worry all they want about vague notions of "accountability" -- but that's the easy way to avoid the truth.

Detroit News Editorial Board.

Thursday, August 18, 2016

Inequity In Illinois School Funding: A Follow Up

I want to add something to yesterday's post about school funding inequity in Chicago, and how the blame is being unjustly placed on the city's teachers and their union:

As I said, it's tricky to compare school spending across districts in any context. But Illinois is especially difficult because many of the state's districts are regional high schools: in other words, rather than comprehensive K-12 districts, many schools are organized into K-8 districts and separate high school districts.

For better or for worse, high schools tend to spend more per pupil than elementary schools. Which means comparing the spending at a K-12 district with that of a K-8 or 9-12 district can lead to a false comparison.

We can use regression-based adjustments to account for this disparity, but we really have to approach the differences between districts with some caution. That said, I think it's instructive to look at these adjusted differences in Cook County, IL (as we did yesterday) to see just how screwed Chicago's schools are.


Data is from the US Census Bureau's F33 school finance files, and the Small Area Income and Poverty Estimates (SAIPE). I've marked the districts with greater than 80 percent high school enrollment in dark blue. The spending figures have been adjusted to account for differences in high school enrollments.

A couple of things jump out right away. There is a clear downward slope in the relationship between spending and student poverty; in other words, in Cook County, as poverty goes up, spending is more likely to come down. This is exactly the opposite of what research on school funding suggests we should do.

Also notice that no high school district in Cook County has as high a student poverty level as Chicago. Yes, Chicago is a K-12 district, but keep in mind, as a pointed out yesterday, it is also a district where the overall average family income is significantly higher than the average income for only those students enrolled in the public schools. Which almost certainly means the city's private schools enroll many more-affluent families, concentrating poverty even more within the public schools.

One last point: Chicago may be getting screwed on school funding, but it isn't alone. Plenty of high-poverty schools in Cook County are getting far less funding than low-poverty schools. Again, I don't blame Evanston for spending substantial amounts on its schools. But it's foolish to think Chicago and other high-poverty districts can achieve comparable results with a high-needs student population and fewer resources than more affluent schools.

Blaming Chicago's teachers for this state of affairs is simply stupid. Illinois is not adequately funding its neediest schools. Forcing Chicago's teachers to take a big cut in take-home pay is not going to change the fundamental inequity baked into the Illinois school funding system.

Are the city's or the state's leaders willing to address this?

"What'd he say?"
"I wasn't listening..."

Wednesday, August 17, 2016

Teacher Pay, Student Poverty, and Inequitably Funded Schools: A Data-Driven Story From Chicago

How much more abuse can Chicago's schools take?
Chicago Public Schools students protested Monday the “racist and discriminatory” firing of district teachers and staff, which they said disproportionately affect low-income schools.
At a rally held outside the Thompson Center, about a dozen young protesters called for quality education and funding to be provided in all schools. The district fired 508 teachers and 521 support staff earlier this month. [emphasis mine]
Before we take a data dive, let's acknowledge something important: every number in a staff cut represents an actual person. As Xian Barrett writes in The Progressive, the students who have developed personal connections to their teachers suffer the most when a teacher is laid off. So while I think there's value in the analysis I present below, let's not forget that we are talking about children and educators -- real people who are going through real hell.

The layoffs took place in an atmosphere of continuing friction between the Chicago Teachers Union and district leadership, who can count on the editorial board of the Chicago Tribune, among others, to lay the blame for the district's continuing fiscal problems at the feet of the union:
The district is a candidate for bankruptcy. Chicago taxes already are rising, but CTU wants more. A CPS contract offer on the table since January is a sweet deal for educators; district CEO Forrest Claypool tells us it won't — can't — get sweeter. 
CPS' proposal offers teachers a generous raise and keeps paying them for added seniority and education. It does make a significant ask: Teachers would have to pay a 7 percent pension tab that CPS now pays but no longer can afford. CPS still would pick up the employer's share of pension costs but asks employees to pay their share. Most Chicagoans, most Americans, understand that, since they too have to save for their own retirement.
Note the framing here: the funding of Chicago's schools is an issue of teacher compensation, which is negotiated by the CTU. And the union just doesn't understand how "sweet" of a deal they've been offered (of course, that "sweet" deal only apples to the teachers who haven't been laid off). Sure, the teachers have to take a pay cut to fund their own pensions... but The Trib knows there really isn't any other choice:
"Reality can't be altered," [Chicago schools CEO Forrest] Claypool tells us. "The reality is we do not have more to give than was offered in January. ... There is not a dollar surplus to this budget." Unless, he adds, the union wants to "cut classrooms and jeopardize not only teacher jobs but more important, the academic progress of our kids."
Teachers who strike wouldn't only jeopardize the education of their students, they would set a lousy example for the children: When what you want is impossible, toss a tantrum. [emphasis mine]
See, more money for Chicago's schools is "impossible" -- I mean, everyone knows that, right? Clearly, Chicago's schools have all they could ever need to provide an adequate and equitable education for the city's children! Everybody just needs to sacrifice a bit more -- and by "everyone," The Trib means Chicago's teachers -- and only the teachers -- who have to understand the gravy train just can't keep chugging along...

When you look at the issue of school funding through the lens of teacher pay, it's easy to ignore some inconvenient facts. Here's one: when Bruce Baker* and the good folks at the Education Law Center put together a list of America's most fiscally disadvantaged school districts, they found: "Chicago and Philadelphia are, year after year, the two most fiscally disadvantaged large urban districts in the nation."

This is the story that The Trib, and everyone else who tut-tuts at the CTU, will not tell you: Chicago's schools, which serve proportionally many more at-risk students than their neighboring districts, are chronically underfunded. This reality, more than any perceived greed on behalf of Chicago's teachers, is what drives the fiscal "crisis" the district faces today.

Let's go to the data:


* * *

Who attends Chicago public schools?

In this analysis, I compare Chicago's schools to the rest of Cook County, IL. I could have expanded out to the entire labor market for the area, or to the entire state. But my goal here is to provide some context for the debate over Chicago teacher salaries; Cook County is a good frame because those other districts are right next to Chicago, and the total population of the county is split almost evenly between Chicago and the other towns. What do we find (click to enlarge)?


Chicago, of course, is not the only city in Illinois that suffers from high rates of student poverty. But compared to the rest of the county, its poverty rate is consistently higher. The gap was closing before the Great Recession hit; since then, however, the difference has stayed roughly the same.

There's something else to keep in mind:


This takes a little explaining, but it's important. Suppose we calculate the average family income for all of the children within a town. Then we calculate the same average, but this time only for the children enrolled in public schools. If the averages were the same, we'd conclude that the children who were enrolled in private schools or homeschooled weren't any wealthier than the children in public schools.

Outside of Chicago, that appears to be the case: public school families have, on average, 95 percent of the income of all families, including those who enroll their children in private school. But in Chicago, the average public school family income is only 80 percent of the income of all families. This implies the concentration of lower-income families is even greater within the Chicago Public Schools than poverty estimates for the city would suggest.

What else do we know about the children enrolled in CPS?


This looks like some noisy data, but it does show a clear pattern across several years: CPS enrolls more English Language Learners (ELL) than the other Cook County districts.

Let's look at special education students:


One of the nastier accusations you'll hear reformy types make about big-city school districts is that they over-classify special education students to make their relative test scores look better. It's an absurd idea: classifying a student puts the district on the hook for more services, which cost more money. It's in a district's interest to not over classify students -- and that appears to be exactly what's happening in Chicago, where the special education rate is consistently (if slightly) below that of the rest of the county.

So that's the Chicago student population: many more children at-risk, lots of ELL students, a sizable special education population, and a likely concentration of poverty within the public schools that is greater than that for the entire city.

Now, we know that students who are at-risk and have special needs need more resources in their schools if they are ever going to achieve parity in outcomes with more affluent students. We know that, within the last couple of decades, our country has demanded higher standards and higher achievement for all students, no matter their background.

So how much has Chicago spent on its schools over the years?

* * *

How much do Chicago schools spend?

Comparing spending across school districts and across time is tricky. There are the obvious differences in the cost of living, and those affect wages in different places and at different times. But there are also differences brought on by enrollment sizes and grade levels served. Let's look at spending a few different ways.

We'll start with a straightforward comparison in total per pupil spending, adjusted across time by the Comparable Wage Index developed by Lori Taylor so we get our figures in constant dollars. Data here is from the US Census Bureau.


Across time, CPS has consistently spent less per pupil than its neighboring school districts. But that doesn't take into account the economies of scale CPS may leverage as a large district. It also doesn't account for the fact that many districts in Cook County are either K-8 districts, or high school districts.

Here's a simple regression-adjusted comparison, using a model that accounts for differences in enrollment and grade levels.


Spending is comparable, but I have some misgivings. One is that Chicago's enrollment is so big compared to the rest of the county it's difficult to model how that might affect spending.** And it's not really fair to "penalize" Chicago for having such a large student population; that's a policy choice, arguably out of the city's control.

Here's another model, this time adjusted only by grade levels:


At worst, Chicago's schools spend less per pupil than the rest of the county; at best, they spend about the same when accounting for differences in grade levels and size.

But don't forget: Chicago's schools have many more at-risk and ELL students. Where is any extra fiscal effort on the part of the state -- or anyone else -- to help Chicago's schools educate so many children with special needs?
* * *

How do these differences in spending and students play out in Chicago's schools?

Education is a "human capital-intensive" endeavor; in other words, most school spending is on personnel, because you need teachers to teach. If a school has more money, it's probably going to spend a lot of it on staff.

This can play out in a couple of different ways: a school can reduce student-to-staff ratios, which would lower class sizes and allow for more individualized instruction. Or a school can get better-qualified staff: fewer inexperienced teachers, for example.

Of course, when you're a district like Chicago, you may not be able to get the same quality or quantity of staff for the same price as your more affluent neighbors. Teachers may have to travel farther to get to schools in economically disadvantaged neighborhoods. Working conditions may be more difficult, and the stresses of teaching at-risk students may be greater.

So let's keep in mind the differences in spending and students we've explored as we look at the current conditions of CPS.


Again, the data is noisy, but the picture is clear across time: CPS has consistently had a larger student-to-teacher ratio than its Cook County neighbors -- even as the district enrolls many more at-risk students.

Are the teachers in CPS more experienced?


CPS has many more inexperienced teachers on its staff than its neighboring districts. We know that teachers gain the most in effectiveness in their first few years of teaching. And yet CPS, with its much larger at-risk student population, sends more inexperienced teachers into its classroom than its neighbors.

Again: what's causing this disparity?


CPS spends about $1,500 less per pupil on instructional salaries than the rest of Cook County's districts, and about $2,000 less on total salaries. Again: this is in spite of the fact that CPS has many more at-risk and ELL students proportionally than its neighboring districts.

* * *

From where I'm sitting, it doesn't much look like The Chicago Tribune, or Mayor Emanuel, or Governor Rauner, or a whole host of other folks in Illinois want to discuss what I've put above. They'd rather tell you that it is "impossible" to get more money into Chicago's schools. They'd rather focus on the "unreasonable" demands of the CTU, who have to "rescue" Chicago's schools and stop "think[ing] only of ... yourselves."

What they don't want you to consider, however, is that there seems to be more money for Winnetka's schools, with their much more affluent families. Northfield's schools also get more funding. It doesn't look like it's "impossible" to get New Trier's schools additional money, even when their student population has far fewer at-risk children than CPS.

Understand, I don't blame those communities for spending what they do on their schools. I teach in an affluent school district; I appreciate that the community values its schools enough to fund them properly. But inequities in school funding are real, and they can't be blamed on middle-class teachers, working in big cities, who have taken on the most challenging assignments.

Chicago's schools, enrolling many at-risk students, are chronically underfunded -- and that is not the fault of its teachers. 

Keep this in mind in the days ahead. See who has the courage to tell the good people of Illinois the truth about their schools. Ask those who demand that Chicago's teachers give up even more of their compensation whether they're ready to have an honest conversation about the chronic inequality running rampant in the state's education funding system.

Yeah, that's not really our thing...



ADDING: Bruce Baker has written several posts over the years about the inequities visited upon Chicago's schools: see here, here, and here for starters. And check out this graphic, which pretty much says it all.


* As always: Bruce is my advisor in the PhD program at Rutgers GSE.

** Better to use a national model, like here.

Tuesday, August 16, 2016

Useless Testing "Gap" Analyses and the Newspapers That Love Them

There is so much wrong with this "analysis" by the The Bergen Record of test scores and other outcomes for the 31 "Abbott" districts -- the (mostly) disadvantaged New Jersey school districts that received additional aid for years (although inconsistently) under a series of court rulings. Among other problems:

  • As Julia Sass Rubin points out, the authors don't include the charter schools, which are a large portion of the student enrollments in many districts.
  • They don't account for changes over time in student characteristics.
  • They assume the proficiency rates consistently measure "proficiency" across time, a huge assumption that, to my knowledge, has never been assessed by the state.
  • They assume the Abbotts got "extra aid," even though, for years, the amount the districts received was not what the state's own law says they need to provide an adequate education (and that amount isn't close to enough to expect equalized school outcomes anyway).
I could go on, but I want to make a larger point about so-called "gap" analyses, and why any attempt, like The Record's, to judge an education policy's efficacy based on "gaps" is fundamentally flawed.

And I'm going to keep this so simple it can be drawn out in Sharpies (click to enlarge):


Let's say you have two schools: one in a wealthier area, one is a less-wealthy place. The advantaged school starts off with a higher proficiency rate than the disadvantaged school. As time goes on, the disadvantaged school improves -- but so does the advantaged school

After several years, both schools are performing better than they did previously. But the gap isn't any smaller. Is that a "failure" on the part of the disadvantaged school? Are the state's education policies "failures" because the gap didn't shrink? Would the policies be "successful" if the advantaged school didn't improve so the "gap" shrunk?

Now, The Record's analysis does note the graduation rate "gap" has closed between the Abbotts and the rest of the state. Why would that be?



You can't have a graduation rate over 100 percent, right? So there's a "ceiling" effect: because so many suburban districts were already graduating almost all of their students, they really couldn't improve. But more disadvantaged districts, with lower graduation rates, could and did improve (why they did is an interesting question I'll get to soon). 

So there was a chance for the "gap" to shrink in graduation rates that wasn't available for proficiency rates (few schools, even in the 'burbs, have nearly 100 percent proficiency) or SAT scores. Which makes The Record's piece largely meaningless.

As Ajay Srikanth and I pointed out, there is a great deal of high-quality evidence that shows school funding reform in New Jersey led to substantial improvements in outcomes. Our review matches the evidence found by the Education Law Center. This comports with a large and growing body of high-quality evidence from around the nation that shows equitable and adequate funding can and does make a meaningful difference in student and school outcomes.

The Record has done some fine education journalism over the years. This, unfortunately, was a serious misstep -- as is most "gap" analysis. Reach out to me next time, folks, and we'll see if we can't come up with something a bit better.

Yeah, on second thought, DON'T mind it...


ADDING: Both Bruce Baker and Matt DiCarlo wrote nice pieces a few years ago about how NJDOE totally messed up its "gap" analysis.

Monday, August 8, 2016

The Real Story of New York's Schools Can't Be Told In Test Scores

What is it about New York and test scores?

Test score releases always generate plenty of heat now matter where you go. Politicians and advocates and stakeholders all furiously send out their press releases right after the scores are posted, confident the "latest data proves" their particular agenda is "getting results," even though "we have a lot of work to do," and [INSERT CLICHE HERE].

But New York, in particular, seems to love debating the meaning of their scores. Maybe it's because there is a big education research community in place in and around the city. Maybe it's because parent advocates are particularly well organized, teachers unions are particularly vocal, and wealthy interests are particularly engaged in education issues (golly, I wonder why...). Maybe it's because the debates over mayoral control and charter school expansion are actually battles in larger political wars, especially the interminable power struggles between Albany and NYC.

Whatever the reason, this last score release by NYSED is once again being presented with little to no context, and little to no understanding of how tests work. Granted, I'm the first to admit I'm barely a few feet above sea level on the climb to the right of Mount Stupid:


But I know enough about standardized tests to know that it's pointless to compare proficiency rates from year-to-year on completely different tests. But don't take my world for it; ask the guy who literally wrote the book on testing:
Meanwhile, the researchers we spoke with said any comparisons between this year’s test and last year’s are inherently flawed. The state acknowledged Friday that the tests are not directly comparable because this year’s tests were shorter, for instance, and untimed.
“The state said in their release that the tests are not comparable,” said Dan Koretz, a professor of education at Harvard. “That’s sort of the end of the story. They’re not comparable.” [emphasis mine]
And yet the Kings of Mount Stupid -- aka the Wall Street Journal editorial board -- would have you believe these results cry out for the immediate, rampant expansion of charter schools:
According to new state testing data, citywide student proficiency increased this year on average by 7.6 percentage points in English and 1.2 percentage points in math to 38% and 36.4%, respectively. Some have attributed the city’s gains, which mirror those statewide, to shorter and easier tests.
Yet strikingly, proficiency at charter schools this year jumped 13.7 percentage points in English and 4.5 percentage points in math to 43% and 47%, respectively. In other words, charter students have improved by two to four times as much as the citywide average.
Yes, one year of noisy, biased, crappy fill-in-the-bubble test scores is all the Journal needs to turn over NYC's schools to Eva Moskowitz and her ilk. And there's no need to account for student population differences, or spending differences, or attrition, or suspension rates, or unobserved differences, or a whole host of other factors. Just shut down Tweed once and for all -- the tests prove we have to!

There's been talk that the state monkeyed around with test scoring this year to jack up the proficiency rates. Lord knows there have been been plenty of shenanigans in the history of New York's test score reporting. I'm not convinced as of yet, however, that the state has done anything more than attempt some basic year-to-year equating...

But that's actually pretty bad by itself. Why would you even try to equate two tests that are inherently different in form? Why not just admit you're setting a new base, and next year's noisy, biased, crappy results will show us...  well, honestly, next to nothing -- it's only one year.

Look, as I've said many times before: I think it's a good idea to have some sort of testing regime for system accountability purposes. We need to set some standards and see if we're meeting them if we're ever going to make a case for the reforms we need (equitable and adequate funding, elimination of child poverty, desegregation, and so on -- you know, stuff that actually works). School funding litigation has relied on test results to argue cases for funding reform, and that's a good thing.

Of course, we could get all the data we need for these purposes with a lot less cost and a lot less intrusion. But we'd still have to provide the appropriate context; only then can we tell the real story of our schools and our policies.

In that spirit, let me offer a few graphs that look over time at some of the things going on in New York State. This isn't even close to comprehensive, but it's a start. All of it is based on a new dataset we're going to be unveiling soon (so, yes, this post is basically a chance for me to give it a test run - sue me).

1) New York City is not the state's leader in charter school proliferation.


The data I have here is a bit lagged: 2014 means the spring of the 2013-14 school year. Still, it's fairly clear that NYC, even with its high-profile expansion of charter schools, is well behind cities like Albany, Buffalo, and Rochester when it comes to charter sector share. I have some reason to believe these upstate cities actually have greater charter shares*, but even on its face, NYC hasn't begun to match many Upstate cities in charter proliferation.

Can the NYC charter sector confidently claim at this point that it will keep its aggregate advantage over the city's public, district schools when 20 to 30 percent of the population is in charters? And what will be the consequences for the district? We know Buffalo's and Albany's district schools have paid a steep fiscal price for their high level of charter proliferation. Can we be certain the same won't happen in NYC?

Certain reformy types who cast themselves as "sector agnostics" will tell you that NYC is where chartering, like a bucket of chicken, has been "done right." I'd argue NYC hasn't even done chartering yet -- not like what's to come. Before the Moskowitz et al. are allowed to do whatever they want, maybe someone should explain how Gotham is going to avoid the problems charters have caused Upstate.

2) Upstate NY cities have seen a huge shift in the demographic characteristics of their students.


I wrote about this a few years ago, but it's still amazing when I look at it. The three big Upstate NY cities have seen a massive amount of white flight from their schools over the past two decades. Syracuse is especially shocking, but Buffalo and Rochester aren't far behind. NYC, by contrast, has had a fairly stable percentage of white students.

As this story from the Hechinger Report notes, the death of manufacturing Upstate has had severe consequences for children:
While gentrifying large cities like New York have grown increasingly prosperous in recent years, smaller urban centers like Syracuse have seen large increases in poverty. 
“Syracuse is now number one in the nation in terms of the percentage of black and Hispanic families living in high-poverty neighborhoods,” said Sharon Contreras, outgoing superintendent of Syracuse City School District. “I know kids can overcome poverty, but when students are living in abject poverty in segregated communities with poor health care and housing, that makes the work for our teachers all the more challenging.”
Which leads us to...

3) Upstate NY cities have seen a dramatic rise in childhood poverty relative to NYC.


Historically, student-aged poverty in NYC has been lower than that found Upstate, although the levels were close in the early 2000s. After the Great Recession, however, there was a significant split: age 5 to 17 poverty rose at a much slower rate in NYC than it did in Rochester, Syracuse, and Buffalo. And I'm not even counting the many smaller cities Upstate, like Utica and Albany, that have also been hit hard.

Before anyone starts making comparisons between NYC and the rest of the state, they ought to at least acknowledge these realities. I'm not saying the 5 boroughs don't have their problems with childhood poverty; I am saying Upstate has been slammed especially hard, and that's a real problem with any simple test score comparison.

One more thing:

4) The per pupil spending gap between NYC and Upstate districts has been changing.




I'm reluctant to post this because I have to attach so many caveats to it. This is purely descriptive data, adjusted for geographic and yearly differences in education costs. It's noisy and I'm only including the Big 5 districts; however, the big inequities in New York State education funding have as much to do with smaller cities as anything else. New York State has all kinds of "stealth inequities" built into its state funding system, and there's reason to believe that's led to some districts engaging in "inefficient" spending patterns.

In other words: the topic of New York State spending is far more complex than I can get into here. But I think it's important to point out that even as poverty has been on the rise in all Upstate cities, education spending, relative to NYC, has not. Syracuse in particular has taken a big hit over the last few years, even as its poverty rates have skyrocketed. Again, this data is noisy and the drop off over the past two years may be an anomaly; however, it does follow several years of declines. 

The point here is that if we buy into the assumption that children in poverty require more resources in their schools -- an assumption upheld by a boatload of evidence --  the large uptick in poverty Upstate should have been followed by a large uptick in resources. But we really haven't seen that since 2009, when Upstate childhood poverty began its sharp climb.


There's so much more I could add to this. But my goal isn't to try to tease out why test scores are coming out the way they are in New York; it's to show that simply pointing to differences in proficiency rates from one year to the next is bad enough, but it's made worse when you don't provide any context.

New York is a big, populous state with enormous differences in wealth. Many of its regions have undergone profound transformations over the last two decades. So simple claims of NYC "catching up" with the rest of the state are mostly meaningless, as are claims about charter schools "beating the odds." There's a much more complex tale to be told, and proficiency rate changes over a year just aren't adequate to the job.

Mercifully quiet this time around.

ADDING: Leonie Haimson gives us a NY test score history lesson:
Then there's the lack of any historical context. The paper of record has a lamentable record of failing to report on the well-documented evidence of inflated test score gains that occurred from 2003-2009, until the state itself admitted what had happened and re-calibrated the cut scores in 2010.  Their unshakeable credulity led to a front page story on August 3, 2009  -- a little more than seven years ago to the day, recounting the big jump in student achievement and giving credit to the Bloomberg reforms.  Like now, they refused to explain the multiple sources of evidence to the contrary, including the fact that the NAEPs showed only modest gains over the same time period.  My argument with the Times editors even made the Village Voice .   As Wayne Barrett wrote,

The Times front page piece last week -- headlined "Gains on Tests in New York Schools Don't Silence Critics" -- failed to quote any real critics, but gave Klein six self-promoting paragraphs. It did bury a single questioning quote from two academics not known as critics of the test scores in the thirty-fourth paragraph, but the top of the story trumpeted success scores that would have silenced any critic. If, that is, they were true. 

Two days after the article ran, the NY Senate voted to renew Mayoral control.  A few months later, Bloomberg was re-elected to a third term.   Sure enough, when the test bubble burst in 2010  all the gains were shown to be illusory. Even after that, though, in 2011 a writer for the NY Times Magazine reported that "since 2006, the city's elementary and middle schools have seen a 22-point increase in the percentage of students at or above grade level in math (to 54 percent) and a 6-pont increase in English (to 42 percent)." 

These statistics were completely fabricated of course, provided by DOE to the reporter, and somehow neither the reporter nor any editors had bothered to  check them.  It turned out the DOE had made up the data by re-adjusting the cut scores to where the state had previously put them, essentially rewriting history as though test score inflation and deflation had never occurred. 

All this is to say: If reporters at the NY Times and other media outlets are prepared to point out the unjustified claims promoted by public officials and some advocacy groups last week, that is good; but they might try also provide some context to explain the larger reasons for skepticism.
Remember:



* Matching charters to their sending districts is complex work, especially in urban areas that don't have highly concentrated core cities. There are, for example, charter schools in Lakawana just south of Buffalo. My method matches them to that suburban town, even though it's very likely much of their population is coming from the city itself.

Caveat regressor.