I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Saturday, April 29, 2017

Desperately Searching For the Merit Pay Fairy

It's been a while since we've talked about the Merit Pay Fairy.

Yo, it's me -- da Merit Pay Fairy, makin' all your reformy dreams come true!

The Merit Pay Fairy lives in the dreams and desires of a great many reform-types, who desperately want to believe that "performance incentives" for teachers will somehow magically improve efforts and, consequently, results in America's classrooms. Because, as we all know, too many teachers are just phoning it in -- which explains why a system of schooling that ranks and orders students continually fails to make all kids perform above average...

One of the arguments you'll hear from believers in the Merit Pay Fairy is that teaching needs to be made more like other jobs in the "real world." But pay tied directly to performance measures is actually quite rare in the private sector (p. 6). It's even more rare in professions where you are judged by the performance of others -- in this case, students, whose test scores vary widely based on factors having nothing to do with their teachers

But that doesn't matter if you believe in the Merit Pay Fairy; all that counts is that some quick, cheap fix be brought in to show that we're doing all we can to improve public education without actually spending more money. And, yes, merit pay as conceived by many (if not most) in the "reform" world, is cheap -- because it involves not raising the overall compensation of the teaching corps, but taking money away from some teachers and giving it to others, using a noisy evaluation system incapable of making fine distinctions in teacher effectiveness.

Which brings us to the latest merit pay study, which has been getting a lot of press:
Student test scores have a modest but statistically significant improvement when an incentive pay plan is in place for their teachers, say researchers who analyzed findings from 44 primary studies between 1997 and 2016.
“Approximately 74 percent of the effect sizes recorded in our review were positive. The influence was relatively similar across the two subject areas, mathematics and English language arts,” said Matthew Springer, assistant professor of public policy and education at Vanderbilt’s Peabody College of Education and Human Development.
The academic increase is roughly equivalent to adding three weeks of learning to the school year, based on studies conducted in U.S. schools, and four weeks based on studies across the globe.
Let's start with the last paragraph first: the notion that you can translate this study's effects into "weeks of learning" is completely without... well, merit. Like so much other research in this field, the authors make the translation based on a paper by Hill et al. (2008). I'll save getting into the weeds for later (and in a more formal setting than this blog), but for now:

Hill et al. make their translation of effect sizes into a time periods based on what are called vertically-scaled tests. These are tests that let at least some students attempt to answer at least some common items between concurrent grade levels, allowing for a limited comparison between grades (see p.17 here).

There is no indication, however, that any of the tests used in any of the 44 studies are vertically scaled -- which makes a conversion into "x weeks of learning" an unvalidated use of test scores. In other words: the authors in no way show that their study can use the methods of Hill et al., because the tests are likely scaled differently.

Furthermore: do we have any idea if the tests used in international contexts are at all educationally equivalent to the tests here in the US? For that matter, what are the contexts for the teaching profession, and how it might be affected by merit pay, in other countries? So far as I'm concerned, the effect size we care about is the one found in studies conducted in this country.

That US effect size is reported in Table 3 (p. 44) as 0.035 standard deviations. How can we interpret this? Plugging into a standard deviation-to-percentiles calculator (here's one), we find this effect moves students at the 50th percentile to 51.4.* It's a very tough haul to argue that this is an educationally meaningful effect.

Which brings us to the next limitation of this meta-analysis: the treatment is not well defined. To their credit, the authors attempt to divide up the different studies by their characteristics, but they only do so in the international aggregate. In other words: they report the differences between a merit pay plan that uses group incentives versus a "rank order tournament" (p. 45, Table 4), but they don't divide these studies up between the US and the rest of the world.

Interestingly, group incentives have a greater effect than individual competitions. But there is obviously huge variation within this category in how a merit pay plan will be implemented. For example: where did the funds for merit pay come from? 

In Newark, merit pay was implemented using funds dedicated by Mark Zuckerberg. Teachers were promised that up to $20 million would be available; of course, it turned out to be far less (and it's worth noting that there's scant little evidence Newark's outcomes have improved). Would this program have different effects if the money had not come from an outside source?** What if the money came, instead, from other teachers' salaries (which may, in fact, be the case in Newark)?

Any large-scale merit pay plan will be subject to all sorts of variations that may (or may not) impact how teachers do their jobs. Look at the descriptions in Table 6 (p. 47), which recounts how various merit pay plans affect teacher recruitment and retention, to see just how diverse these schemes are.

I think it's safe to say that "merit pay" in the current conversation is not really about giving bonuses for working in hard-to-staff assignments, or for taking on extra responsibilities, or even for working in a group that meets a particular goal. I'm not suggesting we shouldn't be looking at the effects of programs like this, but I don't think it's helpful to put them into the same category as "merit pay."

I think, instead, that "merit pay" is commonly understood as being a system of compensation that differs from how we currently pay teachers: one where pay raises are based on individual performance instead of experience or credentials. The Chalkbeat article certainly implies this by making this comparison:
Teacher pay is significant because salaries account for nearly 60 percent of school expenses nationwide, and research is clear that teachers matter more to student achievement than any other aspect of schooling (although out-of-school factors matter more). About 95 percent of public school districts set teacher pay based on years of experience and highest degree earned, but merit pay advocates argue that the approach needs to change. [emphasis mine]
Take a look at a sample of articles on teacher merit pay -- here, here, here, here, and here for example -- and you'll see merit pay contrasted with step guides that increase pay for more years of experience or higher degrees. You'll also notice none of the proponents of merit pay are suggesting that the overall amount spent on our teaching corps should increase.

I can understand the point of writers like Matt Barnum who argue that merit pay can come in all sorts of flavors. But I contend we're not talking about things like hard-to-staff bonuses or group incentives: When America debates merit pay, it's really discussing whether we should take pay from some teachers and give it to others.

Unfortunately, by analyzing all of these different types of studies together, the Vanderbilt meta-analysis isn't answering the central question: should we ditch step guides and move to a performance based system? That said, the study may still be giving us a clue: the payoff will likely be, at best, a meager increase in test scores.

Of course, we have to weigh that against the cost -- or, more precisely, the risk. Radically changing how teachers are paid would create huge upheavals throughout the profession. Would teachers who were in their current assignments stay on their guides, or would they potentially take huge hits in pay? If they were grandfathered out of a merit pay scheme, how would they work with new teachers who were being compensated differently?

Would merit pay be doled out on the basis of test scores? How much would VAMs or SGPs be weighted? How would teachers of non-tested subjects be eligible? Would the recipients of merit pay be publicly announced? In New Jersey and many other states, teacher salaries are public information. Would that continue? And how, then, would students be assigned to the teachers who receive merit pay? Will parents get to appeal if their child is assigned to a "merit-less" teacher?

The chaos that would result from implementing an actual merit pay plan is a very high cost for a potential 0.035 standard deviation improvement in test scores.

I know believers in the Merit Pay Fairy would like to think otherwise, but clapping harder just isn't going to make these very real issues go away.

Don't listen to dat Jazzman guy! Just clap harder, ya bums!

ADDING: More from Peter Greene:
Researchers' fondness for describing learning in units of years, weeks, or days is great example of how far removed this stuff is from the actual experience of actual live humans in actual classrooms, where learning is not a featureless tofu-like slab from which we slice an equal, qualitatively-identical serving every day. In short, measuring "learning" in days, weeks, or months is absurd. As absurd as applying the same measure to researchers and claiming, for instance, that I can see that Springer's paper represents three more weeks of research than less-accomplished research papers.

* Some folks don't much care for making this kind of conversion. In my view, it's much more defensible than converting to "x weeks of learning," which, even setting aside the problems of converting from vertically scaled tests, suffers from unjustified precision. In addition, the implications behind the translation are subject to wild misinterpretation.

Converting to percentiles might a bit problematic. But it's not nearly as bad as using "x weeks of learning."

** We'll never know because no one has bothered to find out if the Newark merit pay program actually worked. Think about it: $100 million in Facebook money, and no one ever considered that maybe reserving a few thousand for a program evaluation was a good idea.

If I was cynical, I might even think folks didn't want to study the results, because they were afraid of what they might find. Good thing I'm not cynical...

Monday, April 10, 2017

Teacher Tenure and Seniority Lawsuits: A Failure of Logic

New Jersey's teacher tenure and seniority lawsuit continues to grind away. Part of a trio of suits here and in New York and Minnesota, these lawsuits are all being brought to the various state courts by the Partnership for Educational Justice, Campbell Brown's secretly funded organization.

Their Minnesota lawsuit was thrown out of court last fall; in New Jersey, however, we had to wait for a state Supreme Court ruling on a Christie administration motion to tie tenure and seniority laws to school funding. The Court ruled it wasn't going to opine on these laws until a lower court takes up the PEJ's case. So now we wait for that ruling -- and the PEJ continues its public relations campaign against tenure and last in-first out (LIFO) seniority rules.

To their credit, PEJ has posted all of the filings in the case. But it's clear after reviewing them that PEJ doesn't have a leg to stand on. Not to say they won't prevail: bad legal reasoning didn't stop Judge Rolf Treu in California from issuing a terrible ruling in Vergara, which was inevitably overturned on appeal. Similarly, the only way PEJ can win here in New Jersey is if the lower court hearing the case sets aside all logic and reason...

Because the PEJ's case simply makes no sense.

When a group like the PEJ goes before the courts to get a statute overturned as unconstitutional -- by which I mean in violation of the state's constitution, not the federal Constitution -- the burden of proof is on them. They may have a problem with the NJ tenure and LIFO statutes, or any other law on the books, but getting the court to overturn a law isn't simply a case of arguing against the law's merits: they have to show how it violates the state's constitution.

The constitution states (Article VIII, Section IV): "The Legislature shall provide for the maintenance and support of a thorough and efficient system of free public schools for the instruction of all the children in the State between the ages of five and eighteen years." Unless and until the PEJ can demonstrate to the courts that tenure and LIFO laws violate this clause, the courts cannot act.

In a long-running series of cases involving school funding in New Jersey, the NJ Supreme Court found the systemically inadequate and inequitable funding of schools was in direct violation of the education clause. Although the litigation has a long and complex history, the basic premise of the lawsuits is comparatively simple: at-risk children need more funding to equalize educational opportunities, the state's system of school funding makes it impossible for those children's communities to raise adequate funds on their own and, therefore, the state needs to intervene.

In contrast, the challenge for the PEJ is to show how tenure and LIFO laws similarly violate the education clause; even the PEJ's own filing concedes this point. The problem is that right after stating what their legal argument should be, they completely ignore the task. 

Yes, they make the case districts with larger proportions of at-risk children show fewer gains in academic outcomes; no one disputes this. Yes, they make the case teachers matter; no one disputes this (although the canard of teachers being "the most important in-school factor" for student achievement is wrong: the student is the most important "in-school factor," not the teacher). Yes, they make the case ineffective teachers should be dismissed; no one disputes this.

They even go further and argue that the quality of teachers suffers in districts that serve many at-risk students. Certainly, there's strong evidence students in these districts are more likely to have less qualified teachers, as judged by their credentials, experience, or scores on knowledge tests (teacher scores, not students). 

But none of this speaks to the central argument PEJ is trying to make:
Hill: The Newark Teachers Union says — about the comment like that — these folks who are tenured, they’ve been through a certain process and if the process determines that they’re no longer an effective teacher, the process has a way of dealing with them. You say? 
[Ralia] Polechronis [PEJ Executive Director]: So, that’s not entirely what we are talking about here. What we’re talking about in LIFO are terminations and layoffs that have to happen only during budget cuts. So that process, that dismissal process, isn’t really at play. We’re talking about a situation when the district is in such dire financial constraints and is having such a problem figuring out its budget that they have to go to teachers, they have to go to laying them off and they have to make that decision, according to the law, by the level of seniority instead of thinking about the great teachers that are in the classroom and that should stay there.
Think about what Polechronis is assuming: that a district like Newark has the ability to differentiate at a very fine level the effectiveness of individual teachers, and then act accordingly in high-stakes decisions.

Let's be very clear: There is no evidence -- none -- that teacher effectiveness can be measured reliably and validly at a level that allows for high-stakes decisions to be made regarding teachers who have already been found to meet a minimal level of effectiveness.

What PEJ argues implicitly is that the Newark Public Schools can simply use its observation rubrics and Student Growth Percentiles and Student Growth Objectives to calculate an overall measure of teacher effectiveness, and then apply that measure to fairly determine who gets the boot when budgets cuts "must" be made. But this contradicts everything we know about measuring teacher effectiveness.

Yes, principals can identify their very worst teachers; they are incapable, however, of differentiating the effectiveness of the vast bulk of teachers in the middle. The phony precision of observation protocols like the Danielson Model have led some to think we can validly use the resulting scores to accurately rank and order teachers; that is a mistaken belief grounded in innumeracy. In the same way, the error that is an inherent part of standardized tests makes the use of SGPs in decisions like this invalid (among many other reasons). And SGOs are, to be blunt, a joke.

The plain truth is that even if PEJ got its way and teachers could be dismissed without regard to seniority, there is no reliable and valid way to evaluate the majority of teachers who are dismissed in reductions-in-force. Yes, we can identify the worst performers; we can and should either get them remediation or remove them from their classrooms. But there's simply to reason to believe Newark, or any district, can accurately rank all teachers by their effectiveness.

But that's not the only failure of logic in PEJ's case. Because even if districts could make accurate decisions based on effectiveness -- again, they can't, but play along -- they would still have to show that districts like Newark were disproportionately affected by LIFO laws.

Unlike school funding -- which, despite all of the lawsuits, is still inequitably distributed across the state -- tenure and LIFO laws apply to every district equally. Newark and more affluent Millburn both have to operate under tenure laws; Camden and more affluent Haddonfield both have LIFO. Yes, the cities have had to make cuts in staff, in large part because charter schools, imposed by the state, have gobbled up more students and more resources. But that's not a function of tenure or LIFO laws; how could it be?

Reading through the PEJ's filings, it's clear they are unable to make a case that urban students have suffered disproportionately by tenure; in fact, as NJEA points out in one of its briefs, there isn't even evidence that any of the plaintiffs' children suffered from having a bad teacher who was spared dismissal by the LIFO laws, calling into question the plaintiffs' standing.

What is clear is that Newark's schools have suffered from inadequate and inequitable funding; even the plaintiffs acknowledge students have suffered from losses of staff like librarians and guidance counselors (p.9-10). But they put forward no argument that removing LIFO laws would have saved those jobs; again, how could they?

Some have argued that dismissing senior, higher-paid employees frees up more funds for lower-paid, less senior staff, thus leading to fewer reductions. This assumes that teacher effectiveness is evenly distributed across experience, which we know is not true -- when you cut experienced teachers, you're more likely to cut effective teachers (and again: we're setting aside the problem that you can't rank and order the vast majority of teachers by effectiveness anyway).

It also assumes that there is so much inefficiency within urban schools that they can cut staff and retain programming and class size. Empirically, however, we know that NJ's urban schools are not systemically its most inefficient ones. We also know that funding adequacy correlates with staff per student in various educational programs, which means the problems of cutting staff and programming have much more to do with inadequate funding than they do with tenure and LIFO -- policies, again, which are enforced in all districts.

Finally, it's important to remember that teachers value tenure and LIFO. If the state gets rid of it, that decreases the overall compensation, momentary and otherwise, of teachers. Are the taxpayers of New Jersey willing to fork over more money to make up for this loss in incentives? Or do they want to see a less qualified pool of prospective teachers enter the profession?

The backers of these lawsuits will make occasional concessions to the idea that schools need adequate and equitable funding to attract qualified people into teaching. But they never seem to be interested in underwriting lawsuits that would get districts like Newark the funds they need to improve both the compensation and the working conditions of teachers.

Instead, they waste their time with lawsuits like this -- suits that fail on legal, empirical, and logical grounds. Suits that do nothing to help deliver the resources all students need to equalize educational opportunities. Suits that do nothing to improve the effectiveness of New Jersey's teaching corps, or the efficiency of its school system. Suits that only serve to further dishearten the people who go to work in public schools every day on behalf of the taxpayers and students of this state.

Maybe one day Campbell Brown and the PEJ will stop trying to take away the hard-fought rights of teachers, and take up the real fight for our state's deserving children.

ADDING: As if on cue:
ATLANTIC CITY — The school district advertised three times for a certified chemistry teacher last summer and fall, and three times they failed to get a candidate to accept the job.
So they turned to Edmentum, a provider of online courses, to fill the gap. This year, four classes at the high school are being taught via the online course, with backup support from a teacher.
The statewide shortage makes the position competitive. At least three area school districts are looking for chemistry teachers next year.  
Ralph Aiello, principal at Cumberland Regional High School, said he’s looking for a combined chemistry/physics teacher for next year. So far, he has had just two applications. 
Linda Smith, president of the New Jersey Science Teachers Association, said she is working with colleges to develop programs that recruit former or retired scientists into teaching as a second career.
“People can just make more money as scientists than they can as science teachers,” she said. “Some do want to teach. But they need training and mentoring. People who are good at science are not always good at explaining it.” [emphasis mine]
Terry Moe, hardly a friend of teachers unions, states: "...most teachers see the security of tenure as being worth tens of thousands of dollars a year.” So please, PEJ: Explain to us how eliminating tenure and LIFO will help recruit better candidates into a profession that is already suffering from serious shortages.

(This should be good...)

Wednesday, March 15, 2017

NJ Charter Schools - A Fool's Gold Rush: Vineland

Here are all the parts of this series so far:


Franklin, NJ

Princeton, NJ

- Vineland, NJ

Here in New Jersey, the state's school funding formula has become a topic of intense debate. Now that Chris Christie has abandoned his "Fairness Formula" -- a ridiculously unfair and inefficient proposal -- attention has turned to Senate President Steve Sweeney's school funding proposal:
HAMMONTON — Senate President Stephen Sweeney said Friday he wants revisions to the school-funding formula in place for the 2017-18 school year.
“Every year, the gap gets greater,” he said during a visit to the Hammonton School District to discuss school funding. “Every year there is flat funding we are losing ground.”
Sweeney said he recognizes some districts would lose funding under a revised plan, but those districts have known for years they are getting more state aid than they should, and they should have been making plans. [emphasis mine]
Sweeney's plan is to create a commission to recommend changes to the state's funding formula, SFRA. I don't necessarily have a problem with asking some districts with relatively high property values to pay more in local taxes, but the devil's in the details, and those details are scarce. I'll save a larger discussion of school funding for another day; right now, I want to focus on this:
Helen Haley, the business administrator for the Vineland School District, said they would lose money under the proposed revisions, and it would be devastating for the district, which is among the poorest in the state.
“We have cut 57 positions through attrition,” she said. “We are a large rural area, so transportation costs are high. Our special-education population is growing. But we have trouble raising taxes because of economic conditions.”
Sweeney said special education is an area that must be addressed, and the district could get additional aid there to make up for losses in other aid.
According to data from the Education Law Center, Vineland's local fair share of taxes is relatively low. Whether the town should pay more is an open question, particularly because, as Jeff Bennett notes, many districts paying low school taxes actually have high municipal taxes. Is Vineland in that group? I can't say at this time...

But I do know this: Vineland, like all other New Jersey school districts, should be seeking to make its schools systems as fiscally efficient as possible. If funds are scarce, and if taxpayers want the maximal value for their investment, school systems should keep administrative costs low and get as much money "into the classroom" as possible.

Which brings us to the Great NJ Charter School Rush of 2017, and the NJDOE's push to approve as many charter school expansions as possible before Chris Christie's term ends -- including the Vineland Charter School.

VCS has a colorful history. Its founder, Ann Garcia, was accused back in 2011 of interfering in an investigation into the fact that she held five concurrent positions in taxpayer-funded schools:
The report states that Garcia held positions in five school districts in 2010-11, earning almost $300,000. That is not illegal, but an attempt to alter one of the contracts during the investigation could lead to future sanctions.
Garcia worked full time as the school business administrator in Winslow Township, earning  $156,375 in 2009-10 and 2010-11, according to the state report, obtained by The Press of Atlantic City through the Open Public Records Act.
 In those two years she was also paid to be the executive director of the Vineland Public Charter School, earning $50,000 in 2009-10 and $60,000 in 2010-11. She also served as the part-time business administrator for two other charter schools, the Charter Tech High School for the Performing Arts in Somers Point, where she earned $54,080 in 2009-10 and $56,243 in 2011-12, and the Environment Community Opportunity Charter School in Camden, where she earned $22,000 in 2009-2010 and $18,640 in 2010-11. She also served as the appointed treasurer for the Voorhees school district, earning $6,780 in 2009-10 and $6,900 in 2010-11.
For all five jobs Garcia earned $298,158 in 2010-11, according to the state report. The Millville charter school opened last month and was not included in the report. [emphasis mine]
Nice work if you can get it. Garcia was cleared of the charges by the Education Commissioner's office, although they noted that holding five positions at a time was highly questionable:
Colleen Schulz-Eskow, of the Department’s Charter School Office, testified that it was common for business administrators – but not lead personnel – to work at multiple charter schools. 6T52-53. Since she was aware that – at the time VPCS was starting up – Murphy-Garcia was doing business administrator work at other schools or districts, Schulz-Eskow questioned Murphy-Garcia about whether her time and commitment to VPCS would be sufficient to adequately oversee the school. 6T44-47. Ultimately, when the DOE approved VPCS, they were not worried about Murphy-Garcia being at the school on a part time basis. 6T55. 
It is not clear to the Commissioner, however, whether Schulz-Eskow was aware of how many schools employed Murphy-Garcia. If she was, it appears that she did not share the concerns of her Department of Education colleague, Martin, about the implications of Murphy-Garcia’s multiple employment. However, as a matter of common sense, the Commissioner must register misgivings about Murphy-Garcia’s – or anyone else’s – ability to actively devote to each of five employers the respective number of hours of work which were promised them. It is likely that such a concern motivated OFAC’s continuation of its investigation into Murphy-Garcia’s employment. 
In sum, after a full review of this case, the Commissioner rejects the specific conclusions about Murphy-Garcia on page 11 of OFAC’s report. However, the Commissioner cautions petitioner VPCS to improve its business practices and administrative oversight and to ensure that its contracts are properly memorialized and that all of its records are secure and accessible. The Commissioner also reiterates his concern that petitioner Murphy-Garcia may have contracted with more employers than she can properly serve. [emphasis mine]
So here's the thing: even as this investigation was going on, the NJDOE was approving expansion plans for Vineland Charter School. In other words, despite the Commissioner's misgivings, VCS was given the green light to grow -- even if the local community had its own problems with expansion:
4/16/15 -- VINELAND – Vineland Public Charter School families are waking up today to the possibility the school may not have a place to call home come September. 
After hearing close to three hours of passionate testimony, the Zoning Board early this morning issued a resounding rejection to a request to allow the charter school to lease space at the former Wallace & Tiernan factory at 1901 W. Garden Road. 
Boiled down, members agreed with municipal staff findings and argument from opponents that the board's priority should be to uphold a quarter-century of city development strategy and investments.
According to this newspaper account, Garcia did not like how the meeting went:
Charter school Executive Director Ann Garcia, who testified extensively at the prior meetings, did not at the final hearing. However, Garcia made her presence known with loud comments and fidgeting from the front row of the audience area during a Magazzu-Stanker exchange. Garcia has acted similarly at previous meetings. 
The display irritated board chairman and Solicitor Frank DiDomenico, who chastised her and suggested she should leave if she could not keep quiet. 
Garcia responded to LoBiondo that she might go and she did leave a few minutes later. She returned as the board started its voting process. 
Greenberg noted that the city's principal planner had found that a school in an industrial zone was a significant problem. Another reason she gave was testimony from the charter school that the factory would provide room for an expansion to accommodate a high school.
The commissioner of the N.J. Department of Education in March denied that expansion request. He laid out why in a letter sent to Garcia. The executive director was asked at the last meeting, under oath, to explain the letter and the commissioner's reasoning. 
Garcia answered then that the denial was based on the charter school, at present, lacking adequate space. However, the education department officially denies her claim and says the reason for the denial was an organizational weakness. [emphasis mine]
So the NJDOE approved VCS's expansion in 2013, even though they were conducting an investigation. In 2015, they found, according to this report, "organizational weakness" in the school. And yet expansion plans went ahead, including the building of a new facility last year.

That facility, and five others in New Jersey, has been financed by Highmark School Development, a Utah-based company that, according to this report, loans funds to aligned non-profit corporations, who in turn build schools which they then lease to a charter.

As Bruce Baker and Gary Miron have pointed out, this practice essentially uses taxpayer funds to put school facilities into private hands. If this facility was being built by the Vineland Public Schools, the taxpayers would own it. That isn't the case here, even though the taxpayers, through payments to VCS, are footing the bill.

Just like they're paying the costs of transporting VCS students:
VINELAND - Transportation issues with the Vineland Public Charter School could cost the district about $400,000 not included in the 2016-17 budget, the district’s transportation director told the school board Wednesday. 
It’s a problem the school district brought upon itself, said Ann Garcia, VPCS executive director. 
“The charter school is a very fragmented business right now; they are supposed to be in one location within the City of Vineland and they are in four locations, one being in Millville, which created a transportation problem for us,” Joe Callavini, the district’s transportation/registration coordinator, told the board Wednesday. 
The VPCS is building a new school along Pennsylvania Avenue to serve pre-school through ninth grade. During construction, the school is using a network of sites on Landis Avenue, Park Avenue, and Main Road as well as Wheaton Avenue in Millville. [emphasis mine]
There's a lot of he-said/she-said here, but the central point is that VCS is creating many extra costs for the taxpayers of Vineland, and New Jersey as a whole: the redundant administration, the building of extra facilities the public will not own, the addition transportation costs, and so on.

If Senator Sweeney and any other stakeholders are interested in saving money on schools, they should be asking if the "gains" Vineland Charter School gets are worth the costs of its expansion.

So let's go to the data and find out what the costs and benefits of VCS really are:

VCS and Vineland Public Schools had close to the same proportion of free lunch-eligible students in 2011. The gap, however, has grown in recent years.

Like so many other charters in NJ, VCS does not educate students whose first language is something other than English.*

And, like so many other charters, VCS educates a much smaller proportion of special needs students, year after year.

The few special education students VCS does take have lower-cost disabilities. This concentration of higher-cost students puts a strain on VPS. I'm glad to see Senator Sweeney acknowledge special education funding is important, but does he understand the effects of this type of concentration, thanks to charter schools?

Let's look at the staff:

Over two-thirds of the VPS certificated staff has less than three years of experience in the classroom. And that staff is still paid less than the teachers in VPS:

Once again, we have the "free rider" problem for VPS: because charter teachers know they can make more if they transfer to a public district school later in their careers, they are likely more willing to take less money at a charter where they start their careers. VCS, in other words, gets away with paying their teachers less because those teachers know they can transfer out in a few years and make more money.

How does VCS spend its funds?

What's interesting here is that VCS spends much more on administration than VPS -- as is typical for charter schools -- but the gap closes considerably when looking only at salaries. Are VCS payments to Highmark going to come in this category? 

Yes, the budgetary costs are less at VCS, but, again, that is largely driven by higher costs at VPS for teacher salaries. VPS also spends more on support services, undoubtedly because they have more students who are classified and need those services. For example:

If we take their preschool out, we find VPS has many more support staff per student than VCS. Again, that's a function of VPS having many more classified students on their rolls.

So there's the cost for Vineland to maintain a charter school: higher administrative costs, lower teacher salaries, fewer experienced teachers, and student segregation by educational need.

So what's the benefit? Again, as I have throughout the series, I use a linear regression model to account for differences in student characteristics when judging test scores. Let's start with English Language Arts:

The "value" that VCS adds is on the order of 10 to 15 points in Grades 3 and 4 -- on a test that has a range of 200 points. Is that impressive? Considering the mixed bag in VPS schools, and the low value added in Grade 5, I'd say it's a decent showing, but hardly a superior outlier.

Here are the upper grades:

There's no denying this is a better showing, although, again, we're talking less than 20 points on a  test with a 200-point scale. The situation gets muddy, however, when we look at math:

VCS adds no value in Grade 3 math, does well in Grade 4, but then plummets in Grade 5. When you see inconsistent results like this, it's reasonable to assume that the school's instructional practices aren't the major factor: more likely, peer effects, self-selection, and unobserved student differences are driving the results.

That's even more evident in the upper grades:

Not a great showing in Grade 6, and little advantage in Grade 7 and Grade 8. But wait -- what about Algebra 1? Doesn't VCS offer it to their students? According to the data files, no scores were reported  for VCS in Algebra 1 -- which is odd, because there were scores reported for Grade 8 Math, which is the test students take if they aren't taking algebra. So data suppression isn't likely the issue (although I can't say for sure).

Notice how well the VPS middle schools do in Algebra 1? You might think that's a function of good instruction. I'm not saying VPS doesn't do a good job, but there's really another explanation: kids who take algebra in middle school instead of high school are much more likely to do well because they wouldn't be enrolled if the district didn't think they were talented in math. But the data doesn't sort students according to their "talent," or "ambition," or whatever.

Which, again, is a critical point to understand about charter school "gains": Self-selection sorts students on characteristics we can't observe, meaning the charter students, on average, are not the same as the public district students. So when you see inconsistent gains, like here, ask yourself: is this an indication of school success, or of student success?

My takeaway is that VCS, like many charters, is a doing a decent job -- but they are hardly far superior to their hosting public school district. Which brings us back to the relevant policy question:

Senator Sweeney says the Vineland Public Schools should have been making plans for years in anticipation of a funding cut. OK... but VPS has no say over charter school approval or expansion. 

If the VPS school board knew cuts were coming, would they have approved a charter school that spends much more on administration, free-rides on staff salaries, builds facilities that the district will never own, segregates students by special need, and gets, at best, inconsistent test score gains?

If Senator Sweeney is serious about getting school budgets under control, he would do well to step back and think a bit about whether NJ school districts and the state can continue to subsidize charters like Vineland Charter School while simultaneously demanding their host districts lose state aid. Maybe the NJDOE's rush to approve as many charters as they can before the end of the Christie era is worth a bit of the Senator's attention.

Middlesex County up next...

It may be shiny, but...

* Corrected. Sorry about that - peril of being your own editor...

Tuesday, March 14, 2017

NJ Charter Schools - A Fool's Gold Rush: Princeton

Here are all the parts of this series so far:


Franklin, NJ

Princeton, NJ

Vineland, NJ

As in Franklin, NJ, it appears that charter schools are splitting the community of Princeton apart. Go to planetprinceton.com, the town's hyperlocal, and you'll find lengthy and heated back-and-forths in the comments section about Princeton Charter School and the consequences of its expansion on the Princeton Public Schools.

This was brought about by the NJDOE's rush to expand charter school enrollment before Chris Christie's term ends. PCS will grow from 348 to 424 students in grades K-8; as I explained before, this will undoubtedly put additional fiscal strain on the public district schools. And, as always, the local school board and the local voters have no say in whether the expansion should take place.

Before I get to the data, a few things: first, while a particular charter school may work well for a particular student, personal stories are a poor way to make public policy. I have no doubt many PCS parents are happy with their children's school: good for them. But, as I have explained previously in this series, their ability to enroll their children in a charter school comes at a cost to the other families in Princeton and the town's taxpayers.

The real question in school "choice" policy is whether the alleged gains for charter school families outweigh the costs to all of the stakeholders. Those costs, by the way, include having parents within a town battling each other over charter school expansion.

Second: you can go to GuideStar.org and look up the tax records of Princeton Charter School and its affiliated 501c3 organizations, of which there are at least two: PCS Endowment & Capital Fund, and Friends of PCS. In 2015, these two nonprofits held over $1.2 million in assets, and granted over $300K to the school. I assume most of this was raised by parents.

There is nothing wrong, of course, with raising outside funds for your child's school: public district schools do so all the time. But any charter school that relies on such large amounts of philanthropic giving -- by my calculation, over $900 per student in 2015 -- ought not to make a claim that they are "doing more with less."

Which brings me to my third point: while the data clearly shows that PCS enrolls students with special needs, the proportion of their student populations with those needs is considerably lower than PCS's. Again, it's great that individual parents may have had positive experiences with a particular charter school -- but that doesn't make overall trends showing segregation by special need any less true.

Let's get to the data:

Like most suburban New Jersey charter schools, PCS enrolls fewer students eligible for free lunch, a proxy measure of economic disadvantage. This is true year after year.

The proportion of white and black students in PCS and PPS are similar; however:

PCS enrolls fewer Hispanic students...

... and considerably more Asian students. But these students do not appear to speak a language other than English in their homes:

Across the state, charters enroll few to no LEP students. PCS is no exception. And, like most NJ charters, they enroll proportionally fewer students with special needs:

Year after year, PCS enrolls fewer classified students than PPS. The last year where data was not suppressed on specific learning disabilities was 2014. That data clearly show that PCS not only had very few classified students; most of those students had lower-cost disabilities (Specific Learning Disabilities and Speech/Language):

The state's 2011 cost report clearly delineates between these disabilities -- but charter school reimbursements only divide students into speech/not-speech categories. This is a serious problem that needs to be addressed; until then, the concentration of higher-cost special education students into public district schools and away from charters remains creates profound fiscal stress for districts like Princeton. 

Let's look at staff:

PCS actually has a relatively experienced staff compared to most New Jersey charters; still, they have fewer highly-experienced staff compared to PPS. As Martin Carnoy recently noted,* this potentially creates a "free rider" problem for public school districts: charter staffs know they can potentially transfer to higher-paying jobs in district schools after a few years, so they are willing to put up with lower pay earlier in their careers.

Which explains this:

The data is noisy because the number of PCS certificated staff who aren't administrators is small. Still, with the exception of the initial bump (which appears to be caused by one exceptionally well-paid novice teacher -- is this data error?), PCS teachers are generally paid less than PPS teachers with similar experience. 

But there is one area where PCS spends considerably more than PPS: administration.

The charter spends much more than the public school district on administration, and that includes spending on salaries for administrators. Again, charters generally can't leverage economies of scale like public school districts can. Which means the luxury of "choice" is costing taxpayers money they would save if charter schools were more efficient.

Now, the response you'll often here to this is that charters spend less in other areas. But that doesn't necessarily translate into efficiency:

I use "budgetary costs per pupil" as the measure here because, as I've explained repeatedly, you can't compare budgets that includes, say, transportation if the charters rely on the districts to provide that transportation. Anyone who compares overall budgets of charters and public district schools without taking this simple fact into account has no idea what they are talking about and should be ignored.

Yes, PCS spends less per pupil. Part of that is that they spend less "in the classroom" -- on salaries for more experienced teachers (again, there's that "free rider" problem). But they also spend less on support services: things like librarians and child study teams and school nurses who provide valuable services, especially for children with special needs.

Let's see one way this plays out between PPS and PCS:

According to the staffing file I have, PCS had one school counselor for 348 students -- that's all of their support staff. I can only go with the data I've got, folks, and this shows PCS doesn't have occupational therapists or social workers or any other number of certificated support staff. 

Maybe PCS, given its student population, feels they don't need them. That's fine... but it also illustrates that different student populations create different staffing needs and, therefore, different fiscal pressures. Which is why the claim of PCS supporters that their school isn't stressing the host district is very weak.

Let's finish up by looking at outcomes. Again: You can't compare student outcomes between different schools without taking into account differences in student characteristics. And there are other factors to consider:

This year's PARCC results had a lot of data suppression regarding opting-out. We know from the year before that some communities had many parents pull their students from testing. Princeton, by many accounts, was one of those places -- data from the high school this past year (which is really all we have that wasn't suppressed) appears to show opt out rates remained high. But we just don't know...

Still, I use a standard technique here to show the "value" added to test scores for PPS and PCS. This is based on a statewide linear regression with five covariates for student characteristics included. It's crude, but it's better than simply comparing mean scale scores or, worse, proficiency rates. Let's start with English Language Arts:

In Grades 3 through 5, PCS pretty much looks like any other school in the town. "Value" fluctuates grade-by-grade, likely due to cohort effects. PCS is behind Witherspoon in Grade 6, and then ahead in Grades 7 & 8 -- but only by a few points. Keep in mind: the PARCC scores go from 650 to 850, so 5 points or so is really not a very big deal.

Here's math:

Again, from Grades 3 to 5, PCS does OK, but is hardly over and above every PPS school. Witherspoon has the advantage in Grade 6; that turns around in Grade 7 and for algebra. But why? If PCS had some clear instructional advantage, wouldn't we see that in many grades? And a 13 point gain in algebra is nice, but is it really earth-shaking? Might it be explained not by differences in schooling, but by unobserved differences in students?

One point worth making about these graphs is that both PPS and PCS regularly beat predicted scores; not always, but quite a bit. Looking through some of the comments at Planet Princeton, at least some of the PCS advocates agree that PPS is a good school system. They contend, however, that PCS is better for their child. Again, that's fine -- but as a matter of policy, we have to weigh the preferences of these parents against the costs of "choice." So what do we find?

The data shows that Princeton Charter School's inconsistent gains on test scores come at the cost of:
  • Segregation by race, class, and special education need;
  • Increased administrative costs; and
  • A "free rider" problem with staff salaries.
Now, maybe PCS advocates think the test scores aren't the true "gain" -- that there is a value in being able to have a "choice" in your child's school. OK... but why, then, must PCS expand? Can't it survive at its current enrollment level? Must it continue to grow, no matter how that harms PPS, or how much running a redundant school system costs taxpayers? Shouldn't one of the "choices" for Princeton's parents be a public school system that isn't fiscally stressed by an expanding charter school?

It would be great if we had unlimited resources to pour into schools, and we could spend money to give every parent exactly what he or she wants for their child. But we can't even fund the state's own law when it comes to school funding. How can we, therefore, possibly justify spending extra money to satisfy the desire of a small group of families in an affluent township -- which already has a very good school district -- to send their children to a redundant and separate school system?

In addition: the Princeton area is well-known for having many fine private schools. Bifulco and Repack (2014) note that "...charter schools can be expected to attract some number of students from private schools." How many PCS students would be enrolled in a private school if the charter did not exist? What extra burden have the citizens of Princeton taken on by having to host a charter school? How much will that burden expand when PCS's enrollment grows? 

These are important questions, and they can't simply be brushed aside because some parents demand "choices" for their children. Public schools are meant to serve the public -- all of the public. And it's very difficult to see how all of Princeton's citizens are served by allowing this expansion to go forward.

We'll head to another town to look at how the Great NJ Charter School Rush of 2017 is affecting their schools -- right after we take care of some other business...

It may be shiny, but...

* Carnoy makes his point in the case of private voucher schools, but the argument is germane to charters as well.

Sunday, March 12, 2017

NJ Charter Schools - A Fool's Gold Rush: Franklin

Here are all the parts of this series so far:


Franklin, NJ

Princeton, NJ

Vineland, NJ

I was in Franklin, NJ a few weeks ago for a panel on charter school expansion. The town already has two charters, and a third is in the works (founded by, of all people, a winner on The Apprentice). One of the schools currently open, Central Jersey College Prep Charter School, has been granted a huge expansion by the NJDOE. This expansion is part of what appears to be a rush to swell charter enrollments across the state before Chris Christie leaves office next year.

What's so discouraging about the Franklin charter expansion is that the community has been split apart. The charter families feel they have been served well by their schools, and want to see them grow. The families of public district school students, on the other hand, rightly worry that the charters are harming their district's finances.

This tension is being played out across the state and across the nation: charter families' demand for "choice" inherently conflicts with public district school families' demand for adequately funded schools. There are also the interests of other taxpayers, who want efficient schools and worry that charters, because they are redundant and usually inefficiently small, waste funds that could be used to either expand programs or reduce taxes.

The conflict is so great that citizens in towns like Franklin are now bringing civil rights complaints against charters, alleging the charters enroll a fundamentally different student population than the district. There is no doubt they are correct: While it may not be intentional, Franklin's charters do not enroll the same types of students as the district.

I don't know why charter parents in the town, and professional charter cheerleaders in the state, find this statement at all controversial. We know New Jersey charters don't enroll nearly as many special education or Limited English Proficient (LEP) students as their hosting school district. We know charters often have different racial profiles compared to their hosts. As I showed at the panel, the data really is beyond dispute:

Thomas Edison EnergySmart clearly has many more Asian students and many less Hispanic and black students proportionally than Franklin Public Schools. Central Jersey College Prep is trending away from the district.

The charter advocates in Franklin have made the case that they don't discriminate because their lotteries are open to all. They neglect to mention, however, that sibling preference can contribute to segregative patterns. Further, we know from a large and growing body of evidence (see the lit review here) that parents negotiate school choice systems differently, and that distance and social networks play a large role in how their children eventually wind up in their schools.

Again, I don't understand why this is a controversial statement: similar people will make similar choices. To be clear, this is exactly what happens when families make school choices through the real estate market. But a community like Franklin, which is relatively integrated compared to many other New Jersey suburbs, should be concerned about how school "choice" is affecting the racial profile of its schools. For example:

This slide shows that TEECS recently moved from a census tract where 32 percent of the population is black to a tract where the black population proportion is 10 percent. CJCPCS is in a tract where the black population is 7 percent. I don't know how this can possibly help the schools become more integrated.

I won't go through my full presentation, but here are a few more slides:

As is typical for New Jersey, Franklin's charter schools enroll far fewer special eduction students.

The few special education students the charters take tend to have low-cost disabilities, such as specific learning disabilities (SLD) or speech disabilities.

Charter teachers have far less experience than public district teachers. As Martin Carnoy recently pointed out, this creates a "free rider" problem in staffing (more on this later in the series).

Charters spend far more on administrative expenses, likely because they can't achieve the economies of scale found at public district schools.

Let me add a few thoughts before I share these last two slides. There's been a lot of study and debate about the alleged charter school "advantage" -- whether charters get better test scores than public district schools. As I have pointed out time and again, it is simply wrong to compare test scores between any two schools without controlling for their differences in student characteristics (and if we want to be really thorough, we should also take into account differences in spending and other resources).

But there are other issues when comparing charter and public district outcomes. Comparing proficiency rates, for example -- as the Franklin charters did in a handout the night of the panel -- masks all the growth that takes place for students who are above or below what is basically an arbitrary cut point. And even growth measures like SGPs are problematic, because they equate growth at different parts of the test score scale. 

All of this, by the way, leaves aside the question of whether the tests themselves are high-quality measures of student learning. But I'll set that aside for now, and instead show some PARCC scores in math for Franklin's schools last year:

Looking at this, it's clear Franklin's charters have an advantage... but is it really that great? Is 20 to 30 points on a test with a 200-point scale really earth-shattering -- especially considering the charters enroll far fewer students with special needs?

I made a statewide model to assess the "value" the schools add to their test scores after accounting for student differences:

If I had to point to one particularly striking finding, it's that Franklin Middle School does exceptionally well with its Algebra 1 students. But even then: these are the kids taking algebra in Grade 7 or 8, as opposed to the kids who take it in Grade 9 or beyond at the high school. Is anyone shocked by this difference?

Which is really the larger point in all of this: If test scores are so dependent on student characteristics, what, then, is being gained from separating Franklin's students into different school governance systems? Is it really so important to sort the kids into these different school communities? Shouldn't Franklin value having all of its students together? Is individual achievement the only thing the community should care about?

There's another issue that is particular to Franklin: Northjersey.com recently reported that the charters are linked to the Turkish Gulen movement. According to the Philadelphia Inquirer, Fethullah Gulen himself pointed to a network of charter schools in the United States to obtain his green card. Are Franklin's charters in fact part of this network? Who are their vendors? What lease deals do they have?

These are questions that public school boards have to answer regularly as part of normal standards of transparency and accountability. Yes, the system does break down -- I am the first to point this out. But charters don't have to answer to the taxpayers in the same way public district schools must.

Are relatively small, inconsistent test score advantages worth the costs of increasing segregation, redundant and inefficient school management, a relatively inexperienced teaching staff, and lower standards of transparency and accountability?

I won't answer that question -- because the people who should be making the decisions about charter proliferation are the people affected by those decisions. But Chris Christie's NJDOE has the only say over charter approval and expansion; the local school board, elected by the town's citizens to ensure the delivery of a quality education for all students, can go hang.

According to FPS Superintendent John Navally:
This is less about your choice and more for us about the rapid expansion, the number of dollars we spend now and the potential number of dollars that could be spent,” he said.
“That, for the board, has become the issue, one of economics,” Ravally said. “There are things that we have to work on and there are things that we can do better, but the one thing that we’ve been doing in the last year, year and a half, is we’re trying to listen.” [emphasis mine]
In other words: The "choices" charter advocates crave come at a price for other students and for local taxpayers. Districts have fixed costs and must enroll all of the students who are not attending the charters -- students who are more likely to have needs that require more revenues. Taxpayers must come up with the additional funds to support redundant school systems, without any say in the extent of charter proliferation or the benefits of increased accountability and transparency.

Trust me -- we are going to see this set of circumstances time and again as we travel across New Jersey. Next stop: Princeton.

Fool's gold: it's shiny, but...