No, Eva, You Can't Do Whatever You Want

When Eva Moskowitz doesn't get exactly what she wants she stomps her feet, takes her ball and goes home:

Success Academy Charter Schools won’t offer pre-kindergarten classes next year after losing a high-profile fight with city and state officials.

The charter network has refused to sign the city’s pre-K contract, arguing that it includes inappropriate regulations about how charter schools manage their time and design their curriculum. But neither Mayor Bill de Blasio nor State Commissioner MaryEllen Elia has allowed Success to bend the rules, and both have insisted that Success sign the contract or lose funding. 

In recent months, Success officials have continued their fight in court. But with no resolution in sight, the city’s largest charter-school network will close its three existing pre-K programs and will not open two more planned for next school year, CEO Eva Moskowitz announced Wednesday.

“It is unbelievably sad to tell parents and teachers that the courts won’t rescue our pre-K program from the mayor’s war on Success in time to open next year,” Moskowitz said.

There are two issues here: first, does the city have the right to demand that Success sign a contract before it gets funding for its Pre-K program? Reading Elia's decision in the matter makes it crystal clear that not only did the city school district have every right to insist on a contract, it had an affirmative duty to do so:

In addition, the Department’s [NY State Education Department's] RFP required that SUFDPK [Statewide Universal Full-day Pre-kindergarten] programs operate under the jurisdiction of the local board of education, which would be responsible for the proper disbursement of and accounting for project funds. The Department’s RFP required applicants to certify that the program would be conducted in accordance with all applicable federal and State laws and regulations, application guidelines and instructions. Applicants were also required to sign a Statement of Assurances regarding the program. The Department’s RFP explained that the project period for grants in year one would be from July 1, 2014 to June 30, 2015, and that grantees would have the opportunity to renew funding on an annual basis provided that the programs meet quality standards and all applicable requirements, and subject to the annual appropriation of funds in the State budget for this purpose. [emphasis mine]

So the NYCDOE was the applicant, was responsible for disbursing the monies to the Pre-K providers -- which could include charter schools like Success -- and was ultimately responsible for making sure the program was run correctly. And it's not like Moskowitz and her team didn't know this:

On August 24, 2015, SA Harlem 1, SA Cobble Hill and SA Williamsburg began their pre-kindergarten classes. By email dated August 27, 2015, DOE congratulated Success Academy NYC on the opening of its pre- kindergarten classes but advised that, until a contract is executed, “you are operating your Pre-K classrooms at your own risk.”

I know Moskowitz doesn't think the rules should apply to her because her schools are so awesome. But what made her think she was going to get her money if she didn't follow regulations? If she thought the terms of the contract were so unreasonable, why did she apply in the first place?

Back in March, Moskowitz took to the New York tabloids, which are always happy to help spread the gospel of "choice," to argue that de Blasio's administration was overstepping its bounds by insisting on a contract:

Two years ago, New York State passed a law giving charter schools the right to operate pre-Ks. The law provided that with respect to oversight, “all such monitoring, programmatic review and operational requirements . . . shall be the responsibility of the charter [authorizer].” Thus, a charter’s pre-K program would be regulated and overseen just like its other grades.

That's not quite right. Here's the law in question; check out section 3:

3. (a) The universal  full-day  pre-kindergarten  program  shall  make
  awards  to  (i)  consolidated applications submitted by school districts
  which include pre-kindergarten programs offered by  schools,  non-profit
  organizations, community-based organizations, charter schools, libraries
  and/or  museums, which shall demonstrate geographic diversity within the
  area to be served as well as diversity of providers; and (ii) non-profit
  organizations, community-based organizations, charter schools, libraries
  and museums, which may apply individually to the extent  allowed  under
  paragraph  (b)  of  this  subdivision. Any consolidated application must
  include, but is not limited to, the names of individual  locations  and
  providers, applicable licenses, facility lease information, and intended
  staffing plans and certifications.


(b)  Prior  to  submission  of  a  consolidated  application, a school
  district shall widely solicit non-profit organizations,  community-based
  organizations, charter schools, libraries and museums located within the
  school  district  to be included in its application. The school district
  shall notify any applicant who has been  denied  for  inclusion  in  the
  consolidated  application no later than two weeks prior to submission of
  such application. Such eligible providers denied for inclusion may apply
  individually as provided in paragraph (a) of this subdivision. [emphasis mine]

I'm not a lawyer, but this is awfully clear: Success Academies didn't have to apply along with the NYCDOE. They could have gone it alone, but chose not to.

So NYC, as the lead applicant in a consolidated application, has every right to demand the other organizations that sign up play by their rules. If the other organizations don't want to go along, fine -- submit your own application. But don't whine afterward that you don't like your school district's conditions when you could have submitted your own application.

Now, I'll admit that the law isn't explicitly clear in saying that charters applying on their own are self-regulating while charters applying as part of a consolidated application are subject to regulation by the school district which leads the application. But, as Elia writes in her decision, it's the only logical conclusion any applicant could come to:

I agree with petitioners that Education Law §3602-ee(12) states that the monitoring, programmatic review and operational requirements for SUFDPK programs shall be the responsibility of the charter entity, in this case the SUNY trustees. However, for the reasons stated below, I disagree with petitioners’ interpretation of this language. Moreover, I disagree with petitioners’ argument that the effect and intent of Education Law §3602-ee(12) is to prohibit school districts such as DOE, which operate SUFDPK programs via consolidated applications that include charter schools as eligible providers, from regulating the pre-kindergarten programs being funded through the consolidated application. To take petitioners’ position to its logical conclusion would mean that DOE would be required to provide charter schools’ pre-kindergarten programs with public funding without any mechanism to ensure that the statutorily required eight quality elements

and other program requirements are being met and that such public funds are being spent in accordance with the requirements of Education Law §3602-ee, the Department’s RFP and DOE’s RFP. 

Again: if Moskowitz didn't like the terms of this deal, she was free to strike out on her own. If she doesn't like the predicament she's in, she has no one to blame but herself.

One more thing: Moskowitz claims that the monitoring of her Pre-K program should have been left to her authorizer, the State University of New York. Had she applied on her own, however, NYSED would have been well within their rights to ask whether or not SUNY actually has the capacity to properly monitor the program. You might not like NYCDOE for all sorts of reasons, but at least it has an infrastructure in place to monitor Pre-K programs; I very much doubt SUNY has anything equivalent (given what's been going on inside Success Academy lately, I think I'm justified in having my doubts).

Which brings us to the second issue: even if NYCDOE had the right to demand a contract with Success Academy, were the demands of the contract too onerous? Here's Moskowitz once again:

New York City, however, has insisted that charter schools agree to a 241-page contract that regulates every aspect of their programs: their curriculum, field trips, professional development, scheduling, discipline, playtime and use of technology. For example, it prohibits more than three field trips per year involving transportation; dictates the precise amount of playtime that must be allowed (2 hours and 7 minutes); and prohibits the use of a SmartBoard (an interactive screen that is essentially a modern blackboard) for more than a 30 minutes a week.

As Elia notes in her decision, however, every one of these restrictions is both reasonable and in accordance with the guidelines set out by NYSED's Division of Early Childhood Education. In other words: had Success struck out on its own, it probably would have had to adhere to guidelines quite similar to these.

The city has limited school resources; it's reasonable for them to put a cap on field trips that use transportation. We know that high-quality Pre-K should include significant amounts of playtime; it's reasonable for the city to set a lower limit. We know we should limit the time young children spend in front of screens; it's reasonable to set a limit on it, SmartBoard or otherwise.

And if Moskowitz doesn't agree, she shouldn't have signed up. As Elia writes:

Moreover, petitioners’ argument ignores the fact that a charter school’s participation in the SUFDPK program is voluntary. As petitioners would have it, a charter school could voluntarily apply for a grant of State funds but assert an exemption from all grant requirements based on Article 56 and still be entitled to such State funds. The Legislature could not have intended such a result, which is both irrational and contrary to public policy.

Moskowitz doesn't seem to understand this very basic idea about accountability. She seems to think that because she's drilled-and-killed her way to some decent test scores, she should be able to do whatever the hell she wants:

However, there is an important principle at stake. Not only is the city violating the pre-K law, the whole idea of charter schools is that they should be freed from the control of the district bureaucracy. If all pre-K programs, including those run by charter schools, have to subject themselves to the rules and requirements imposed by the city, then we are back to an educational monopoly.

The city says its requirements are reasonable. That’s debatable, but it’s beside the point. If the Obama administration were to seek the power to regulate the content of this newspaper, the issue wouldn’t be whether or not the regulations were reasonable, but whether the government ought to regulate the free press. In our case, while we are subject to regulation, it is regulation by a different government entity to ensure that the city doesn’t have an educational monopoly.

The city also says it has to be able to assure that the pre-K program Success offers is of high quality. However, that is the responsibility of our authorizer, the State University of New York.

Moreover, being subject to the city’s control will diminish the quality of our program, not enhance it. Success’ students pass state tests at more than two times the rate of the district schools and more than three times the rate of the district schools in the poor neighborhoods we primarily serve.

This is a ridiculous argument -- on multiple levels. First, passing noisy, questionably valid assessments doesn't automatically give you carte blanche to ignore basic rules of transparency and accountability. The taxpayers of New York have every right to expect that organizations receiving public funds follow the rules laid out by their duly elected representatives.

Next, given everything we've learned this past year about how Success actually operates, it takes a special kind of smug for Moskowitz to claim superiority over the public district schools.

Finally: has Moskowitz never heard of the First Amendment? Her comparison of the government regulating the content of a newspaper to insisting a public contractor sign a contract is so dumb that only an outlet as bad as the Daily News would put it into print.

This is actually really simple: New York City has clearly laid out regulations regarding Pre-K programs. Maybe you don't like them; fine, challenge them however you want. But you don't get to take public monies and then decide, after the fact, that you don't want to play by the rules.

I wish I could say that Moskowitz is an outlier -- but she's not. The charter sector, despite its many, many, many, many, many failings, has done a terrible job of policing itself. I have no doubt there are many decent, sincere educators operating charter schools -- but they have been largely silent on calling out bad actors within the sector, or addressing the structural deficiencies that encourage bad behavior.

Instead, it appears they are asking for more autonomy, less regulation, and unchecked growth, no matter how badly that may hurt public district schools. And when anyone dares suggest tightening up the regulation of charters, watch out: Eva Moskowitz and her fellow charter cheerleaders, like Campbell Brown, will work themselves up a froth of righteous indignation:

You might say: He was referring to your position about pre-K, which seems inflexible. Every other charter school apparently signed an agreement to follow city rules, why do you think you’re above doing that?

We have fought for 10 years to build schools free from any influence that might dilute or compromise our program for our students. Our independence from district oversight is inextricable from that program.

Why? So you can use a SmartBoard more? So you can have the kids play less? Or, as Elia's decision suggests, might there be another reason?

In response to petitioners’ complaint regarding Article 5(b)(2) of DOE’s proposed contract, which limits SUFDPK providers’ authority to “reject, suspend, expel or otherwise refuse to provide any or all of the Services to any eligible child,” DOE explains that it has a “strong interest in ensuring that disciplinary measures in the consolidated UPK program are age and developmentally appropriate and that 4 year-olds receiving services within the consolidated UPK program are not subject to suspensions or expulsions.” DOE has articulated a rationale for its requirement that is based in sound educational policy and is not inconsistent with Education Law §3602-ee. While petitioners have included such provision in their “non- exhaustive” list, they have articulated no rationale as to how such provision violates either Article 56 or Education Law §3602-ee.

We have extensive documentation of Success Academy's disciplinary practices, including suspensions for very young children. We know how Moskowitz feels about NYC's attempts to remake its disciplinary codes. Is this the "influence" she and Brown are worried about? That the city might stop Success from imposing its "no excuses" philosophy on 3-year-olds?

If that's the case, it's probably better for everyone if Moskowitz just stays out of Pre-K.

Happier times.

ADDING: More from Peter Greene.

Who's Responsible For Misusing Washington DC's Test Scores?

Recently, Matthew Chingos and Kristin Blagg of the Urban Institute published a blog post about the rise in national test scores for schools in Washington, DC. The post has been republished elsewhere and, consequently, cited frequently in the edu-bloggosphere and press.

I have four specific objections to their post, which I'll lay out below (from least to most technical). But let me start by saying I don't think Chingos or Blagg are hacks; far from it. I want to engage on what they wrote because I think it's important and I think they are serious. My tweeting yesterday could easily be interpreted as my saying otherwise, and for that, I apologize.

However...

We now live in an atmosphere where, far too often, education research is being willfully misrepresented by people with ideological, personal, or other agendas. As I've said many times here, I don't pretend that I don't have a point of view. But I also work hard to keep the evidence I cite in proper context.

Others, like Michelle Rhee and Jonathan Chait, have no such qualms. It is, therefore, far past time for public intellectuals and researchers to start asking themselves what responsibilities they bear for the willful misuse of their work. For example...

Objection #1: The framing of the research. I realize this was just a blog post, and that the standards of a peer-reviewed article don't apply here. But I also understand that Urban is an influential think tank with an extensive communications shop, so anything they put out is going to get traction. Which is why I have a problem with starting the post with this paragraph:

Student performance in the nation’s capital has increased so dramatically that it has attracted significant attention and prompted many to ask whether gentrification, rather than an improvement in school quality, is behind the higher scores. Our new analysis shows that demographic change explains some, but by no means all, of the increase in scores.

Chingos and Blagg have got two big, big assumptions laid out right at the top of their post (I'll be referring to this version from Urban Institute's website throughout; all emphases are mine):

1. Student performance in DC has "increased so dramatically."
2. The two factors that influence test scores are gentrification -- in other words, changes in the demographic composition of DC's student body -- and improvements in school quality.

The fact is, as I show below, there is at least one other factor that has contributed significantly to changes in DC test scores, and there are almost certainly others as well.

Continuing directly:

The National Assessment of Educational Progress (NAEP), known as the “nation’s report card,” tests representative samples of fourth- and eighth-grade students in mathematics and reading every two years. NAEP scores reflect not just school quality, but also the characteristics of the students taking the test. For example, the difference in scores between Massachusetts and Mississippi reflects both the impact of the state’s schools and differences in state poverty rates and other demographics. Likewise, changes in NAEP performance over time can result from changes in both school quality and student demographics. 

The question, then, is whether DC’s sizable improvement is the result of changing demographics, as some commentators claim, or improving quality. Our analysis of student-level NAEP data from DC, including students from charter and traditional public schools, compares the increase in scores from 2005 to 2013 with the increase that might have been expected based on shifts in demographic factors including race/ethnicity, gender, age, and language spoken at home. The methodology is similar to the one used in a new online tool showing state NAEP performance (the tool excludes DC because it is not a state). 

We are now three paragraphs in and the authors have repeatedly set up a particular research framework: changes in the NAEP are due either to changes in student demographics or improving quality in schools.

Now, as I said above, Chingos and Blagg are not hacks. Which is why they admirably explain, in detail, why they did not include the most important student demographic factor affecting test scores -- socio-economic status -- in their analysis.

But it's also why, in the third-to-last paragraph, they write this:

The bottom line is that gentrification alone cannot explain why student scores improved in Washington, DC, a conclusion that echoes previous analyses using publicly available data. DC education saw many changes over this period, including reform-oriented chancellors, mayoral control, and a rapidly expanding charter sector, but we cannot identify which policy changes, if any, produced these results.

Go ahead and read the entire post;  you'll find that "if any" clause is the only time they suggest there may be other factors influencing DC's NAEP scores besides student characteristics and school quality.

Am I picking nits? I suppose that's a matter of debate. But when you write a post that's framed in this way, you shouldn't be the least surprised when an education policy dilettante like Jonathan Chait picks up on your cues:

But here is an odd thing that none of these sources mention: Rhee’s policies have worked. Studies have found that Rhee’s teacher-evaluation system has indeed increased student learning. What’s more, the overall performance of D.C. public school students on the National Assessment of Education Progress (NAEP) has risen dramatically and outpaced the rest of the country. And if you suspect cheating or “teaching to the test” is the cause, bear in mind NAEP tests are not the ones used in teacher evaluations; it’s a test used to assess national trends, with no incentive to cheat. (My wife works for a D.C. charter school.)

Some critics have suggested that perhaps the changing demographics in Washington (which has grown whiter and more affluent) account for the improvement. Kristin Blagg and Matthew Chingos at the Urban Institute dig into the data, and the answer is: Nope, that’s not it. The amount of improvement that would be expected by demographic change alone (the blue bars) is exceeded by the actual improvement (the gray bars): [emphasis mine]

We'll save the claims about Rhee's innumerate teacher evaluation system for another day. Does it look like "if any" was enough to stop Chait from making a way overblown claim about Rhee's policies being the driver of DC's test score rise? Read the whole thing: does it seem to you that Chait is willing to entertain any other possibilities for DC's "dramatic" rise in NAEP scores? And are you at all surprised he used Blagg and Chingos's post the way he did, given how they wrote it?

Michelle Rhee, former superintendent of the DCPS, couldn't wait to get in front of a camera and use Blagg and Chingos's post to justify herself:

(0:20) Well, over the last decade really, the scores in DC have outpaced the rest of the nation. In fact, over the last few years, the gains that DC kids have seen have actually led the nation.

We'll also set aside Rhee's claims about "school choice" in DC for now and, instead, point out that at least she is willing to compare DC's "gains" to other systems. Her comparison is, as I'll show below, highly problematic, but at least she's willing to acknowledge that a "dramatic" gain must be viewed in relative terms. (I'm also going to stay away from her response to the allegations of cheating in DC; I'm sure others with more knowledge about this topic will weigh in.)

Rhee, however, is also not willing to acknowledge that DC's gains might be due to factors other than student characteristics or policy changes.

(2:43) And then people said, well it was because of shifts in demographics; you know, basically meaning more white kids are coming in. And this most recent study last week showed that was not the case either. And so it's high time people realized, no, actually, poor and minority kids can learn if they have great teachers in front of them every single day and if families have school choice and are allowed to choose the programs that are serving their kids best. If you put those dynamics in place, wow, kids can actually learn.

You'll notice how Rhee conflates "poor" and "minority" students, even though Blagg and Chingos did not address socio-economic status directly in their analysis.* Nowhere in this interview, however, does Rhee acknowledge there may be others factors involved in the rise of DC's NAEP scores.

Let me be clear here: Blagg and Chingos did not put words into Rhee's mouth or Chait's blog. But they know know how the debate is being conducted now; they know folks like Rhee and Chait are out there waiting to pounce on this stuff. Why, then, frame your conclusions this way? Why limit your cautions to a mere "if any" when you know for certain someone is going to take your stuff and use it for their own ends?

At some point, you have to own at least some of the consequences of putting your research into the public sphere in a particular way. At the very least, you need to point out when it's being abused. Will Blagg and Chingos do that? I hope so.

* * *

This next part is more technical. Skip down if you're not interested.

Objection #2: Where are the secular effects? Bob Somerby of The Daily Howler is usually not included in the rolls of the edu-bloggosphere, as much of his work is media criticism. But he's been writing about the abuse of NAEP scores and other education topics for a long time. Here, he addresses Chait's post about Blagg and Chingos, making a rather obvious point:

Truly, this is sad. Chait accepts the Urban Institute's unexplained demographic projection without even batting an eye. Incomparably, we decided to do something which made a bit more sense:

We decided to compare DC's score gains during that period to those recorded in other big cities. Our decision to run this simple check required almost no IQ points.

Duh. As everyone knows except New York Times readers, NAEP scores were rising all over the country during the years in question. To simplify the demographic confusion, we looked at how DC's black kids did during that period, as compared to their peers in other cities.

In what you see below, we're including every city school system which took part in the NAEP in 2005 and 2013. As you can probably see, the score gain in DCPS was remarkably average:

Growth in average scores, Grade 8 math
2005-2013, NAEP, black students only
Atlanta 19.78
Los Angeles 16.72
Boston 15.17
Chicago 14.29
Houston 13.23
DCPS 11.48
Charlotte: 7.83
San Diego: 7.39
New York City 5.82
Cleveland: 5.34
Austin 4.85

For all NAEP data, start here.

Let me take this a bit further. Here are some quick and dirty graphs using NAEP scores as taken from the Trial Urban District Assessment (TUDA), and the national NAEP. Understand, we have to approach these with great caution: I haven't done anything to adjust for student characteristics or a myriad of other factors that influence test scores. But this is good enough to make a larger point.

Here are the average scale scores for all students on the Grade 4 Reading NAEP test. No question, DC's score rose -- but so did the scores of many other cities. Even the nation as a whole rose a bit during the time from 2003 to 2013.

Economists will sometimes call this a secular effect.** The basic idea is that if you're going to look at a gain for a specific individual, you have to take into account the gains across all individuals. In other words: it's just not warranted to say a gain is "dramatic" unless you put it into some sort of context. You must ask: how do DC's gains stack up against other cities' gains? Other states?***

Maybe Blagg and Chingos don't have the data to run this analysis. OK -- then don't say the gains are "dramatic" or "sizable." But even if you do a comparative analysis and you conclude they are, you have to deal with another issue:

Objection #3: Equating gains on different parts of a scale. Again, here's Somerby:

How well did DC's eighth-graders score in math in 2013, at the end of Rhee's reign?

In theory, it's easier to produce large score gains if you're starting from a very low point. DCPS was low-scoring, even compared to other big cities, when Rhee's tenure began. [emphasis mine]

As Rhee points out, DC scores did rise more relative to many other cities. But gains at the low part of the score distribution are not necessarily equivalent to gains at the high part. It may well be easier to get gains when you're at the bottom, if only because you have nowhere to go but up.

This is one of the biggest problems with the claims of DC's "dramatic" test scores gains: it's hard to find another school district with which to compare it.  Here, for example, are the Grade 8 Reading scores over the period in question:

Atlanta, Cleveland, and DC started in 2003 at close to the same place. Cleveland finished below DC, but Atlanta finished above. Los Angeles, meanwhile, which was below all three, rocketed above DC.

Does this tell us anything about policy changes and their effects on student outcomes? Can we confidently say LA rocks? Absolutely not; we didn't adjust for any student demographic changes or other factors -- but even if we did, we couldn't be sure what was causing the changes.

The point here, however, is that even with these crude, unadjusted measures, there is far more to the story than simply stating that DC has "dramatic" and "sizable" gains. And the gains of high-flyers like Boston and Charlotte can't be equated with the gains of cities on the lower part of the scale.

Again: I didn't see any cautions in Blagg and Chingos's work about this.

Objection #4: Assuming fixed demographic effects. This is going to get a bit technical, so bear with me. Somerby complains that Blagg and Chingos's methods are "unexplained." That's not really fair: they do include a link to a NAEP analysis site from Urban that includes a Technical Appendix that spells out the basic idea behind their score adjustments:

We performed adjustments by estimating regression coefficients using the student-level data from 2003, roughly halfway between the available periods for mathematics (1996–2013) and reading (1998– 2013). Specifically, we regress the test score of each student in 2003 on the set of control variables described above.1 Control variables are included in the regression using dummy variables identifying each of the groups of students for each construct, with the exception of the one arbitrarily chosen group that is the omitted category (except for age, which is included as a continuous variable). 

Using this regression, we estimate a residual for each student across all assessment years, which is the difference between their actual score and a predicted score based on the relationship between the predictors and test scores in 2003. We calculate the adjusted state score as the sum of the mean residual and the mean score in the given test year (e.g., 1996, 1998, and so on), re-normed to the mean of the given test year. Essentially, we perform an adjustment for each testing year, but we base the adjustment on the relationship between NAEP scores and the control variables in 2003. [emphasis mine]

So here's the thing: this method is based on an assumption that the effects of student characteristics on test scores remain fixed across time. In other words: the effect of a student qualifying for free lunch (a proxy measure of SES) in 2003 is assumed to be the same for every test administration between 1996 and 2013.

But we don't know if that's true. If, for example, there is a peer effect that comes from changing cohort demographics over time, there may be a change in how a student's characteristics affect their test scores, even if the characteristics themselves don't change between individuals at different points in time.

Now, there's actually something a researcher can do about this, and Blagg and Chingos likely have the data they need. Rather than just regressing one cross section of test scores on student demographics, regress all the years you're studying but add year dummy variables and interactions with the demographic measures and years. That way, you'll see whether the effects of student characteristics change over time, and you'll get a better comparison between years.

In many cases you couldn't do this, because each interaction adds a covariate, which costs you another degree of freedom in your regression. But Blagg and Chingos are working with microdata, so their n is going to be very high. Adding those interactions shouldn't be a problem.

Of course, it's quite possible doing all this won't change much. And you'd still have the problem of equating different parts of the scale. But I think this is a more defensible technique; of course, given that we're really in the weeds here, I'm open to disagreement.

 * * *

Look, I don't want to equate my methodological qualms about Blagg and Chingos's post with the outright abuse perpetrated by Chait and Rhee. But I'm also not about to let Blagg and Chingos off the hook entirely for what Chait and Rhee did. 

The sad truth is that the willing misapplication of education research has become a defining feature of our current debate about school policy. Is my "side" guilty of this misconduct? Sure; there's stuff out there that supports my point of view that I don't think is rigorous enough to be cited. Of course, my "side" hardly has the infrastructure of the "reformers" for producing and promoting research of questionable value in service of a particular agenda. Still, that doesn't excuse it, and I'll take a deserved hit for not doing more to call it out.

But it would be nice to be joined by folks from the other "side" who feel the same way.

Let me say something else about all this: there's actually little doubt in my mind that Rhee's policies did increase test scores. How could they not? Rhee had her staff focused on test score gains pretty much to the exclusion of everything else. It's not far-fetched to imagine that concentrating on local test-taking carried over and affected outcomes on the NAEP. 

But Rhee's obsession with scores trumped any obligation she may have had to create an atmosphere of collegiality and collaboration; that's borne out by the resounding rejection of Mayor Adrian Fenty, whose reelection was largely a referendum on Rhee.

And that, perhaps, is the biggest problem with Blagg and Chingos's analysis -- our, more accurately, the entire test-based debate about education policy. I'm not saying test scores don't count for something... but they surely don't count for everything. Was it worth breaking the DC teachers union and destroying teacher morale just to get a few more points on the NAEP? Is anyone really prepared to make that argument?

One more thing: I was at a panel at the American Educational Research Association's annual conference this past spring about Washington DC's schools. The National Research Council presented a report on the state of a project to create a rigorous evaluation system for DC's schools.

It was, frankly, depressing. As this report from last year describes, the district has not built up the data capacity necessary to determine whether or not the specific policies implemented in DC have actually worked. Anthony Bryk, a highly respected researcher, was visibly angry as he described the inadequacy of the DC data, and how no one seemed to be able to do anything about improving it.**** From the report:

The District was making progress in collecting education data and making it publicly available during the time of this evaluation, but the city does not have a fully operational, comprehensive infrastructure for data that meets PERAA’s goals or its own needs as a state for purposes of education. To meet these needs, the report says D.C. should have a single online “data warehouse” accessible to educators, researchers, and the public that provides data about learning conditions and academic outcomes in both DCPS and charter schools.  Such a warehouse would allow users to examine trends over time, aggregate data about students and student groups, and coordinate data collection and analysis across agencies (education, justice, and human services).

To confront the serious and persistent disparities in learning opportunities and academic progress across student groups and wards, the city should address needs for centralized, systemwide monitoring and oversight of all public schools and students, fair distribution of educational resources across wards, ongoing assessment of strategies for improving teacher quality, and more effective collaboration among public agencies and the private sector.  In addition, the committee recommended that D.C. support ongoing, independent evaluation of its educational system, including the collection and analysis of primary data at the school level. [emphasis mine]

When you live in a data hole, this is what you get: incomplete research, easily subject to abuse. When we get down to it, we must admit that we know next to nothing about whether any of the educational experiments foisted on the children of Washington DC have done anything to help them.

Given this reality, I would respectfully suggest to anyone who wades into an analysis of Washington DC's schools that they stop and think very carefully about the implications of their research before publishing it. In the absence of good data and rigorous analysis, what you put out there matters even more than it would in places where the data are more comprehensive.

Caveat regressor.


ADDING: It's important to point out, as Bruce Baker does here, that the NAEP gains in DC largely pre-date Rhee-form.


* They did use parent education as a proxy measure of SES in their Grade 8 analyses. But that measure is a self-reported, very rough categorical measure that breaks down parental education into: didn't finish high school, finished high school, had some education after high school, finished college. You go with the data you've got, but this measure is obviously quite crude.

** I first came across the term in Murnane and Willett's Methods Matter, which I highly recommend to anyone interested in quantitative methods in the social sciences.

*** No, DC is not a state. But it does have a population larger than Vermont and Wyoming.

**** To be fair, his opinion wasn't shared by everyone on the panel. Michael Feuer of GWU stated that he thought data collection and analysis had improved, even if it wasn't where it needed to be.

Charter Schools and the De-Professionalization of Teaching

Hey, New Jersey -- and especially all of you "conservatives":

Did you know Chris Christie is using your tax dollars to make what essentially are commercials for charter schools that are linked to a Turkish, Muslim cleric?



Bergen Arts & Science Charter School and Thomas Edison EnergySmart Charter School, lauded by Christie in this taxpayer-financed video, have both been linked by the Gulen Charter Schools website to the Gulenist movement.

No, this isn't a conspiracy theory: the Gulenist charter school phenomenon has been reported by CBS News, The Atlantic, The New York Times, and The Wall St. Journal. These schools, all linked to Fethullah Gulen, have been popping up all over the country and are the subject of concerns expressed by the federal State Department due to their use of H1B visas to admit Turkish nationals into the US.

You'd think that someone in the NJ press would find it notable that Chris Christie, now Donald Trump's transition chair, was using taxpayer funds to promote charter schools tied to a Muslim cleric. But no, that's not news for them -- and neither, apparently, is what I'm going to document below:

Last week, I told the story of Thomas Edison EnergySmart, a school that enrolls far fewer special education and economically disadvantaged students than Franklin Township, its host public school district. Christie sings the charter's praises, even as it drains funds from Franklin's public schools, which educate the kids the Gulen-linked charter does not take in.

Let me quickly show that Bergen A&S is following the same playbook before I get to the heart of the matter. This charter gets most of its students from Garfield, but Hackensack and Lodi are also sending districts. How do these districts compare to the charters in enrolling students eligible for free lunch, a proxy measure of economic disadvantage?

Year after year, Bergen A&S does not enroll nearly as many students proportionally in economic disadvantage. How about students who have a special education need?

The charter school Chris Christie spends your money touting educates far fewer students, proportionally, who have a special need compared to its host districts. And not only that:

The special needs students Bergen A&S does enroll are far less likely to have a high-cost disability. Those include autism, visual impairments, multiple disabilities, traumatic brain injury, emotional disturbance, and so on. SLD (specific learning disabilities) and SPL (speech and language disabilities) are low-cost disabilities comparatively; these are the majority of the few special eduction students Bergen A&S allows into its school. 

Just to show how this plays out in practice, here are the raw counts of enrolled students:

Chris Christie spends your money making videos for schools that have abdicated their responsibility for educating the most challenging, most costly students -- schools that are, by all appearances, linked to a foreign religious and political movement that has been buying influence in the US government.

Everyone OK with this?

* * *

So here's the question I keep coming back to: why would Chris Christie go all in with a couple of Gulenist schools? We all know how his leash-holder feels about Muslims; why, then, is Christie risking the wrath of Trump's base by openly praising these schools? What's the attraction?

The answer, as always, comes back to Chris Christie's pathological hatred for the teaching profession:

Central Jersey College Prep CS is another Gulen-linked charter that draws from Franklin Township; I'm including it in the analyses that follow. What we see here is that the Gulen-linked charters in red have fewer teachers with standard certifications than Franklin's public school district -- in the case of TEESCS, far fewer. These charter schools rely on a teaching staff that does not have traditional training. That's also the case with Bergen A&S:

Again, the charter school has fewer teachers proportionally with a standard credential than the host public school districts. But here's the thing: certificates of eligibility and provisional certifications suggest inexperienced teachers. Stay in your position for a couple of years and you'll gain your license. Are the teachers in these charters less experienced?*

The charter schools Chris Christie loves employ many more inexperienced teachers than the public schools. Why does this matter? Three reasons immediately come to mind:

1) Experienced teachers are good for students. Teaching experience brings gains in student performance well into a teacher's second decade and perhaps beyond.

2) The USDOE has declared, rightly, that the distribution of inexperienced teachers is an issue of racial inequity. It is not fair to burden students with inordinate numbers of inexperienced teachers, even as a matter of "choice."

3) As in every other profession, teacher pay is tied to experience. If you want to cut teacher pay and de-professionalize teaching, the best way to do so is to get rid of experienced teachers.**

These two charts break down experience by years along the horizontal axis. The public schools have a mix of inexperienced, mid-career, and highly experienced teachers. Not so the charters: Overwhelmingly, charters employ a teaching corps with far less experience than their hosting districts.

How does this affect pay?

In North Jersey, teachers get a significant bump in pay around their second decade. But Bergen A&S doesn't have to worry about that: they don't have any teachers with that much experience. Unlike charter schools, the public district schools are bearing the cost of making teaching a true profession with true professional wages by employing teachers with experience and paying them for their years of service.

It's the same in Somerset County: The professional wage teachers earn at mid-career is borne exclusively by the public district schools, and not the charters.

This is one of the biggest problems with the proliferation of charter schools, and yet it is rarely discussed. Charters employ many more inexperienced teachers. In many cases, they pay those teachers a wage similar to inexperienced public school teachers (in some cases, it's even higher) -- but they then expect those teachers to move on after a few years. 

The charter model can't sustain a teacher corps that includes significant numbers of experienced teachers; charters rely on a constantly churning teaching corps.

No true profession consists solely of novices. And that is, I believe, why Chris Christie loves charter schools -- including charters linked to a Muslim cleric. So long as those schools pay teachers less, our boy Chris is happy. Doesn't matter if the charters don't enroll as many kids in poverty or with special needs. Doesn't matter if the charters churn their teaching staffs and perpetually employ teachers with less experience and alternative credentials. Doesn't matter how many scandals are linked to these charters.

All that matters to Chris Christie and others who share his view is that charter schools are de-professionalizing teaching.

Stop bringing up the Gulen thing!

And stop asking for professional wages for teachers!

Just shut up!

Shut up!

* Inexperienced means less than three years total teaching experience in any jurisdiction.

** FTE is "full-time equivalent." I weight the salaries this way so part-timers don't drag down the average wage.

The True History of NJ Teachers Getting Shafted On Their Benefits (for @BillSpadea)

When Former NJ Education Commissioner Jim Gearhart* left NJ 101.5, I thought there might be a chance for some sanity to return to New Jersey's #1 teacher-bashing talk radio station. Gearhart, of course, reveled in beating up on the NJEA, all while making the laughable claim that teachers didn't really want their union to protect their workplace rights, health insurance, and pensions.

When Gearhart left the airwaves, I thought the station might moderate the tone of its drive-time show. It had before: the truly odious Casey Bartholomew had been axed in 2011 to make way for the much more reasonable Deminski & Doyle (I've heard they each have relatives who are teachers). Maybe Gearhart's replacement would be less likely to take pot shots at teachers and their unions.

Looks like I was wrong:

"For the governor to talk about a $250 million cost cutting effort -- which is truthfully a drop in the bucket but still you've got to start somewhere -- and the NJEA just puts their foot down and says: 'No, we're not even going to discuss it,' is, I think, outrageous. I think it's time for teachers to start pushing back."

Bill, I'm a teacher, and I'm happy to push back -- at you.

You see, Bill, I talk to a lot of teachers too. And those teachers have watched while their take home pay has eroded and their health care costs have skyrocketed and their pensions have been devalued, even as their jobs have become more demanding. And they've all come to the same conclusion that I have:

For years, we were forced to pay more and get less on our benefits and pensions with the promise that the state would finally start paying what it promised us for work we have already done. But the state has never followed through. So we're not going to make our families sacrifice any more until we see some sign that the state is going to follow through on its legally binding commitments.

Just in case anyone doubts what I'm saying, let's have a little history review. This timeline from the Communications Workers of America is a good place to start; with a few minor changes, it could describe the continued screwing of just about every public employee in the state.

- 1995: Governor Jim Florio begins the modern era of New Jersey pension underfunding.

- 1997: Governor Christie Todd Whitman essentially pays for tax cuts by underfunding the pensions.

- 2001: Yes, Governor Donald DiFrancesco does raise pension benefits, but he does so basically using the same sort of revaluation tactics that Florio and Whitman had used. I find it funny that so many of the state's conservatives are appalled at DiFrancesco's revaluations, which befitted workers, while they say nothing about Whitman's revaluations, which led to tax cuts.

- 2004: Teachers' mandatory contribution to the pension, which had been as low as 3 percent, is raised to 5 percent.

- 2007: By now, everyone (except Chris Christie) knew the pensions were in trouble and had to be fixed. Teachers and state workers now paid 5.5 percent into their pensions and saw the retirement age go up 5 years. State workers now paid 1.5 percent of their pay to health benefits. The deal everyone agrees to is that in exchange for these concessions the state will start funding the pensions.

Those payments lasted only about two years.

- 2008: The state raises the retirement age again.

- 2010: Governor Chris Christie makes significant changes to the pension for all new hires, and now requires all current employees to pay 1.5 percent of their salaries toward health benefits. At this time, a report is released from respected Labor and Employment Relations Professor Jeffrey Keefe of Rutgers University which shows: "...full-time state and local employees are under-compensated by 5.88% in New Jersey, in comparison to otherwise similar private-sector workers."

- 2011: After running a campaign in which he said explicitly: "Nothing about your pension is going to change when I am governor," Chris Christie, with the support of many Democrats in the Legislature, passes a sweeping pension and benefits overhaul law. As CWA explains:

The plan for increased worker contributions is phased in over four years. At the end, in 2015, workers pay 25% more to get 30% less pension. Workers are required to pay an increasing amount of the health care premium, with a top rate of 35% of premium for the highest earners. The state budget includes a 1/7th payment for FY12. Christie makes the payment on the final day of the fiscal year, in June 2012. In response to the 2010 Supreme Court ruling, the legislature creates a contractual right to the funding of the pension.

- 2013: During the gubernatorial election, no one in the press cares to ask either Chris Christie or his opponent, Barbara Buono, how they plan to raise the revenues for a full pension payment by 2017.

- 2014: To the surprise of no one sentient, Christie refuses to make the payments his own law requires. Meanwhile, reports begin to surface about inordinately high management fees paid to Wall Street firms linked to Christie. The unions file suit.

- 2015: Reports of malfeasance in the management of the pensions continue. The fourth year of the 2011 Ben-Pen law starts: a teacher making $65K now pays at least 19% of her premium for family medical coverage. As NJ Spotlight notes:

Today, however, while the cost of New Jersey public employee health insurance coverage remains the third-highest in the nation, most New Jersey public employees are paying more than the national average for state government workers toward their health insurance costs, an NJ Spotlight analysis shows. 

In fact, the average New Jersey government employee is paying more for individual health insurance coverage than government workers in any other state and the 10th-highest average premium for family coverage in the country. 

Further, state and local government workers are paying a much higher percentage of the cost of their individual health insurance policies than private-sector employees in New Jersey have been paying, and not much less than the percentage paid by the state’s private-sector workers for family coverage. [emphasis mine]

Meanwhile, New Jersey's public employee pensions have devolved into one of the least generous in  the nation, according to a New Jersey Policy Perspective analysis. And yet the state's highest court rules New Jersey doesn't have to fund the pensions, and can instead set the state up for a looming disaster.

Christie's only policy response to all of this is to appoint a commission to put out a plan that everyone with any sense knows can't possibly work.


And so here we are, halfway through 2016 and not a serious plan in sight for how to deal with any of this. As I've shown here (again, thanks to CWA for their excellent timeline), the teachers and police officers and firefighters and state workers have been making sacrifices for years, each premised on the idea that if we just give up some more, the state will finally do the right thing and start paying its fair share.

Well, every time we've paid more, the state has screwed us over. Our pension payments were 3 percent not very long ago; now they're 7.5 percent, our COLA is under attack, and our benefits are worse. Our health care was a negotiated part of our total compensation; now the state has compelled us to pay up to 35 percent of our premiums, and the care we're getting is worse.

And this has all been during a period when our raises were shrinking -- and we were already behind the private sector to begin with.

Bill, I'm sure the NJEA would support all sorts of health care insurance reforms for its members if those reforms saved money without diminishing care. In fact, reports from your own radio station confirm this:

NJEA President Wendell Steinhauer said the unions would favor changes that lower premiums but that the union mistrusts Christie because the state didn’t follow through on promises under 2011 pension and health benefits reforms. 

“No, we’re not looking to make changes in it just because the governor says ‘I’m demanding $250 million.’ We don’t accept that,” Steinhauer said. “He’s just saying, ‘Find money,’ he’s not saying, ‘Find it in a win-win situation for members and for the state.’” 

Last year, the NJEA agreed to an earlier round of money-saving changes on things such as compound prescriptions and hepatitis C medicine. The State Health Benefits Plan, for workers other than school employees, also approved three additional smaller changes. 

Between the two health plans, those earlier changes are expected to save the state $197 million in the coming year – $135 million for the SHPB and $62 million for the SEHBP. Even with that, state healthcare costs are still projected to increase by $290 million in 2017.

Bill, when you say the NJEA won't even discuss changes to health benefits, you're dead wrong -- again, according to your own reporters. What NJEA won't do, what the teachers of this state won't do, and what every public employee won't do anymore is to continue getting the shaft by giving up more in compensation without the state paying its agreed upon fair share.

New Jersey public employees are the only ones who have made their full payments into the pension system. We and our families are the only ones who've made pension and health care sacrifices in an effort to fix a mess we didn't create. But now, we refuse to continue to be patsies. 

After all of our previous concessions, we're not going to give up one more damn thing until this state starts meeting its obligations once and for all.

One more thing, Bill: if you want to try to play the game where you separate teachers from the union leaders, good luck. I've never agreed with my union about everything; I don't think any public employee is ever 100 percent aligned with his union. But I damn sure trust my union way more than I'll ever trust Chris Christie or the Democrats who supported his repeated screwing of public employees.

If there is dissension in my union right now, it's largely because there are who members don't think NJEA has fought hard enough against the steady erosion of our pay, our benefits, and our workplace rights. Whether that's true or not remains an open question; what's not open to debate is whether teachers and other public employees think they have been treated fairly over the last few years. I guarantee you they have had enough of giving and giving and giving and getting nothing back in return.

These are decent, hardworking people who do important jobs. They teach our kids and protect our streets and put out our fires and build our bridges and provide social services to our neediest and work hard to keep vital governmental services available to the citizens of New Jersey. They have, perhaps grudgingly, made many sacrifices to try to keep this state afloat.

But they aren't suckers.

ADDING: Bill, it takes a lot for me to give up on a media figure. Want to hash this out? Drop me a line.

* It's a joke. Well, not really...