Christie Visits ANOTHER Gulen-Linked Charter School

There are times when I am astonished that the press doesn't pick up on a particular story. For example: according to activists on Facebook, Chris Christie is having a private meeting tomorrow, June 30, 2016, at Paterson Charter School for Science and Technology.

If this is the case, it will be the third time since this spring that Chris Christie has visited a charter school linked to the controversial Muslim cleric Fethullah Gulen, a Turkish expatriate living in seclusion in the United States.

On May 16, Christie visited Thomas Edison EnergySmart Charter School in Franklin. Two days later, he trekked to Bergen Arts & Sciences Charter School in Hackensack. Thomas Edison, Bergen A&S, and Paterson Science & Tech have all been linked by the Gulen Charter Schools website to the Gulenist movement in the US.

As I've written previously, the proliferation of Gulenist charter schools is not some wild-eyed conspiracy theory: it's been reported on by CBS News, The Atlantic, The New York Times, and The Wall St. Journal. These schools, all linked to Gulen's movement, have been popping up all over the country and are the subject of concerns expressed by the federal State Department due to their use of H1B visas to admit Turkish nationals into the US.

Given how closely tied Christie is to Donald Trump -- who wants a ban on Muslims entering the country (although even he doesn't seem to understand his own plan) -- I can't understand why no one in the state press has pursued this story. Why is Christie praising so many Gulen-linked charters? Why is he visiting so many of them?

Back in 2011, Leslie Brody, when she was writing for The Record, broached the subject with the CEO of Bergen A&S:

In some states, such as Texas, charter schools led by Turkish immigrants have caused controversy, with critics claiming the schools were used to bring in teachers from Turkey and give contracts to Turkish businesses without fair bidding. In June, The New York Times ran a lengthy examination of these schools, citing some researchers' findings that many were inspired by the views of Fethullah Gulen, a Muslim preacher. Gulen, who lives in the Pocono Mountains, has promoted peaceful dialogue and tolerance but had critics who feared his influence in Turkish government. 

Guvercin said he admires Gulen — just as he admires Gandhi — but his teachers never talk in class about Gulen's philosophy. He stressed that charter opponents should visit before forming opinions. "Some people are not comfortable with any ethnic backgrounds," he said.

Sorry, but that's way too easy. The concern over Gulen-linked charters isn't about staff ethnicities or creeds; it's about transparency. The Ohio press has reported on Gulen-linked charters' use of H1B visas to import Turkish nationals as teachers. In 2011, the NY Times investigated how Gulen-linked charters in Texas made deals with Gulen-linked contractors. The Turkish government (admittedly, hardly an impartial party) recently filed a complaint in Texas related to this investigation. New Jersey taxpayers have every right to know whether there are similar issues with the Gulen-linked charters in this state.

Chris Christie recently proposed a radical transformation of the state's school aid system. Under his plan, aid would be slashed for urban districts -- but, supposedly, not for charter schools. If charters are going to get more state funding than their host districts, that's all the more reason to start asking how that money is being spent.

Here is comparative spending by category for the Paterson Public Schools and PCSST, according to NJDOE data. Yes, budgetary per pupil costs are higher in the public schools -- but much of that is driven by this:

The Paterson Public Schools enroll more students proportionally with a special education need than PCSST. Notice, in the spending graph, that PPS spends about $3,300 per pupil on support services: the services that are especially critical for special needs students. The charter, however, spends nothing on those services according to this data. Instead, they spend more on administration, and much more on their physical plant.

Where is that money going? The Gulen Charter Schools website has a detailed description of the byzantine financial transactions involved in the leases PCSST pays to Apple Educational Services, and how New Jersey Economic Development Authority Charter School Revenue Bonds were used to finance the deal. Apple ES is a foreign non-profit corporation registered in New Jersey. According to its 2014 tax forms (obtained at Guidestar.org), Apple ES controls $36 million in net assets, but also has $34 million in liabilities, primarily bonds issued by the NJEDA. It also pays out $738 thousand in compensation to its staff.

This sort of behavior is quite typical for the charter sector: taxpayer-backed bonds are being used to acquire property that is transferred to private, albeit non-profit, hands,and the bonds are paid off with public funds collected by charters from their host districts. The only way this works, of course, is for the charters to keep expenses low enough to divert enough funds into their lease payments. How does PCSST do this?

PCSST certificated staff have far less experience than Paterson Public Schools staff. And that helps keep costs low:

PCSST staff are actually paid slightly more than PPS staff when accounting for experience. But there isn't even one PCSST staff member with more than 10 years of experience according to staffing files -- and that keeps overall staff expenses low. The money can then be diverted into lease payments. Of course, the evidence continues to pile up that teaching experience matters, even in the later part of a teacher's career. PCSST's students are being denied the benefits of an experienced faculty, but there appears to be plenty of money for bond payments.

Now, all this might be acceptable if PCSST was getting far superior results. But when you account for different student populations as I do here*, it turns out the charter is doing OK -- but not much better than that (click to enlarge).

Compared to PPS, PCSST does relatively well on Adjusted Grade 5 math -- but it's not at the top of the pack.

PCSST's Algebra 1 scores are relatively weak.

PCSST does fairly well on adjusted Grade 6 English Language Arts scores.

Grade 10 ELA is average.

PCSST is, undoubtedly, full of hard-working, talented students and dedicated teachers. I congratulate them on their successes, but let's be clear: Paterson Charter School for Science and Technology is not far and above the Paterson Public Schools in terms of its test-based outcomes.

Why, then, should should PCSST be exempt from Chris Christie's budget slashing tax plan, but Paterson's public schools -- which serve more special needs and Limited English Proficient students -- have to suffer? Especially since there is so much about PCSST's governance and funding that we simply do not know?

I wish someone in the NJ press corps cared enough to ask these rather basic questions.

Fethullah Gulen? Never heard of him...

ADDING: By the way: 

PATERSON – The city school district is in the process of making as much as $20 million in last-minute spending cuts in its 2016-2017 budget, a belt-tightening that comes on the heels of $45 million in reductions made several months ago. 

Paterson Press reported two weeks ago that the district would have to make millions of dollars in extra cuts, but at that time officials had not disclosed exactly how much spending would have to be trimmed. 

Board of Education President Christopher Irving said state-appointed schools superintendent Donnie Evans informed him that between $15 million and $20 million must be trimmed from the district’s $468 million budget that takes effect on July 1. An extra $20 million in cuts would amount to about 4.2 percent of the budget. 

[...] 

Cuts imposed a year ago eliminated more than 350 district jobs, including about 170 teaching positions. The district struggled to recover from those reductions and had to use substitute teachers for some classes during this past academic year. Local education advocates say the relentless series of spending reductions will undermine the district’s moderately-successful efforts to improve graduation rates and student test scores. [emphasis mine]

Charter advocates are always making the case that parents are "choosing with their feet" when they enroll their children in charters. But if a parent has to choose between chronically underfunded public district schools and charters that serve fewer special needs children and are not transparently managed...

What kind of "choice" is that?



* I use a linear regression model to adjust scale scores on the 2015 PARCC exams. The model is:

ScaleScore = f(pctFreeReducedPriceLunch, pctSpecEd, pctLEP)

Free lunch and LEP are from 2016; there are some clearly misreported 2015 demographic figures for PPS, so I opted for more recent figures but matched them with 2015 test scores. Special education is a three-year average from 2012 to 2014. FL and SpecEd are significant at the p < 0.01 level. LEP significance varies; I decided to keep it in all models for consistency's sake.

Robust standard errors (N is between 1215 (Grade 5 Math) and 393 (Grade 9 ELA)) with typical colinearity checks (VIFs). 

Caveat regressor.

The Illogic and Cruelty of Christie's "Fairness Formula"

I'm working on an analysis of the "Fairness Formula" -- Chris Christie's regressive school funding scheme -- and will have it ready shortly. Until then, read my piece at NJ Spotlight on this awful plan.

And, for now, let me add this:

Christie's plan is not only divisive, destructive and, yes, racist -- it is also wholly illogical, for at least two reasons:

1) The plan will inevitably drain money from Christie's beloved charter schools, which rely on "pass-through" funding from the less-affluent local districts Christie is planning to screw over.

NJ charter schools get their funds from local districts, which pass on the state aid and local tax revenue they collect based on the number of students a charter enrolls, and whether those students are at-risk (qualify for free or reduced-price lunch, a proxy measure of economic disadvantage), are Limited English Proficient, or have a special-education need.

You'll often hear the NJ charter industry complain they don't get as much money per pupil as the public district schools. While it's true the law says they get 90 percent of the budgeted amount per pupil compared to the districts*, and while adjustment aid plays a role in some districts, the main reason for the disparity is that charters, on average, enroll fewer at-risk, LEP, and special education students compared to their host districts. The state's law says districts -- and charters -- get more money for these students, a policy based on reams of research. If you don't enroll them, you don't get the funds. It's that simple.

These charters are disproportionately concentrated in the districts that will suffer most under the "Fairness Formula": the "A" and "B" districts as designated by the state's District Factor Group (DFG) classification system. Which means that even though they enroll fewer at-risk students proportionally compared to their hosts, they still enroll many more than the 'burbs.

And that means the charters will also suffer under the "Fairness Formula," even with their different student populations -- if Christie really means what he says:

If we were to take the amount of aid we send directly to the school districts today (in excess of $9.1 billion) and send it equally to every K-12 student in New Jersey, each student would receive $6,599 from the State of New Jersey and its taxpayers.  Every child has potential.  Every child has goals.  Every child has dreams.  No child’s dreams are less worthy than any others.  No child deserves less funding from the state’s taxpayers.  That goal must be reached, especially after watching the last 30 years of failed governmental engineering which has failed families in the 31 SDA districts and taxpayers all across New Jersey.

OK -- then that means the charter students only get $6,599 each, right?

NJS: Would this hurt charter schools in these districts as well?

C: I’m open to talking to charters about a “pay for success” model. If you kept the charter law as it is now, you would have to go to a new way of funding charters. It couldn’t be tied -- and it shouldn’t be tied -- to funding a percentage of a district’s funding … It would have to change.

We would have to go back to a way of funding charter schools that is not tied to what a district gets. [emphasis mine]

Sorry, but it is impossible to reconcile these two statements. You can't claim the moral high ground in demanding that all children should get the same funding, then turn around and in the same breath add: "But not these children."

Increased per pupil charter funding either has to come either from local property taxes or from state aid. In either case, Christie would be drawing funds from one group of children to pay more to another. He's trying to have it both ways. No one should let him get away with this.

2) The plan will inevitably raise taxes on districts with low property values, in contradiction to Christie's stated preferences:

Governor Christie: No child in this state is worth more state aid than another. No family in this state should have to disproportionately pay- you’ve got a bigger house, you’re going to pay more in property taxes, you’ve got more land, you’re going to pay more in property taxes- that’s not what I’m talking about. We’re talking about 52% of your property taxes going to schools in 546 districts and 26% of your property taxes going to schools in 31 districts. We’re talking about 546 districts having to divide $88 billion over the last thirty years and 31 districts diving $97 billion. Where did the money go? And what did you get in return for it? But an even more important question than what did you get in return for it- what did those children and their families get in return for it? Underachieving schools that do not prepare them for the jobs of the future or the careers and college life of the future. We are paying a king’s ransom in those 546 districts but in most of them, we are preparing our students for the future. We now should share with each other the money that our state taxes. It can be fixed. We don’t have to rely upon the courts to do it. [emphasis mine]

As I will show in a bit, New Jersey's most underprivileged students have, in fact, made substantial gains during the period of school funding reform. But look at the bolded part of the statement: Christie agrees that property-wealthy school districts should pay more in taxes.

I can't stress this enough: even the most conservative education pundits agree that it's unfair to force property-poor districts to raise their tax rates relative to property-rich districts. In a recent piece at the very reformy The 74, two of the nation's biggest conservative critics of public school spending concede this basic point:

Even some conservatives say the new plan, an effort by Christie to lower property taxes, goes too far. Mike Petrilli, of the right-of-center Fordham Institute, told The 74 in an email, "Governor Christie may be right that property taxes are out of control, as is government spending in some municipalities. But taking it out on kids from poor families is like punishing a gambling addict by taking away his kids' lunch money.”

Christie’s argument that every student should get the exact same state resources may seem like common sense, but in fact it would strongly favor students from wealthier, mostly white communities. More affluent areas are better able to raise money for schools because their high income and property values creates a much larger local tax base. Equal state funding on top of unequal local funding leads to inequity overall. 

“Virtually every state in the union recognizes that state funding must compensate for low property tax bases when local property taxes are a significant proportion of total funding,” said Hanushek. “Particularly given the large number and highly variable districts in [New Jersey], not recognizing differences in tax bases would introduce some very destabilizing impacts across districts.” [emphasis mine]

I don't much care for the undertone of Petrilli's metaphor (equating urban districts to gambling addicts is a form of dog-whistling, intentional or not). But he and Hanushek are correct in pointing out that when you shift more of the burden for funding schools on to regressive property taxes, you're going to wind up lowering the effective tax rates in wealthy districts while jacking up rates in property-poor districts.

Again, Christie can't have it both ways: you can't be for flat, let alone progressive, taxation and base a larger portion of school revenues on regressive property taxes while keeping state aid flat.

As I said in my last post, Chris Christie has a nasty habit of slapping together education policies whenever he needs a political boost. The illogic of the "Fairness Formula" is clearly the product of haste. Its self-contradictions betray Christie's complete lack of seriousness.

Maybe I should have thought this through a little more...

ADDING: Paul Mulshine, columnist for the Star-Ledger, has been a primary driver in the media of this cruel and foolish scheme. I can't claim to have read everything Mulshine's written on this (I have a pretty strong stomach, but even I have my limits), but his m.o. appears to be throwing out a-contextual anecdotes about things like pre-K truant officers in Elizabeth to make the case that state funding can be slashed in the cities without bringing harm to students or causing local urban property taxes to skyrocket.

Mulshine's conservative commenters then take these flimsy arguments and use them to justify their own greed. Case in point:

2 hours ago

So many school districts, police depts. and municipal agencies in such a close space - all fully funded by John Doe the taxpayer - this is a recipe for disaster - can't wait for my last kid to finish school!!!!  Time to move...

Yes, please, keep your kid in New Jersey's high-performing schools, taking advantage of their rich resourcing and historical (if inconsistent) commitment to funding equity. Then, after they get great educations... get the hell out!

This cynicism is the product of decades where the likes of Paul Mulshine are given space in the press and our national discourse while the left is told to get stuffed. As Atrios has said: the only acceptable arguments in America are between New Republic and Free Republic. Far-right Randians like Mulshine get access to the op-ed pages of New Jersey's largest newspaper, "balanced" by anti-union neo-liberals like Tom Moran, who champion "market-based" reforms while ignoring all the evidence against them.

In this world, education isn't a public good or a universal right; it's a commodity to be purchased, like everything else. Appeals to the collective good are actually considered unpatriotic; the only virtue in Mulshine's America is self-interest. The less enlightened, the better.

Is it too early to start drinking?


* The ostensible reason for the 90 percent is that even the architects of NJ's charter funding law understood that public school districts have responsibilities that charters do not, and it would be unfair to give charters equivalent funds if they aren't doing equivalent jobs. Transportation, for example, is the responsibility of the districts; why should charters be paid for providing busing if they don't actually provide it?

Whether the 90 percent rule adequately addresses this difference is an open question. I think there's good reason to believe it's not an adequate equalizer.

Once Again, @GovChristie Treats Education Policy Like a Joke

I have a piece over at NJ Spotlight about Chris Christie's insane school funding proposal. If there is an upside to this madness, it's that we just might finally have a serious discussion about how education funding really works in this state. Obviously, I'm going to have a lot more to say about this, so stand by.

But before I do, I want to remind everyone of one immutable truth: Chris Christie has never been serious abut education policy. If you doubt me, just look at the very first piece of propaganda he's put out in support of his "Fairness Formula."

Christie went to the home of a South Plainfield family and literally sat at their kitchen table and pitched his scheme. Toward the end, Christie spoke to the family's 10-year-old son:

Governor Christie:  Aiden how you doing man? You’ve been hanging in there during all this, is it all right? What are you doing over the summer? 

Aiden Carlisle: Summer baseball and summer soccer. 

Governor Christie: Great. Have fun this summer. You’ve earned it. You worked hard in school, earn your summer. It’s good. Enjoy it. [emphasis mine]

Really? Young Aiden here has "earned" his summer off? I seem to remember almost exactly three years ago that Chris Christie had some very different ideas about summer and school:

The governor told a friendly Bergenfield crowd Tuesday that Garden State students are in need of more hours in the classroom and longer school years in order to stay competitive. Christie blamed special interests with blocking those changes for purely their own personal interests.

“They don’t want a longer school year, they like having the summer off,” said Christie, referring to the adults – not the students – who he accuses of blocking the reforms.

Christie argued longer school days and years are needed to ensure students are educated. [emphasis mine]

Why wasn't the governor telling young Aiden here that he wants him back in the classroom instead of playing baseball and soccer? Why wan't he trying to make the case to Aiden's parents that their son needs to spend his summer in the classroom to "stay competitive"?

The answer, of course, is that Chris Christie was never serious about lengthening the school year -- it was all a diversion from Bridgegate**. Christie never had a plan to pay for a longer year, never had a plan to upgrade all schools so they all had air-conditioning, and never even asked whether this was something New Jersey families actually wanted. It was all a cheap political ploy.

One more thing:

Paul Carlisle: That’s the deal. What’s the deal? Tell him what the deal is. 

Aiden Carlisle: School first and then sports. 

Governor Christie: Excellent. Well I’ll tell you, you can do it all the way through. My oldest son just graduated from college and he was baseball player in college, he played for Princeton and they just won the IVY League championship and went to the NCAA tournament. But he got his good grades and he graduated on time. So, you can play sports and do school but I had the same deal with him. School first, sports second. Grades aren’t good you can say adios to the coach. And it worked. He went all the way to college, he played baseball in college and it was a great time for him and it made his college even better. He made lots of great friends and had lots of great experiences. So sports is great and if you do that while getting good grades you’ll do really well. So good for you. Keep it up buddy. That’s really great. Thanks for letting me come to your house. [emphasis mine]

Yes, Christie's son did play high school baseball -- at a private school that currently charges $36,900 in tuition, far in excess of what any NJ public school spends per pupil (and tuition doesn't even cover the full expenses of the school). A school so big and beautiful...

... you could literally land a taxpayer-funded helicopter on its fields.*

Apparently, it's OK to spend that kind of scratch on school when you're Chris Christie's kid; not so much if you're a student in a school that serves many children in economic disadvantage. But don't point out this screaming hypocrisy to Christie:



Like I said: if Christie's insane "Fairness Formula" leads to a meaningful conversation about school funding in New Jersey, that's great. But let's not pretend for one minute that Christie has put forward a serious policy: he's never been serious about education. He is nothing more than a political opportunist, cynically chucking out incoherent and destructive policies to his political base like so much rancid red meat.

More to come...

Stop pointing out my incoherence and hypocrisy!

Shut up! Just shut up!

ADDING: Speaking of Chris Christie and school sports, this, from 2010, is worth pulling out of the memory hole:

Does it worry you that, as a result of your budget cuts, some districts are cutting back on sports or having athletes pay to join high school teams? 

It does. I don't think it should have to. We're looking at this all backwards. Teachers and administrators should be looking at what they're paid and what their raises are, and if they're really worried about the kids, sports are an integral part of a kid's education. When I see some people being restricted in any way from participating in that, it does concern me, because I was very involved in it as a kid at public schools here. I also know it's a false choice. They can make other choices and they refuse to.

Christie's point is quite clear here: greedy, overpaid teachers were keeping kids from playing sports.  Of course, the idea of overpaid teachers is a myth, and Christie had just refused to renew the millionaire's tax. But, hey, why pass up a chance at getting a few more cheap shots in at teachers, amiright?


* Just a reminder of why Christie had to take an expensive helicopter flight to go see his kid play baseball:

Like Daniels, Christie never offered proof. Like Daniels, Christie blew everything out of proportion. And, like Daniels, Christie has abused his office for personal uses. My personal favorite was when he got the state police to fly him in a helicopter to his son's private school to watch him play a baseball game, then ducked out early to meet with GOP bigwigs from Iowa.

Tell those damn teachers to get back to work!

That particular incident got a good bit of play in the press. What barely got mentioned, however, was that Christie also used those same helicopter privileges to fly to a state-funded "town hall" meeting in Nutley, where he proceeded to say New Jersey's teachers were "lying" to their students.

No, I am not making that up.

2018 can't come fast enough.

** To be clear: Christie made noise about extending the school day and year before Bridgegate. But he brought it up again, after having stopped talking about the idea for a while, in his 2014 State of the State address in a clear attempt to divert attention from the scandal.

Does Opting-Out "Punish" Schools? Not As Much As Serving High-Needs Students

The opt-out scolds -- those who spend their days tut-tutting at parents who've decided to take their children out of high-stakes standardized tests -- having been warning over and over that there will be serious financial consequences for schools that do not have high test participation rates.

Could they actually be right?

BROOKLYN — The state has penalized 16 high-performing city schools — potentially costing them each up to $75,000 in grant money — because of their exam opt-out rates, DNAinfo New York has learned.

These schools were on track to win recognition from the state as “Reward Schools" — an annual honor that makes schools eligible to apply for grants — but were not included in the list because they failed to meet a 95 percent participation rate on the exams, state education officials confirmed. 

“While U.S. Department of Education [USDE] guidelines allow states to impose sanctions on districts specifically for failure to meet participation requirements [of the tests], including the withholding of state funds, New York State has not taken such action against any district or school,” State Education Department spokeswoman Jeanne Beattie said.

“However, under New York’s flexibility waiver approved by USDE in 2010, a school must meet all applicable participation rate requirements to be designated as a Reward School and therefore eligible for a grant.” [emphasis mine]

Angry Andy Cuomo had said that districts wouldn't be punished for high opt-out rates. But it looks like schools that have significant numbers of kids who don't take tests are less likely to be eligible for these grants:

The city’s Department of Education declined to comment on schools losing the Reward designation, but noted that schools were informed of the state’s criteria in a DOE FAQ, updated in March.

“Regardless of the reason (i.e., absence or refusal), if fewer than 95 percent of a school’s students or one or more of its subgroups of students (e.g., less than 95 percent of black students, students with disabilities, etc.) take the math or ELA assessments, the school is designated having failed to make “Adequate Yearly Progress” for that school year,” the FAQ noted.

“Schools that do not meet the participation rate criteria are not eligible to be considered for ‘Reward School’ status,” the FAQ continued, “which highlights schools identified as demonstrating high performance or high progress relative to other schools in the state.”       

Last year, 143 city schools were selected as Reward Schools, according to the state, but only eight of them met the eligibility requirements for the grants. [emphasis mine]

Wait -- there's only a 6 percent chance of NYC schools actually getting money, even if they are "Reward" schools?  Doesn't seem like most of these schools were going to miss out on much -- but all this raises an interesting question...

If these grants only go to "Reward" schools, what does it take to get "rewarded"?

In other words: are there any characteristics that "Reward" schools, which get a crack at additional funding, share? To answer that question, let's compare "Reward" schools in New York State with "Priority" and "Focus" schools, which have been identified this way by the NY State Education Department (NYSED):

• Focus Districts have schools with low academic performance on the Grades 3-8 ELA and Math Tests or low graduation rates for certain groups of students, such as those who are economically disadvantaged, students with disabilities, and English language learners. The performance of these schools is not improving.
• Focus Schools are schools within Focus Districts that have low academic performance that is not improving.
• Priority Schools are schools with the overall lowest student academic performance on state assessments and persistently low graduation rates.

I'll bet you already know exactly where this is going (click to enlarge)...

Want to become a "Reward" school in New York State, and avoid getting designated as "Focus" or "Priority"? First thing you need to do is make sure you keep your Limited English Proficiency (LEP) rate low; no foreign language students for you. Next, make sure you have small proportions of students of color, either black or Hispanic. Then stock up on Asian and white kids.

Bad as all that is, the two columns furthest to the right make the situation even more awful. Incredibly, the schools New York State punishes have many more students proportionally who have a learning disability (SWD) than the schools the state rewards. And "Focus" and "Priority" schools are far more likely to have large proportions of students who are in economic disadvantage.

Am I the only one thinks it's insane to make special grants, even if they are small, available to schools that serve fewer students of color, fewer students with special needs, fewer students who speak a language besides English at home, and fewer students who are in economic disadvantage?

New York State already engages in all sorts of stealth inequities when it comes to funding its schools, as this report by Bruce Baker notes.* A few $75K grants are small change compared to the literally billions have that have been denied to New York's neediest school districts. Still, it's amazing that a grant program like this, which rewards schools enrolling the least needy students, exists.

Just once, I'd like the opt-out scolds to acknowledge some of this. Just once, I'd like them to point out how illogical it is for politicians to simultaneously demand that schools meet new high standards while refusing to provide the money their own laws say is necessary to properly fund education systems. Just once, I'd like them to stop worrying so much about students who opt-out of tests, and start worrying about politicians who opt-out of funding the schools that enroll the neediest students.

But that doesn't tell the story the opt-out scolds' patrons want to tell, does it?

My new favorite cartoon. via Jeff Parker.

*As always, Bruce is my advisor at Rutgers GSE. He also diid a similar analysis of New Jersey's "Priority," "Focus," and "Reward" schools back in 2012.

No, Eva, You Can't Do Whatever You Want

When Eva Moskowitz doesn't get exactly what she wants she stomps her feet, takes her ball and goes home:

Success Academy Charter Schools won’t offer pre-kindergarten classes next year after losing a high-profile fight with city and state officials.

The charter network has refused to sign the city’s pre-K contract, arguing that it includes inappropriate regulations about how charter schools manage their time and design their curriculum. But neither Mayor Bill de Blasio nor State Commissioner MaryEllen Elia has allowed Success to bend the rules, and both have insisted that Success sign the contract or lose funding. 

In recent months, Success officials have continued their fight in court. But with no resolution in sight, the city’s largest charter-school network will close its three existing pre-K programs and will not open two more planned for next school year, CEO Eva Moskowitz announced Wednesday.

“It is unbelievably sad to tell parents and teachers that the courts won’t rescue our pre-K program from the mayor’s war on Success in time to open next year,” Moskowitz said.

There are two issues here: first, does the city have the right to demand that Success sign a contract before it gets funding for its Pre-K program? Reading Elia's decision in the matter makes it crystal clear that not only did the city school district have every right to insist on a contract, it had an affirmative duty to do so:

In addition, the Department’s [NY State Education Department's] RFP required that SUFDPK [Statewide Universal Full-day Pre-kindergarten] programs operate under the jurisdiction of the local board of education, which would be responsible for the proper disbursement of and accounting for project funds. The Department’s RFP required applicants to certify that the program would be conducted in accordance with all applicable federal and State laws and regulations, application guidelines and instructions. Applicants were also required to sign a Statement of Assurances regarding the program. The Department’s RFP explained that the project period for grants in year one would be from July 1, 2014 to June 30, 2015, and that grantees would have the opportunity to renew funding on an annual basis provided that the programs meet quality standards and all applicable requirements, and subject to the annual appropriation of funds in the State budget for this purpose. [emphasis mine]

So the NYCDOE was the applicant, was responsible for disbursing the monies to the Pre-K providers -- which could include charter schools like Success -- and was ultimately responsible for making sure the program was run correctly. And it's not like Moskowitz and her team didn't know this:

On August 24, 2015, SA Harlem 1, SA Cobble Hill and SA Williamsburg began their pre-kindergarten classes. By email dated August 27, 2015, DOE congratulated Success Academy NYC on the opening of its pre- kindergarten classes but advised that, until a contract is executed, “you are operating your Pre-K classrooms at your own risk.”

I know Moskowitz doesn't think the rules should apply to her because her schools are so awesome. But what made her think she was going to get her money if she didn't follow regulations? If she thought the terms of the contract were so unreasonable, why did she apply in the first place?

Back in March, Moskowitz took to the New York tabloids, which are always happy to help spread the gospel of "choice," to argue that de Blasio's administration was overstepping its bounds by insisting on a contract:

Two years ago, New York State passed a law giving charter schools the right to operate pre-Ks. The law provided that with respect to oversight, “all such monitoring, programmatic review and operational requirements . . . shall be the responsibility of the charter [authorizer].” Thus, a charter’s pre-K program would be regulated and overseen just like its other grades.

That's not quite right. Here's the law in question; check out section 3:

3. (a) The universal  full-day  pre-kindergarten  program  shall  make
  awards  to  (i)  consolidated applications submitted by school districts
  which include pre-kindergarten programs offered by  schools,  non-profit
  organizations, community-based organizations, charter schools, libraries
  and/or  museums, which shall demonstrate geographic diversity within the
  area to be served as well as diversity of providers; and (ii) non-profit
  organizations, community-based organizations, charter schools, libraries
  and museums, which may apply individually to the extent  allowed  under
  paragraph  (b)  of  this  subdivision. Any consolidated application must
  include, but is not limited to, the names of individual  locations  and
  providers, applicable licenses, facility lease information, and intended
  staffing plans and certifications.


(b)  Prior  to  submission  of  a  consolidated  application, a school
  district shall widely solicit non-profit organizations,  community-based
  organizations, charter schools, libraries and museums located within the
  school  district  to be included in its application. The school district
  shall notify any applicant who has been  denied  for  inclusion  in  the
  consolidated  application no later than two weeks prior to submission of
  such application. Such eligible providers denied for inclusion may apply
  individually as provided in paragraph (a) of this subdivision. [emphasis mine]

I'm not a lawyer, but this is awfully clear: Success Academies didn't have to apply along with the NYCDOE. They could have gone it alone, but chose not to.

So NYC, as the lead applicant in a consolidated application, has every right to demand the other organizations that sign up play by their rules. If the other organizations don't want to go along, fine -- submit your own application. But don't whine afterward that you don't like your school district's conditions when you could have submitted your own application.

Now, I'll admit that the law isn't explicitly clear in saying that charters applying on their own are self-regulating while charters applying as part of a consolidated application are subject to regulation by the school district which leads the application. But, as Elia writes in her decision, it's the only logical conclusion any applicant could come to:

I agree with petitioners that Education Law §3602-ee(12) states that the monitoring, programmatic review and operational requirements for SUFDPK programs shall be the responsibility of the charter entity, in this case the SUNY trustees. However, for the reasons stated below, I disagree with petitioners’ interpretation of this language. Moreover, I disagree with petitioners’ argument that the effect and intent of Education Law §3602-ee(12) is to prohibit school districts such as DOE, which operate SUFDPK programs via consolidated applications that include charter schools as eligible providers, from regulating the pre-kindergarten programs being funded through the consolidated application. To take petitioners’ position to its logical conclusion would mean that DOE would be required to provide charter schools’ pre-kindergarten programs with public funding without any mechanism to ensure that the statutorily required eight quality elements

and other program requirements are being met and that such public funds are being spent in accordance with the requirements of Education Law §3602-ee, the Department’s RFP and DOE’s RFP. 

Again: if Moskowitz didn't like the terms of this deal, she was free to strike out on her own. If she doesn't like the predicament she's in, she has no one to blame but herself.

One more thing: Moskowitz claims that the monitoring of her Pre-K program should have been left to her authorizer, the State University of New York. Had she applied on her own, however, NYSED would have been well within their rights to ask whether or not SUNY actually has the capacity to properly monitor the program. You might not like NYCDOE for all sorts of reasons, but at least it has an infrastructure in place to monitor Pre-K programs; I very much doubt SUNY has anything equivalent (given what's been going on inside Success Academy lately, I think I'm justified in having my doubts).

Which brings us to the second issue: even if NYCDOE had the right to demand a contract with Success Academy, were the demands of the contract too onerous? Here's Moskowitz once again:

New York City, however, has insisted that charter schools agree to a 241-page contract that regulates every aspect of their programs: their curriculum, field trips, professional development, scheduling, discipline, playtime and use of technology. For example, it prohibits more than three field trips per year involving transportation; dictates the precise amount of playtime that must be allowed (2 hours and 7 minutes); and prohibits the use of a SmartBoard (an interactive screen that is essentially a modern blackboard) for more than a 30 minutes a week.

As Elia notes in her decision, however, every one of these restrictions is both reasonable and in accordance with the guidelines set out by NYSED's Division of Early Childhood Education. In other words: had Success struck out on its own, it probably would have had to adhere to guidelines quite similar to these.

The city has limited school resources; it's reasonable for them to put a cap on field trips that use transportation. We know that high-quality Pre-K should include significant amounts of playtime; it's reasonable for the city to set a lower limit. We know we should limit the time young children spend in front of screens; it's reasonable to set a limit on it, SmartBoard or otherwise.

And if Moskowitz doesn't agree, she shouldn't have signed up. As Elia writes:

Moreover, petitioners’ argument ignores the fact that a charter school’s participation in the SUFDPK program is voluntary. As petitioners would have it, a charter school could voluntarily apply for a grant of State funds but assert an exemption from all grant requirements based on Article 56 and still be entitled to such State funds. The Legislature could not have intended such a result, which is both irrational and contrary to public policy.

Moskowitz doesn't seem to understand this very basic idea about accountability. She seems to think that because she's drilled-and-killed her way to some decent test scores, she should be able to do whatever the hell she wants:

However, there is an important principle at stake. Not only is the city violating the pre-K law, the whole idea of charter schools is that they should be freed from the control of the district bureaucracy. If all pre-K programs, including those run by charter schools, have to subject themselves to the rules and requirements imposed by the city, then we are back to an educational monopoly.

The city says its requirements are reasonable. That’s debatable, but it’s beside the point. If the Obama administration were to seek the power to regulate the content of this newspaper, the issue wouldn’t be whether or not the regulations were reasonable, but whether the government ought to regulate the free press. In our case, while we are subject to regulation, it is regulation by a different government entity to ensure that the city doesn’t have an educational monopoly.

The city also says it has to be able to assure that the pre-K program Success offers is of high quality. However, that is the responsibility of our authorizer, the State University of New York.

Moreover, being subject to the city’s control will diminish the quality of our program, not enhance it. Success’ students pass state tests at more than two times the rate of the district schools and more than three times the rate of the district schools in the poor neighborhoods we primarily serve.

This is a ridiculous argument -- on multiple levels. First, passing noisy, questionably valid assessments doesn't automatically give you carte blanche to ignore basic rules of transparency and accountability. The taxpayers of New York have every right to expect that organizations receiving public funds follow the rules laid out by their duly elected representatives.

Next, given everything we've learned this past year about how Success actually operates, it takes a special kind of smug for Moskowitz to claim superiority over the public district schools.

Finally: has Moskowitz never heard of the First Amendment? Her comparison of the government regulating the content of a newspaper to insisting a public contractor sign a contract is so dumb that only an outlet as bad as the Daily News would put it into print.

This is actually really simple: New York City has clearly laid out regulations regarding Pre-K programs. Maybe you don't like them; fine, challenge them however you want. But you don't get to take public monies and then decide, after the fact, that you don't want to play by the rules.

I wish I could say that Moskowitz is an outlier -- but she's not. The charter sector, despite its many, many, many, many, many failings, has done a terrible job of policing itself. I have no doubt there are many decent, sincere educators operating charter schools -- but they have been largely silent on calling out bad actors within the sector, or addressing the structural deficiencies that encourage bad behavior.

Instead, it appears they are asking for more autonomy, less regulation, and unchecked growth, no matter how badly that may hurt public district schools. And when anyone dares suggest tightening up the regulation of charters, watch out: Eva Moskowitz and her fellow charter cheerleaders, like Campbell Brown, will work themselves up a froth of righteous indignation:

You might say: He was referring to your position about pre-K, which seems inflexible. Every other charter school apparently signed an agreement to follow city rules, why do you think you’re above doing that?

We have fought for 10 years to build schools free from any influence that might dilute or compromise our program for our students. Our independence from district oversight is inextricable from that program.

Why? So you can use a SmartBoard more? So you can have the kids play less? Or, as Elia's decision suggests, might there be another reason?

In response to petitioners’ complaint regarding Article 5(b)(2) of DOE’s proposed contract, which limits SUFDPK providers’ authority to “reject, suspend, expel or otherwise refuse to provide any or all of the Services to any eligible child,” DOE explains that it has a “strong interest in ensuring that disciplinary measures in the consolidated UPK program are age and developmentally appropriate and that 4 year-olds receiving services within the consolidated UPK program are not subject to suspensions or expulsions.” DOE has articulated a rationale for its requirement that is based in sound educational policy and is not inconsistent with Education Law §3602-ee. While petitioners have included such provision in their “non- exhaustive” list, they have articulated no rationale as to how such provision violates either Article 56 or Education Law §3602-ee.

We have extensive documentation of Success Academy's disciplinary practices, including suspensions for very young children. We know how Moskowitz feels about NYC's attempts to remake its disciplinary codes. Is this the "influence" she and Brown are worried about? That the city might stop Success from imposing its "no excuses" philosophy on 3-year-olds?

If that's the case, it's probably better for everyone if Moskowitz just stays out of Pre-K.

Happier times.

ADDING: More from Peter Greene.

Who's Responsible For Misusing Washington DC's Test Scores?

Recently, Matthew Chingos and Kristin Blagg of the Urban Institute published a blog post about the rise in national test scores for schools in Washington, DC. The post has been republished elsewhere and, consequently, cited frequently in the edu-bloggosphere and press.

I have four specific objections to their post, which I'll lay out below (from least to most technical). But let me start by saying I don't think Chingos or Blagg are hacks; far from it. I want to engage on what they wrote because I think it's important and I think they are serious. My tweeting yesterday could easily be interpreted as my saying otherwise, and for that, I apologize.

However...

We now live in an atmosphere where, far too often, education research is being willfully misrepresented by people with ideological, personal, or other agendas. As I've said many times here, I don't pretend that I don't have a point of view. But I also work hard to keep the evidence I cite in proper context.

Others, like Michelle Rhee and Jonathan Chait, have no such qualms. It is, therefore, far past time for public intellectuals and researchers to start asking themselves what responsibilities they bear for the willful misuse of their work. For example...

Objection #1: The framing of the research. I realize this was just a blog post, and that the standards of a peer-reviewed article don't apply here. But I also understand that Urban is an influential think tank with an extensive communications shop, so anything they put out is going to get traction. Which is why I have a problem with starting the post with this paragraph:

Student performance in the nation’s capital has increased so dramatically that it has attracted significant attention and prompted many to ask whether gentrification, rather than an improvement in school quality, is behind the higher scores. Our new analysis shows that demographic change explains some, but by no means all, of the increase in scores.

Chingos and Blagg have got two big, big assumptions laid out right at the top of their post (I'll be referring to this version from Urban Institute's website throughout; all emphases are mine):

1. Student performance in DC has "increased so dramatically."
2. The two factors that influence test scores are gentrification -- in other words, changes in the demographic composition of DC's student body -- and improvements in school quality.

The fact is, as I show below, there is at least one other factor that has contributed significantly to changes in DC test scores, and there are almost certainly others as well.

Continuing directly:

The National Assessment of Educational Progress (NAEP), known as the “nation’s report card,” tests representative samples of fourth- and eighth-grade students in mathematics and reading every two years. NAEP scores reflect not just school quality, but also the characteristics of the students taking the test. For example, the difference in scores between Massachusetts and Mississippi reflects both the impact of the state’s schools and differences in state poverty rates and other demographics. Likewise, changes in NAEP performance over time can result from changes in both school quality and student demographics. 

The question, then, is whether DC’s sizable improvement is the result of changing demographics, as some commentators claim, or improving quality. Our analysis of student-level NAEP data from DC, including students from charter and traditional public schools, compares the increase in scores from 2005 to 2013 with the increase that might have been expected based on shifts in demographic factors including race/ethnicity, gender, age, and language spoken at home. The methodology is similar to the one used in a new online tool showing state NAEP performance (the tool excludes DC because it is not a state). 

We are now three paragraphs in and the authors have repeatedly set up a particular research framework: changes in the NAEP are due either to changes in student demographics or improving quality in schools.

Now, as I said above, Chingos and Blagg are not hacks. Which is why they admirably explain, in detail, why they did not include the most important student demographic factor affecting test scores -- socio-economic status -- in their analysis.

But it's also why, in the third-to-last paragraph, they write this:

The bottom line is that gentrification alone cannot explain why student scores improved in Washington, DC, a conclusion that echoes previous analyses using publicly available data. DC education saw many changes over this period, including reform-oriented chancellors, mayoral control, and a rapidly expanding charter sector, but we cannot identify which policy changes, if any, produced these results.

Go ahead and read the entire post;  you'll find that "if any" clause is the only time they suggest there may be other factors influencing DC's NAEP scores besides student characteristics and school quality.

Am I picking nits? I suppose that's a matter of debate. But when you write a post that's framed in this way, you shouldn't be the least surprised when an education policy dilettante like Jonathan Chait picks up on your cues:

But here is an odd thing that none of these sources mention: Rhee’s policies have worked. Studies have found that Rhee’s teacher-evaluation system has indeed increased student learning. What’s more, the overall performance of D.C. public school students on the National Assessment of Education Progress (NAEP) has risen dramatically and outpaced the rest of the country. And if you suspect cheating or “teaching to the test” is the cause, bear in mind NAEP tests are not the ones used in teacher evaluations; it’s a test used to assess national trends, with no incentive to cheat. (My wife works for a D.C. charter school.)

Some critics have suggested that perhaps the changing demographics in Washington (which has grown whiter and more affluent) account for the improvement. Kristin Blagg and Matthew Chingos at the Urban Institute dig into the data, and the answer is: Nope, that’s not it. The amount of improvement that would be expected by demographic change alone (the blue bars) is exceeded by the actual improvement (the gray bars): [emphasis mine]

We'll save the claims about Rhee's innumerate teacher evaluation system for another day. Does it look like "if any" was enough to stop Chait from making a way overblown claim about Rhee's policies being the driver of DC's test score rise? Read the whole thing: does it seem to you that Chait is willing to entertain any other possibilities for DC's "dramatic" rise in NAEP scores? And are you at all surprised he used Blagg and Chingos's post the way he did, given how they wrote it?

Michelle Rhee, former superintendent of the DCPS, couldn't wait to get in front of a camera and use Blagg and Chingos's post to justify herself:

(0:20) Well, over the last decade really, the scores in DC have outpaced the rest of the nation. In fact, over the last few years, the gains that DC kids have seen have actually led the nation.

We'll also set aside Rhee's claims about "school choice" in DC for now and, instead, point out that at least she is willing to compare DC's "gains" to other systems. Her comparison is, as I'll show below, highly problematic, but at least she's willing to acknowledge that a "dramatic" gain must be viewed in relative terms. (I'm also going to stay away from her response to the allegations of cheating in DC; I'm sure others with more knowledge about this topic will weigh in.)

Rhee, however, is also not willing to acknowledge that DC's gains might be due to factors other than student characteristics or policy changes.

(2:43) And then people said, well it was because of shifts in demographics; you know, basically meaning more white kids are coming in. And this most recent study last week showed that was not the case either. And so it's high time people realized, no, actually, poor and minority kids can learn if they have great teachers in front of them every single day and if families have school choice and are allowed to choose the programs that are serving their kids best. If you put those dynamics in place, wow, kids can actually learn.

You'll notice how Rhee conflates "poor" and "minority" students, even though Blagg and Chingos did not address socio-economic status directly in their analysis.* Nowhere in this interview, however, does Rhee acknowledge there may be others factors involved in the rise of DC's NAEP scores.

Let me be clear here: Blagg and Chingos did not put words into Rhee's mouth or Chait's blog. But they know know how the debate is being conducted now; they know folks like Rhee and Chait are out there waiting to pounce on this stuff. Why, then, frame your conclusions this way? Why limit your cautions to a mere "if any" when you know for certain someone is going to take your stuff and use it for their own ends?

At some point, you have to own at least some of the consequences of putting your research into the public sphere in a particular way. At the very least, you need to point out when it's being abused. Will Blagg and Chingos do that? I hope so.

* * *

This next part is more technical. Skip down if you're not interested.

Objection #2: Where are the secular effects? Bob Somerby of The Daily Howler is usually not included in the rolls of the edu-bloggosphere, as much of his work is media criticism. But he's been writing about the abuse of NAEP scores and other education topics for a long time. Here, he addresses Chait's post about Blagg and Chingos, making a rather obvious point:

Truly, this is sad. Chait accepts the Urban Institute's unexplained demographic projection without even batting an eye. Incomparably, we decided to do something which made a bit more sense:

We decided to compare DC's score gains during that period to those recorded in other big cities. Our decision to run this simple check required almost no IQ points.

Duh. As everyone knows except New York Times readers, NAEP scores were rising all over the country during the years in question. To simplify the demographic confusion, we looked at how DC's black kids did during that period, as compared to their peers in other cities.

In what you see below, we're including every city school system which took part in the NAEP in 2005 and 2013. As you can probably see, the score gain in DCPS was remarkably average:

Growth in average scores, Grade 8 math
2005-2013, NAEP, black students only
Atlanta 19.78
Los Angeles 16.72
Boston 15.17
Chicago 14.29
Houston 13.23
DCPS 11.48
Charlotte: 7.83
San Diego: 7.39
New York City 5.82
Cleveland: 5.34
Austin 4.85

For all NAEP data, start here.

Let me take this a bit further. Here are some quick and dirty graphs using NAEP scores as taken from the Trial Urban District Assessment (TUDA), and the national NAEP. Understand, we have to approach these with great caution: I haven't done anything to adjust for student characteristics or a myriad of other factors that influence test scores. But this is good enough to make a larger point.

Here are the average scale scores for all students on the Grade 4 Reading NAEP test. No question, DC's score rose -- but so did the scores of many other cities. Even the nation as a whole rose a bit during the time from 2003 to 2013.

Economists will sometimes call this a secular effect.** The basic idea is that if you're going to look at a gain for a specific individual, you have to take into account the gains across all individuals. In other words: it's just not warranted to say a gain is "dramatic" unless you put it into some sort of context. You must ask: how do DC's gains stack up against other cities' gains? Other states?***

Maybe Blagg and Chingos don't have the data to run this analysis. OK -- then don't say the gains are "dramatic" or "sizable." But even if you do a comparative analysis and you conclude they are, you have to deal with another issue:

Objection #3: Equating gains on different parts of a scale. Again, here's Somerby:

How well did DC's eighth-graders score in math in 2013, at the end of Rhee's reign?

In theory, it's easier to produce large score gains if you're starting from a very low point. DCPS was low-scoring, even compared to other big cities, when Rhee's tenure began. [emphasis mine]

As Rhee points out, DC scores did rise more relative to many other cities. But gains at the low part of the score distribution are not necessarily equivalent to gains at the high part. It may well be easier to get gains when you're at the bottom, if only because you have nowhere to go but up.

This is one of the biggest problems with the claims of DC's "dramatic" test scores gains: it's hard to find another school district with which to compare it.  Here, for example, are the Grade 8 Reading scores over the period in question:

Atlanta, Cleveland, and DC started in 2003 at close to the same place. Cleveland finished below DC, but Atlanta finished above. Los Angeles, meanwhile, which was below all three, rocketed above DC.

Does this tell us anything about policy changes and their effects on student outcomes? Can we confidently say LA rocks? Absolutely not; we didn't adjust for any student demographic changes or other factors -- but even if we did, we couldn't be sure what was causing the changes.

The point here, however, is that even with these crude, unadjusted measures, there is far more to the story than simply stating that DC has "dramatic" and "sizable" gains. And the gains of high-flyers like Boston and Charlotte can't be equated with the gains of cities on the lower part of the scale.

Again: I didn't see any cautions in Blagg and Chingos's work about this.

Objection #4: Assuming fixed demographic effects. This is going to get a bit technical, so bear with me. Somerby complains that Blagg and Chingos's methods are "unexplained." That's not really fair: they do include a link to a NAEP analysis site from Urban that includes a Technical Appendix that spells out the basic idea behind their score adjustments:

We performed adjustments by estimating regression coefficients using the student-level data from 2003, roughly halfway between the available periods for mathematics (1996–2013) and reading (1998– 2013). Specifically, we regress the test score of each student in 2003 on the set of control variables described above.1 Control variables are included in the regression using dummy variables identifying each of the groups of students for each construct, with the exception of the one arbitrarily chosen group that is the omitted category (except for age, which is included as a continuous variable). 

Using this regression, we estimate a residual for each student across all assessment years, which is the difference between their actual score and a predicted score based on the relationship between the predictors and test scores in 2003. We calculate the adjusted state score as the sum of the mean residual and the mean score in the given test year (e.g., 1996, 1998, and so on), re-normed to the mean of the given test year. Essentially, we perform an adjustment for each testing year, but we base the adjustment on the relationship between NAEP scores and the control variables in 2003. [emphasis mine]

So here's the thing: this method is based on an assumption that the effects of student characteristics on test scores remain fixed across time. In other words: the effect of a student qualifying for free lunch (a proxy measure of SES) in 2003 is assumed to be the same for every test administration between 1996 and 2013.

But we don't know if that's true. If, for example, there is a peer effect that comes from changing cohort demographics over time, there may be a change in how a student's characteristics affect their test scores, even if the characteristics themselves don't change between individuals at different points in time.

Now, there's actually something a researcher can do about this, and Blagg and Chingos likely have the data they need. Rather than just regressing one cross section of test scores on student demographics, regress all the years you're studying but add year dummy variables and interactions with the demographic measures and years. That way, you'll see whether the effects of student characteristics change over time, and you'll get a better comparison between years.

In many cases you couldn't do this, because each interaction adds a covariate, which costs you another degree of freedom in your regression. But Blagg and Chingos are working with microdata, so their n is going to be very high. Adding those interactions shouldn't be a problem.

Of course, it's quite possible doing all this won't change much. And you'd still have the problem of equating different parts of the scale. But I think this is a more defensible technique; of course, given that we're really in the weeds here, I'm open to disagreement.

 * * *

Look, I don't want to equate my methodological qualms about Blagg and Chingos's post with the outright abuse perpetrated by Chait and Rhee. But I'm also not about to let Blagg and Chingos off the hook entirely for what Chait and Rhee did. 

The sad truth is that the willing misapplication of education research has become a defining feature of our current debate about school policy. Is my "side" guilty of this misconduct? Sure; there's stuff out there that supports my point of view that I don't think is rigorous enough to be cited. Of course, my "side" hardly has the infrastructure of the "reformers" for producing and promoting research of questionable value in service of a particular agenda. Still, that doesn't excuse it, and I'll take a deserved hit for not doing more to call it out.

But it would be nice to be joined by folks from the other "side" who feel the same way.

Let me say something else about all this: there's actually little doubt in my mind that Rhee's policies did increase test scores. How could they not? Rhee had her staff focused on test score gains pretty much to the exclusion of everything else. It's not far-fetched to imagine that concentrating on local test-taking carried over and affected outcomes on the NAEP. 

But Rhee's obsession with scores trumped any obligation she may have had to create an atmosphere of collegiality and collaboration; that's borne out by the resounding rejection of Mayor Adrian Fenty, whose reelection was largely a referendum on Rhee.

And that, perhaps, is the biggest problem with Blagg and Chingos's analysis -- our, more accurately, the entire test-based debate about education policy. I'm not saying test scores don't count for something... but they surely don't count for everything. Was it worth breaking the DC teachers union and destroying teacher morale just to get a few more points on the NAEP? Is anyone really prepared to make that argument?

One more thing: I was at a panel at the American Educational Research Association's annual conference this past spring about Washington DC's schools. The National Research Council presented a report on the state of a project to create a rigorous evaluation system for DC's schools.

It was, frankly, depressing. As this report from last year describes, the district has not built up the data capacity necessary to determine whether or not the specific policies implemented in DC have actually worked. Anthony Bryk, a highly respected researcher, was visibly angry as he described the inadequacy of the DC data, and how no one seemed to be able to do anything about improving it.**** From the report:

The District was making progress in collecting education data and making it publicly available during the time of this evaluation, but the city does not have a fully operational, comprehensive infrastructure for data that meets PERAA’s goals or its own needs as a state for purposes of education. To meet these needs, the report says D.C. should have a single online “data warehouse” accessible to educators, researchers, and the public that provides data about learning conditions and academic outcomes in both DCPS and charter schools.  Such a warehouse would allow users to examine trends over time, aggregate data about students and student groups, and coordinate data collection and analysis across agencies (education, justice, and human services).

To confront the serious and persistent disparities in learning opportunities and academic progress across student groups and wards, the city should address needs for centralized, systemwide monitoring and oversight of all public schools and students, fair distribution of educational resources across wards, ongoing assessment of strategies for improving teacher quality, and more effective collaboration among public agencies and the private sector.  In addition, the committee recommended that D.C. support ongoing, independent evaluation of its educational system, including the collection and analysis of primary data at the school level. [emphasis mine]

When you live in a data hole, this is what you get: incomplete research, easily subject to abuse. When we get down to it, we must admit that we know next to nothing about whether any of the educational experiments foisted on the children of Washington DC have done anything to help them.

Given this reality, I would respectfully suggest to anyone who wades into an analysis of Washington DC's schools that they stop and think very carefully about the implications of their research before publishing it. In the absence of good data and rigorous analysis, what you put out there matters even more than it would in places where the data are more comprehensive.

Caveat regressor.


ADDING: It's important to point out, as Bruce Baker does here, that the NAEP gains in DC largely pre-date Rhee-form.


* They did use parent education as a proxy measure of SES in their Grade 8 analyses. But that measure is a self-reported, very rough categorical measure that breaks down parental education into: didn't finish high school, finished high school, had some education after high school, finished college. You go with the data you've got, but this measure is obviously quite crude.

** I first came across the term in Murnane and Willett's Methods Matter, which I highly recommend to anyone interested in quantitative methods in the social sciences.

*** No, DC is not a state. But it does have a population larger than Vermont and Wyoming.

**** To be fair, his opinion wasn't shared by everyone on the panel. Michael Feuer of GWU stated that he thought data collection and analysis had improved, even if it wasn't where it needed to be.