I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Sunday, June 8, 2014

UPDATED Chartery Snake Oil: Tennessee Grade

UPDATE, 6/10/14: So Bluff City Education has responded to this post. I won't waste my time or yours arguing semantics; I'll only point out the author apparently has no argument to make against my data and/or analysis.

And I'll point out, once again, what really bugged me about Aycock's original piece:
Yes, reducing poverty is an essential piece of the puzzle in solving the achievement gap. However, schools don’t control housing, economic development, or social welfare policy. Schools only control what goes on inside the walls of the building. To argue that we can’t improve education until poverty is solved only gives a pass to schools that are failing our children. [emphasis mine]
As I say below: it'd be one thing if Aycock was just cheering on his school. But he went well beyond that: he called out other schools that are "failing." As far as I'm concerned, he asked for everything I gave him. 

If you're going to throw down a gauntlet, you'd better be ready for someone to pick it up and throw it back in your face. If you can't handle that, you'd do well to keep your mouth shut about everyone else's "failure."

* * *

You know what really bugs me these days? Especially in the world of education "reform"?


I'm not talking about pride. Pride is great (to a point). Everyone who works at a school should be proud of their students, their staff, their leaders. If you talk up the good things happening at your school -- charter or public -- that's fine with me.

And I'm not talking about confidence. Hey, if you can back up what you say, more power to you. Understand you've left yourself open for critique, but if you feel that you, as an educator, can show you are getting it done and you want to share that with the world, I say go ahead.

No, what irks me is this: reformy types who puff up themselves and their "miracle" schools to the point where they actually think they've discovered the cure for what's ailing our horrible American public schools (which, in reality, are pretty good, considering the circumstances).

Educating children -- especially children who are at-risk or have special needs -- is complex, difficult work. Most people who do this job are humbled by it; they know that there are no easy answers, and that techniques and strategies that may have worked once are never guaranteed to work again in the future. And, while we teachers have no problem with accountability, we also understand that it's really easy to tut-tut at the failings of others when your own grasp on success is so tenuous.

Not so the smug reformer. No such self-doubts plague him: he knows that his superior moral center and indomitable will are the keys to his absolute and unquestioned excellence. If only the rest of us would follow in his footsteps, we could transform the entire education system in a few short years. If only we excuse-makers would get out of his way, he could solve society's ills faster than a French economist could raise your taxes.

Often, these reformy, self-aggrandizing braggarts have credentials as phony as a carnival barker's: TFA, Broad, SUPES, TNTP, Relay, NLNS, etc. Invariably, they have some sort of link to a charter chain: KIPP, Uncommon, Success, ASD, Mastery, Academica, etc.

And, inevitably, they take to the media at some point to boast on their awesomeness:
Critics of education reform often blame poverty for poor academic achievement. In fact, many argue that you can’t fix education until you fix poverty. 
This simply is not true. 
Yes, reducing poverty is an essential piece of the puzzle in solving the achievement gap. However, schools don’t control housing, economic development, or social welfare policy. Schools only control what goes on inside the walls of the building. To argue that we can’t improve education until poverty is solved only gives a pass to schools that are failing our children. 
Schools can help reduce poverty, but only by increasing opportunity for children. 
And I’m proud to say that I’m involved in two schools doing just that. I’m involved in one as a parent and the other as a teacher.
So, even though reducing poverty is "essential" for raising educational outcomes, we don't have to actually do it. Yeah, that makes sense...

I'm going to skip over the school to which the author, James Aycock, sends his own children for two reasons. First, the rest of this post uses 2013 data from the Tennessee Department of Education, and KIPP: Memphis Collegiate Elementary doesn't report test scores; it only has a kindergarten and first grade (apparently, a school that only serves two primary grades can "prove" poverty doesn't matter - amazing, huh?). Second, it's probably better to leave Aycock's choices for his own kids out of this debate; let's concentrate on something just a little less personal.

We'll look, instead, at the vaunted school where Aycock works. What is this astonishing citadel of learning that proves that you don't have to "fix" poverty before you "fix" education?
There are several key takeaways that I have from my experiences with KIPP: Memphis Collegiate Elementary and Grizzlies Prep.
First and foremost, poverty does not have to be destiny; zip code does not determine potential. We can’t afford to use poverty as an excuse to continue the status quo. We can’t wait until poverty is solved. We have to set expectations high for all kids.
Secondly, parents should not be so afraid to educate their kids alongside kids of different socio-economic, racial, or other backgrounds. Specifically, white parents need to be okay with their kids being the minority. This is Memphis after all, the first majority African American metro area in the nation. And, since median household income in Memphis is $35,000, middle class parents need to be okay with their kids being around those with less income. [emphasis mine]
Wow - this Grizzlies Prep must be some school! Not only is it named after an NBA franchise; it proves that poverty just doesn't matter! And it teaches less enlightened parents, through example, that you can send your middle-class child to a school full of students with lower incomes without fear! Step back, haters: Grizzlies Prep is proving that it's your weak status quo excuses -- and not poverty -- that are holding our kids back! And I'm sure a look at Grizzlies, using publicly available data and standard statistical methods, will prove Aycock is correct.

Won't it?

Let's start with the obvious: poverty matters. Anyone who tells you otherwise is full of crap. Above, I've plotted every school in the Greatest State in the Land of the Free that reports Grade 3 through 8 test takers on a graph with two axes. The y-axis is the number of students at proficient or above on reading/language arts (RLA) tests. The x-axis is the number of children in the school who qualify for free lunch, a proxy measure for economic disadvantage.

What we find is the most obvious thing in the world: as poverty goes up, test scores go down. The correlation in Tennessee is so strong that over 70 percent of the variation in test scores can be explained solely by free lunch eligibility.

See that green line through the middle? That "regression line" is kinda-sorta an average: it's a prediction, based on all the points, of how many students will be proficient in a school if we know that school's free lunch eligible rate. The prediction here is very, very good, because the correlation between poverty and test scores is very, very strong.

This, by itself, is more than enough to dismiss Aycock's ridiculous claim that "poverty isn't destiny." It doesn't matter that there are outliers -- points above or below the line. The overall pattern is so compelling that the vast majority of the outliers don't vary much from the line. In Tennessee, as in every other school system in the world, poverty matters. 

But let's take a look at Grizzlies Prep anyway. Let's see if Grizzlies chows down on poverty like a bear scarfs a Sockeye.

Remember when I said the green line was a prediction? If a school is above the green line, that means it beats predicition: it over-performs where we would expect given its free lunch rate. But look at Grizzlies Prep: it's under the line. Grizzlies Prep underperforms prediction -- a very strong prediction.

I've marked out the other charters in Tennessee that reported Grade 3-8 proficiency rates in red triangles (using the TNDOE's markers for whether the school is a charter). And I've marked a few that are part of the vaunted Achievement School District, pet project of reformy blowhard and TN Commissioner of Education Kevin Huffman.

Does it look to you as if Grizzlies or ASD is proving that "poverty does not have to be destiny"? Or could it be that Tennessee's charter industry really proves exactly the opposite: all schools, charter or otherwise, follow a pattern where test-based outcomes are highly correlated to measures of economic disadvantage.

For kicks, let's look at math scores:

The correlation here is significant, but not as strong as in RLA. Grizzlies (and ASD), however, actually falls further behind prediction here: they are farther below the trend-line.

How, then, can Aycock claim with a straight face that "Grizzlies Prep is proving that you can educate both the more and the less prepared together successfully"? I suppose he might make the argument that he needs more time; that Grizzlies' kids are too far behind and need to catch up. OK... but don't go around bragging that you've proved poverty doesn't matter until you've actually done it!

Don't smugly look down your nose at "schools that are failing our children" when many of those schools, burdened with higher rates of student economic disadvantage than yours, outperform your school!

And don't attempt to take the moral high ground with middle-class parents about economic segregation without first looking in the mirror.

I don't know Memphis at all (which is a shame, because this Jersey boy loves him some barbecue). For this graph, I looked at a map, found Grizzlies Prep (zip code: 38103), and picked out a bunch of zip codes in the immediate area. I then plotted the free lunch percentage for all the equivalent schools: the ones that report valid test takers for grades 3 through 8.

I've switched over to free lunch, as opposed to free and reduced-price lunch, because central Memphis has many children in economic disadvantage, and FL is a better indicator of the variance in this disadvantage (you-know-who breaks this argument down here). The point is clearly made: Grizzlies Prep -- and, indeed, the entire Memphis charter sector -- serves fewer students in deep economic disadvantage than the rest of the city's public schools.

Let me be clear: a 65 percent free lunch rate is high, relative to the rest of the state. But it is not high relative to Memphis. I won't say this is a deliberate economic segregation; most likely it is not, at least in the sense that some students are being told outright they can't attend because the quota of poor children has been filled. It's far more likely deficits in social capital are playing a role here; however, deliberate or not, this segregation is real.

Like I said, I don't really know Memphis, but let's pull back a little: near as I can tell, these are all the schools (again, reporting valid test-takers for grades 3-8) within the "loop" of I-40, I-240, and the river. I've only marked Grizzlies Prep this time.

Still not looking like the rest of the city's schools, is it?

Let me stop here and give the caveat I always give in posts like this: I have no doubt that Grizzlies Prep is full of wonderful, deserving kids and hard-working, dedicated educators and staff. Every one of them should take pride in their school. My beef is only with Aycock -- and not with his work as an educator, of which I know nothing.

No, my problem here is that he (and the TN Charter School Center, which reprinted his piece) has polluted the conversation about America's schools and childhood poverty, and he's done so in a way that keeps us from having an honest discussion about what's really going on.

Aycock goes well beyond praising his own school; again, I really have no problem with that. But he takes it to the next level: he claims his school is better than the schools that "fail" our children and make "excuses." What Aycock fails to point out however, is that his school's outcomes are really no better than those schools he excoriates, even as his student population has significantly fewer students in economic disadvantage.

Poverty matters, James. Your own school proves this to be true, and it's not an "excuse" to say so, loudly and clearly.

It's probably a little unfair for me to come down so hard on Aycock: he's merely the latest in a long line of reformy swaggerers whose claims, upon closer inspection, almost never hold up to scrutiny.  But their voices have become so loud, so insistent, and so pervasive, that I and the few others who are attempting to stand up for real public education reform have little choice at this point:

It is time to state, in no uncertain terms, that the people who push the notion that poverty is an "excuse" for academic outcomes are little better than snake-oil salesmen. Their claims are pernicious and dangerous. They are providing an excuse for plutocrats to continue economic policies that concentrate more wealth into the hands of a few at the expense of millions of children, from both middle-class and working-poor families.

These "reformers" are the enablers of a level of inequity we have not seen for a century. And they are, through the tenets of "reform," embedding a system of social reproduction within our urban schools, all while claiming they are doing exactly the opposite. They are abetting a transition to privatized schooling that abrogates the rights of already disenfranchised families, often instilling a pedagogy of compliance that would never be tolerated in wealthier communities.

In short: the people who tell you that their miracle schools prove that "poverty is not destiny" need to stop.


Tennessee's finest grade.

H/t to the always excellent Gary Rubinstein for pointing this piece out.

Geek Notes: This is my first time working with Tennessee data; like all states, it has its quirks. You can get Excel tables with composite Grade 3 through 8 proficiency rates, or just Grade 3, or just Grade 7 (why not the others?). Proficiency is broken down to four levels, but I couldn't find scale scores. There are also tables with "3-year average NCEs" in a variety of subjects, but I'm not touching those without fully understanding the methodology behind them.

Proficiency is an incomplete measure; any test-based outcome is an incomplete measure. Again, I'm not saying Grizzlies, or any Tennessee charter, is a bad/good school: I'm saying the data doesn't back up the overblown claims Aycock makes in his piece.

I ran the regression and made the tables in Stata. I excluded any school that didn't have test takers in Grades 3 through 8 because I wanted to look at equivalent schools, but let's face it: there could be a lot of variation even within this group, and there's no guarantee we've got a clean apples-to-apples comparison here. Caveat regressor.


Meghan Vaziri said...

Thank you so much for this! I'm a Memphis teacher. Your piece is spot-on.

Michael Fiorillo said...

A wonderful post, but you're far too kind: these people suffer not just from smugness, but from insufferable arrogance and mendacity.

Duke said...

Thx, Meghan. You good folks have put up with too much crap for far too long down there. TVAAS is awful - hopefully, more on that later this summer.

Mike, why are you always goading me on, bro? ;-)

Michael Fiorillo said...

Duke, I'm here to make everyone else seem calm and reasonable by comparison...

jcg said...

Jazz, Thanks for featuring one cave in the Hellhole that is TN education. We have Kevin Huffman (Arne's BFF), Haslam, a tea party lege, TFA & Broadies infesting tevery section of the state and it's going to take some serious work to rid ourselves of them. It may not happen in my lifetime. Tennesseans, particularly teachers, are soooo very complacent.
No one cares about Memphis in East TN and many of the teachers I am working with in KNoxville can't see that the Memphis experiment is our future.
Hang in there & ya'll come visit, ya hear?

jcg said...

BTW, Huffman screwed up royally when he failed to get the TCAP scores to the schools before TN schools were out for the summer. Talk about FUBAR. One teacher called it a HUBAR- HUff'd up beyond all repair.

MommaBears was all over it here:http://www.mommabears.org/blog/call-to-action-blitz-gov-haslam

There's talk that he's out a here. Wonder if Duncan will find him a nice patronage position? Mr Rhee, like his ex, depends on the kindness of the 1% that comes with a lifetime immunity to accountability.

PhantomMinuet said...

HUBAR was mine. :-)

The Memphis experiment is a disaster. Not only do you have outside groups, like KIPP, coming in and attempting to skim what little cream there is off the top, you have local leaders and politicians who see charter schools as a way to pad their resumes and/or their pockets. And it will get worse before, and if, it ever gets better.

Duke said...

jcg, my sister's in Asheville, NC, and it's been too long since I've been down to see her. So maybe I'll visit, then swing around to your neck of the woods, provided I can get a sip of that famous TN nectar of the gods whilst I visit... ;-)

It astonishes me that Duncan continues to hold TN up as a model while the state treats its teachers like dirt. This started, as I understand it, pre-Huffman, although he's made things far, far worse.

TN's educational descent is a long story that has not been told properly. Who will tell the tale?

Phantom, claim HUBAR proudly! Hopefully, this Yankee will get down there one of these days and we can visit.

Jon said...

You can find my response to your piece here http://wp.me/p3JZlh-lg

bob jones said...

Anither great job, Jazzman!

bob jones said...

@ Jon

Clearly, Grizzly does not indicate any real numerical data to suggest their "strategic thinking and hard work by a dedicated faculty with the goal of continuous improvement" is any better than any of the other schools in it's category. Further, viewing the raw data, one could see that Grizzly is under-performing and thus not providing the adequate educational opportunities that other schools with higher percentages of disadvantaged students are. You need time to see improvements? so do all schools who educate thoe at risk. Just a question, what percentage of special needs students attend the Grizzly school?