Wars Of Attrition

Diane Ravitch has been having a bit of a battle with the charter school cheerleaders at KIPP (a battle I've happily joined). At issue is not whether KIPP's schools are "successful"; the question is whether KIPP has found the "secret sauce" that makes their teaching methods replicable on a large scale. Paul Thomas, Bruce Baker, and Ed Fuller have much more on this.

See, it's much, much easier to teach kids when you only teach the ones who are compatible with your system. As Gerald Coles points out, there's now a great deal of evidence that KIPP does exactly this. No one should be surprised; Matt DiCarlo rightly makes the case that, in a climate of school "choice," it's inevitable that families will choose schools that are the right "fit" for their children. Why would anyone be surprised that many children aren't the right "fit" for KIPP?

The problem is that the KIPPsters seem willing to let themselves be used to make a larger argument: that poverty doesn't matter, and all students can learn. Coles points to this op-ed from Bill Carpenter:

These are “no excuses” schools which clearly show that when given the opportunity students living in poverty can achieve. While some will say that this performance is a result of “skimming” the best from this population, all objective assessments have shown this not to be the case. Rather, the differentiating characteristic tends to be a parent who is seeking an “alternative” to the existing public school option.  Given the state of inner city education, this is hardly a criterion for differentiation but rather admiration.

Well, if KIPP only admits children whose parents sign on to its system, isn't that a form of skimming? Or is Carpenter, like Joel Klein and Arne Duncan and Chris Cerf, using isolated examples of "success" to create a logical fallacy:

• There is a undeniable correlation between poverty and learning.
• But some poor children do learn. Therefore...
• There is no correlation between poverty and learning.

Ravitch addressed the issue with a simple thought exercise, posed as a challenge: KIPP should run an entire school district. If they can overcome the effects of poverty for more than just a self-selected handful of students, we could reasonably assume that poverty does not matter after all.

The fact that KIPP has backed away from Ravitch's dare should speak volumes. But too many people have too much invested in poverty-denial; they will not go away quietly.

Which brings us to the latest reformy soldier in the War Of Attrition: Dr. Daniel Musher, an apparently brilliant researcher of infectious diseases. I say "apparently" because I am in no way qualified to judge the man's medical work; I'm an educator, and I know my limitations. Dr. Musher, however, suffers from no such boundaries; he finds himself quite capable of judging Ravitch's scholarship, and even offers some of his own:

Despite her academic title, Ravitch has a long track record of selective referencing, and she tends to cite opinion pieces rather than data. Shown in the two figures and table below are results from a published study that had no corporate sponsors and that Ravitch will never cite (Musher KK, Musher DM, Graviss EA, Strudler RM. Can an Academically Intense Educational Experience for Self-selected Students Improve Academic Performance on Objective Tests? Results from One Charter School. The Educational Forum 69:352-66, 2005).
Students from the first two grades to enroll in KIPP Houston were tested at the time of enrollment (entering fifth grade), and again at the end of fifth, sixth and seventh grades. Four subtests of the Woodcock Johnson-Revised test were administerd to one group of children (upper graph) and six tests to another (lower graph). The data show that students began at or below grade level and, three years later, were at or well above grade level. The overall increase in achievement scores was about 5.3 years during three years of schooling. The differences were highly significant, as the reader can see by referring to the original paper. [emphasis mine]

Here's the paper; yes, the title directly references the fact that KIPP's students are "self-selcted." That ought to end this discussion by itself.

But I went a little deeper and quickly read through the work. Here's the comment I left at Ravitch's blog for Musher:

Dr. Musher, I looked (quickly) at your study here:

http://www.eric.ed.gov/PDFS/EJ724885.pdf

The attrition rate of the first cohort taking the WJ-R was 37% between 5th and 7th grade. The second cohort’s attrition rate was 30%.

Yet you dismiss this quite casually in the paper. As you are not an education researcher, you may not know that attrition is one of the most important issues in determining the “success” of charters like KIPP. See:

http://schoolfinance101.wordpress.com/2010/12/02/truly-uncommon-in-newark/

Dr. Ravitch has commended KIPP many times for its fine schools; that’s not the issue at hand.

The real question is whether the KIPP model is replicable on a large scale. I’m sorry, but your paper does little to shed any light on this.

I have to wonder: if Dr. Musher was doing a study on the efficacy of a new vaccine for Ebola virus, would he so casually dismiss the notion that there might be a problem with a trial that excluded nearly 40% of the treatment group in the final analysis? Wouldn't he want to know why those subjects were excluded? Would he be satisfied with the company that developed the drug simply saying, "Oh, we don't exclude anyone from our trials! See, look at all these healthy people over here! The vaccine obviously works!"

So goes our eduction debate.

ADDING: The Perimeter Primate speaks from the comments:

This effort to defend KIPP is — as my daughter might declare — an “epic fail.” What is not disclosed in Dr. Musher’s comment as well as in the 2005 article published by The Educational Forum is that coauthor Karol Musher has a long, long history with the sample school. She appears in Jay Mathews book as helping with KIPP’s Houston launch and has also been a board member for KIPP Inc. (the Houston branch) for many years. According to a 2008 Houston Chronicle profile, Mrs. Musher worked as a consultant for KIPP Academy from 1995-1997 (the first KIPP school in its Houston cluster), then served on the Houston KIPP board from 1997-2002. She also appears on KIPP Inc.’s 2005 & 2006 Form 990s as Emeritus Director, but is listed as an active director on 2008 & 2009 Form 990s (EIN 133875888; returns for 2010 & 2011 not yet publicly available). Mrs. Musher is currently listed on KIPP Houston’s website as a member of its executive board.

Oh, my.