I have had several Twitter back-and-forths with Mehlhorn, and I've found them extremely unsatisfying. To be clear, that's not his fault, nor is it mine: it's the inherent limitations of 140 characters that have kept us from having a substantive debate.
The funny thing is that I enjoy our exchanges. I think Mehlhorn is a sincere advocate for policies he believes will genuinely help America's students. I also believe, however, that he's wrong about nearly everything when it comes to education -- particularly when it comes to charter schools.
At Dmitri's suggestion, we are going to have a dialog about charters here on my blog. I promised him that I would let his words stand here free of any editing on my part [I have added a few links in the text, but that's all], and that I would make my opposing case in separate posts.
I don't know how long our exchange will go, but I will respond to what Mehlhorn wrote below in a couple of days. I hope he'll reply back; this dialog about charter schools could be very helpful in clarifying one of the key issues in education "reform."
Thank you, Dmitri, for engaging in this conversation. I encourage all of our readers to weigh in through comments.
Thank you, Dmitri, for engaging in this conversation. I encourage all of our readers to weigh in through comments.
Mark Weber
aka Jersey Jazzman
ADDING: Here is an index for the entire exchange:
Part IV: My second reply.
Part V: Mehlhorn's second response.
Part VI: My final reply.
Since the Enlightenment, humanity has increasingly valued science, evidence, and the inherent nobility of individuals. Progress has been inconsistent and incomplete, but over the past two centuries this focus has brought us new political movements, new policy ideas, new businesses, and new technologies. As a result, we have less poverty, warfare, torture, and drudgery, along with longer lives and greater literacy.
Over the past few decades, this focus on evidence and human potential has come to social science research in education. Since the 1966 Coleman Report, the United States government has gathered and published extensive data on how different inputs (including money, teacher credentials, and student socioeconomic backgrounds) influence outcomes (including in-school literacy and numeracy as measured by tests, graduation rates, lifetime incomes, and adult crime rates). The Gates Foundation, which primarily funds global health initiatives such as malaria prevention, has spent hundreds of millions of dollars to evaluate teaching and schools. Interested parties, including charter school networks and the teachers’ unions, have commissioned additional research.
These studies, collectively, are still imperfect. They tend not to capture important elements of American education, including ethics, tolerance, and community-building. Despite good-faith efforts at transparency and objectivity, all such studies may be accused of bias. Nonetheless, all of this research represents an increasingly systematic effort to improve how we engage and empower the rising generation. Education policy debates, at their best, holistically consider the results, methodology, and motives behind such studies to determine the best course of action for the future.
With that as context, I am grateful to my fellow Democrats and education advocates at Jersey Jazzman for entertaining a conversation about what the current evidence states about charter schools in this country. Although they sharply disagree with me about education policy, and although education debates have become acrimonious, their commitment to children is clear from their willingness to have this open exchange with me. The long-term evidence is still thin, since charters have only been a meaningful part of the nation’s education landscape, but we have enough evidence to know a few things.
ADDING: Here is an index for the entire exchange:
Part I: Mehlhorn's opener.
Part II: My reply.
Part V: Mehlhorn's second response.
Part VI: My final reply.
* * *
Research about charter schools
Since the Enlightenment, humanity has increasingly valued science, evidence, and the inherent nobility of individuals. Progress has been inconsistent and incomplete, but over the past two centuries this focus has brought us new political movements, new policy ideas, new businesses, and new technologies. As a result, we have less poverty, warfare, torture, and drudgery, along with longer lives and greater literacy.
Over the past few decades, this focus on evidence and human potential has come to social science research in education. Since the 1966 Coleman Report, the United States government has gathered and published extensive data on how different inputs (including money, teacher credentials, and student socioeconomic backgrounds) influence outcomes (including in-school literacy and numeracy as measured by tests, graduation rates, lifetime incomes, and adult crime rates). The Gates Foundation, which primarily funds global health initiatives such as malaria prevention, has spent hundreds of millions of dollars to evaluate teaching and schools. Interested parties, including charter school networks and the teachers’ unions, have commissioned additional research.
These studies, collectively, are still imperfect. They tend not to capture important elements of American education, including ethics, tolerance, and community-building. Despite good-faith efforts at transparency and objectivity, all such studies may be accused of bias. Nonetheless, all of this research represents an increasingly systematic effort to improve how we engage and empower the rising generation. Education policy debates, at their best, holistically consider the results, methodology, and motives behind such studies to determine the best course of action for the future.
With that as context, I am grateful to my fellow Democrats and education advocates at Jersey Jazzman for entertaining a conversation about what the current evidence states about charter schools in this country. Although they sharply disagree with me about education policy, and although education debates have become acrimonious, their commitment to children is clear from their willingness to have this open exchange with me. The long-term evidence is still thin, since charters have only been a meaningful part of the nation’s education landscape, but we have enough evidence to know a few things.
The starting point for aggregate charter school research is the Stanford University Center forResearch on Education Outcomes (CREDO), which has conducted numerous studies of the charter sector. Their methodology uses a “virtual pairing” that compares charter students with comparable students at traditional public schools. Their methodology also relies on standardized tests to evaluate literacy and numeracy, and makes assumptions to translate those tests into imputed days of learning. These methodologies have weaknesses, but a when a 2009 CREDO report concluded that charters in the aggregate did no better than traditional public schools, many reform critics praised their work and methodology.
In 2013, CREDO did another national study, and found that the intervening four years had changed the sector. Low-performing charters had closed, and high-performing charters had expanded. As a result, the 2013 study showed that students at charter schools now outperformed students at traditional public schools. The overall performance difference was modest, but the improvement was significant, as part of the original theory of charter schools was that the sector would be able to improve more quickly than traditional schools. Also notable were charter demographics; overall, the sector served a population overall less wealthy and white than the traditional sector, which was important for the equity-based arguments of charter advocates.
In 2015, CREDO did a deep-dive analysis of charter schools in 41 urban areas. Education reform advocates argue that urban parents often lack the political and social capital that suburban parents use to navigate local school bureaucracies, so this urban-specific research was important. The 2015 urban study showed that on average, students gained 40 additional days of learning per year in math, and 28 additional days of learning per year in reading. Assuming a 180-day school year, those gains would be the equivalent of extending the school year by 22% in math and 16% in reading.
If this result holds up under scrutiny, it would mean that charter schools already out-perform traditional public schools in ways that are greater than the impacts of smaller class sizes, intensive preschool interventions, and other popular union-backed education investments. Additionally, the CREDO results suggest that charter schools are getting steadily better over time, so that this performance gap will only continue to grow. To be sure, there are many legitimate criticisms of the CREDO study, but the power of the 2015 results puts the burden of proof squarely on those who would attack the efficacy of charter schools.
Turning to those criticisms, the most frequent is that CREDO is affiliated with the conservative Hoover Institution, which has received funding from conservative-leaning philanthropists such as the Walton Family Foundation. On the other hand, almost all of the published criticisms of CREDO have come authors affiliated with the National Education Policy Center, which is funded by the Great Lakes Center, which in turn is funded by the teachers’ unions. Since many charter teachers are not unionized, union leaders have a direct financial incentive to attack charter research. The Walton Family Foundation, by contrast, gets its funding from the anti-union Waltons, but the Waltons have no direct economic contact with the teachers’ unions. Allegations of bias alone, therefore, are not sufficient to dismiss the CREDO results.
There are, however, several much better substantive criticisms of CREDO.
The first strong argument is that charter schools might still be skimming the best students. This is because CREDO’s “virtual pair” methodology is imperfect: although it captures initial test performance, it does not capture degrees of poverty or degrees of special needs. In other words, it is possible that CREDO shows superior performance for charters because they serve slightly less poor students (students who get their lunches at reduced price, rather than who get their lunches for free), and slightly easier special needs students (mild SPED rather than severe SPED). They might serve these slightly less hard-to-teach students by cherry-picking students (since charter schools admit students by lottery, the mechanism for this would be that they somehow encourage “easier” students to enroll in the lottery), or by selectively pushing out students (some charter schools have come under criticism for having unusually high suspension rates). The CREDO studies do not directly address these claims, because their data sets do not differentiate free vs. reduced-price lunches, or severe from mild SPED.
Substantial evidence disputes this “skimming effect” argument. Mathematica research has studied KIPP, Achievement First, and Uncommon Schools, all of which overwhelmingly serve black and Hispanic students, and has concluded that those charter schools have superior retention rates to comparable traditional public schools. Several scholars have compared students who won lottery slots to charter schools, vs. those who enrolled in lotteries but did not gain slots. These studies control for selection bias, and reveal strong performance for charter school chains such as Achievement First. Of course, these high-performing charter organizations are not the only charter organizations, but the point of the CREDO 2013 study is that better charters are expanding while weaker charters are shrinking. Increasingly, the better charters are the norm, and that’s already true in cities and states that have competent charter authorizers. For example, the University of Michigan’s Sue Dynarski published a 2013 NBER paper showing that charter schools in Boston delivered significantly better results in college enrollment and placement in ways that were highly unlikely to be from skimming.
As their final trump card, charter critics pivot to the broader claim that these studies do not generally assess students who are “left behind” in traditional public schools. According to this argument, even if skimming does not account for the gains charters deliver, skimming might still leave traditional public schools in poor urban areas with a high average concentration of difficult students. In addition, because schools have fixed costs, an outflow of students to charters can leave traditional schools with budgetary problems as they adapt their fixed costs to new enrollment levels.
In considering students left behind in traditional public schools, three conclusions appear inescapable.
First, the evidence for the claim is speculative as best. Even the strongest charter critics, at places like the union-funded NEPC, are only able to conclude that the evidence is “mixed.” Of course individual schools face budgetary struggles, but charters have become a scapegoat for those challenges. My mom’s school district was known as Richmond Unified, until it went bankrupt and had to be renamed West Contra Costa Unified under pressure from the City of Richmond that didn’t want to the negative brand association. This all happened without any charters or choice in the area. Today, only 6% of students in the country attend charters, due largely to caps. We simply do not have any evidence that charters hurt traditional schools.
Second, the evidence that we do have suggests the opposite – namely, that charters improve system-wide performance. A 2008 study in the Journal of Urban Economics showed that charter schools in Texas actually stimulate performance improvements in traditional schools, through a combination of competitive effects and demonstration effects. The most powerful study to this effect comes from New Orleans, where the entire school district was converted to a charter system in the aftermath of Hurricane Katrina. In 2015, researcher Doug Harris compared New Orleans with Baton Rouge, another hurricane-impacted city that did not shift to charters. The entire district of New Orleans saw dramatic performance improvements under a system-wide charter regime, a result that would be impossible if the criticism were correct.
Finally, and most importantly, charter critics are on untenable moral ground when they pivot from individual benefits for students to alleged system-wide effects of the schools that are “left behind.” Prosperous Americans already have choice; they buy homes in areas near schools they like, and they use their social and political capital to guide their students through those schools. We do not know exactly how many poor families of color are on charter wait lists, because some families apply more than once, but we know that over one million applications are on wait lists, and we know that at least tens of thousands of families want choice in how public money gets spent on behalf of their children. The argument to cap charters, despite these wait lists, amounts to public compulsion that these parents remain in schools that are not right for their children. Knowingly sending a child to a bad school is morally bankrupt; forcing parents to send their kids to such schools on pain of jail time is even worse. Charter critics can decry residential segregation and ask for all schools to become good, but until that happens, it is simply immoral to cap charter schools and thus deny poor parents the same choice that wealthy parents already have.
