Clapping Harder For the Merit Pay Fairy

UPDATE BELOW

Earlier this week, I wrote about the death of the Merit Pay Fairy in Newark, New Jersey.

Hey, Jazzman, you bum -- I ain't dead yet!

Back in 2012, Newark began an experiment in teacher merit pay, fueled by funds from Facebook's Mark Zuckerberg. Teachers were promised up to $20 million over three years in extra incentive pay -- but in the first year, only $1.4 million was disbursed, and most of that appears to have comes from other teachers, who had their pay docked because they were deemed "ineffective."

Merit pay, in other words, was little more than a broken promise to the teachers of Newark right from the start. A survey of Newark teachers in the first year found a large majority did not see the compensation system as "reasonable, fair, and appropriate." (p. 24) It's not a surprise, therefore, that this past month both the teachers union in Newark, the NTU, and the district's administration decided that the program was not worth continuing. 

But some reformy folks believe in merit pay the same way some children believe in fairies: they don't want to acknowledge the evidence that shows, even in the most generous reading, that the benefits of merit pay are very small and likely are not indicative of true increases in student learning. Like Peter Pan, these true believers hope against hope that fairies can be brought back to life simply by clapping harder:

In 2012, Newark Public Schools did something remarkable. The district reached an agreement with the Newark Teachers Union that would fundamentally shift how teachers are not just evaluated but paid. 

Then-Gov. Chris Christie and Randi Weingarten, the President of the American Federation of Teachers, announced the groundbreaking deal together on national television. At the time, my organization, the National Council on Teacher Quality, called the contract “a model to which other districts should aspire.”

That's from Kate Walsh, president of NCTQ, an organization that has previously held up Newark Public Schools (NPS) as an exemplar of teacher evaluation, claiming the district was "getting results" from its system:

The district gave the evaluation system a chance to work. While Newark saw students’ achievement decline initially after the new evaluation system was implemented, the district persevered and student achievement rose to the level it had been before and, in English, exceeded previous levels. (p.12)

It's hard to be more vague than that -- how was student achievement measured? What was the improvement? Most importantly: how do we know if the teacher evaluation system was affecting results?

As Bruce Baker and I pointed out in our review of NPS "reforms," plenty of other districts with similar demographics were showing similar growth in student achievement, without things like merit pay. In addition, Newark has not seen the same demographic shifts over time many comparison districts have.

You must account for this stuff if you're going to make a causal claim about merit pay in Newark. Alas, Walsh still seems wholly uninterested in digging into these details; for example:

The district would also start to use their dollars in the same way that other employers do. Pay would become a strategic tool to attract the best teachers to where they were most needed. Teachers who were ineffective would no longer receive an annual raise. Teachers who were rated as highly effective earned a healthy $5,000 bonus. Even better, high performing teachers who were either able to teach subjects that were hard for the district to staff, particularly in the lowest performing schools, could earn even more, up to $12,500 a year.

In the first year of the contract, Newark had about 3,200 teachers. How many qualified for the highest bonus, $12,500? Only eleven. Is Walsh really trying to make the case this small disbursal made a significant difference in teacher quality in Newark? She continues:

The results spoke for themselves. After five years of implementation, 96 percent of highly effective teachers chose to stay in Newark and 49 percent of ineffective teachers were voluntarily leaving the district—exactly the sort of pattern schools need to see but rarely do. Accordingly, the district has higher student enrollment now than at any other time in recent history, suggesting parents gained a renewed confidence in the district.

First of all, we have no way of knowing whether these teacher attrition and retention rates are significantly better than they would be in the absence of the merit pay scheme. We don't know how they compare to similar districts' rates. We don't even know how they compare to rates before merit pay in Newark. Again, it's completely unwarranted to make any sort of causal claim without at least some attempt to compare these rates to a counterfactual.

Second, the notion that there's any evidence that shows student enrollment has increased because of "renewed confidence" due to merit pay is absurd on its face: "I was going to move my family, but now that Newark has teacher merit pay, we're staying!" Maybe the city's child population is simply increasing. Is Walsh so enamored with merit pay she's willing to make wild stretches like this?

Apparently, she is:

Where once compensation was used as a strategic deployment of resources to ensure the district can fill its vacancies, keep its best teachers, and ensure the most vulnerable students have access to them, soon there will be nothing but raises based on years of experience, and requiring teachers to spend precious time and money earning another degree they more than likely do not need. 

Research shows over and over again that advanced degrees do not make teachers more effective with the exception of math and science. 

First of all, there is a large body of evidence that shows teacher experience correlates with effectiveness. And while gains in effectiveness are strongest in the first few years, gains do persist up through the third decade of a teacher's career. Tying compensation to experience is hardly a policy without evidence to support it.

Next: "the exception of math and science" is a very big exception. Is Walsh prepared to offer a bonus only to math and science teachers with advanced degrees? And is she really sure French teachers don't benefit from degrees in French, or music teachers don't benefit from degrees in conducting, or that teachers in many other subjects don't benefit from gaining expertise through earning an advanced degree in that subject?

I've been looking at the research on advanced degrees and teaching for some time now, and the conclusion I've come to is that it is highly limited. Most studies don't account for alignment of teaching subject and degree concentration; in other words, the results are likely skewed because they don't separate getting a degree in what you teach from getting any degree.

These studies also usually don't account for variations in the quality of the degree-granting programs: crappy on-line programs are lumped in with rigorous degrees from research universities. In addition, the student outcomes are almost always measured by test scores, which limits the teachers studied to tested grades (3-8) and only two subjects (math and English).

Walsh's sweeping statement is simply not justified. Further, she ignores the reality that NPS must compete with other districts that offer masters pay bumps to attract qualified teaching candidates. Is the district suppose to ignore this reality? Especially because there is no evidence whatsoever that NPS has attracted better candidates to its teaching staff than other districts?

As I said in the last post: there was supposed to be an ongoing study of merit pay in NPS. But that study ended after a single year. We have no evidence whatsoever that Newark attracted better teaching candidates, improved student outcomes, or raised teacher effectiveness by using merit pay.

But this lack of evidence isn't stopping Walsh from clapping harder:

In 2012, Newark Public Schools took bold steps to create a compensation system that would help to attract and keep the best teachers. The district used resources strategically to ensure the most vulnerable students had access to the best teachers, an accomplishment that many districts struggle to achieve. With this new contract, instead of being a leader in strategic compensation, Newark becomes a district that takes a one-size-fits-all approach to its teachers, to the detriment of its students and teachers alike.

Even by current reformy standards, this statement is way over the top. We have no evidence the best teachers in Newark went to the neediest students. We have no evidence Newark was better at teacher allocation than districts that didn't implement merit pay. We have no evidence Newark is now a "one-size-fits-all" district. We have no evidence merit pay was a benefit to Newark's teachers and students.

What we do know is that the majority of Newark teachers didn't think the system was fair. That, by itself, is enough to declare Newark's merit pay experiment dead -- even if some folks keep clapping for it.



UPDATE: After I posted this blog, I went back and looked at the teacher survey again, which is part of a report commissioned by the American Institutes for Research. AIR is an excellent research organization, and they produce high-quality work. That said, there are a few oddities in their Newark report:

In response to a set of questions about their knowledge of the current evaluation process, 83 percent of teachers and 99 percent of school leaders reported that they have a clear understanding of the evaluation process. In addition, in response to a set of questions about the fairness of the evaluation process, 72 percent of teachers and 92 percent of school leaders reported that the evaluation process is fair, which is larger than the 30 percent reported fairness by teachers in an evaluation of 25 districts in New Jersey (Firestone, Nordin, Shcherbakov, Kirova, & Blitz, 2014) and the 39 percent reported fairness by teachers in 10 districts in Arizona (Ruffini, Makkonen, Tejwani, & Diaz, 2014). [p.20]

Let's set aside the Arizona survey, which I haven't yet read, and just focus on the New Jersey one. That report, which I'm well-acquainted with as it came out of Rutgers (where I got my PhD and current teach part-time), did not have a sample of representative districts. It was, instead, an evaluation of a pilot program of teacher evaluation conducted in 25 districts across the state. It had a low response rate (39 percent), but more important, it was conducting a survey after the state had imposed a new law on districts, TEACHNJ, forcing them to rework their evaluation systems.

I can tell you as a teacher who lived through that time: TEACHNJ was not popular with many working teachers in the state. So it shouldn't be surprising the popularity of the new system was so low. Unlike Newark in its first year, there wasn't a whole bunch of money promised from an outside source going to these districts.

My point here is that the comparison is, at best, strained. And AIR really should have spelled out more clearly the limits of the comparison of the two reports.

Here's another oddity from the AIR report:

Figure 2 shows that the retention rates among teachers rated “effective” and “highly effective” exceed 90 percent, whereas retention rates among teachers rated “partially effective” and “ineffective” are 72 percent and 63 percent, respectively. In contrast, the most recent results from the national 2012–13 Teacher Follow-Up Survey indicate that 84 percent of public school teachers are retained, on average (Goldring, Taie, & Riddles, 2014).

Golding et al. does find 84.3 percent of teachers stayed in their positions -- but that's all teachers, not "effective" ones. There's simply no way to know, based on this report, whether Newark did any better in retaining its better teachers thanks to merit pay.

Again, it's fine to include this data point, but a little more context is probably in order.

Teacher Pensions: The Stupid Keeps On Comin'

Make it stop! MAKE IT STOP!!!

As the Philadelphia School District scrambles to come up with enough cash to open for students in September, a new report suggests the city’s school funding crisis is on par with municipal issues in Detroit, Chicago and elsewhere. 

At the heart of it all: pensions. 

Payments to retired teachers and public employees are a growing threat to government budgets everywhere, and it is no different in Philadelphia. A new report from the Thomas Fordham Institute, a conservative education nonprofit, estimates the district’s total retirement costs will balloon from $73 million in 2011 to $349 million by 2020. On a per-pupil basis, that works out to $900 per pupil in the district for 2011, growing to $2,300 per pupil by 2020. [emphasis mine]

This is like the perfect reformy storm: the Philadelphia Inquirer, which, under their new owner, George Norcross, must always find a way to blame teachers for school funding woes, even when it's not their fault; Fordham, the font of all reformimess; and the Franklin Center, one of the top propaganda outfits funded by the rabidly right-wing Koch brothers. You couldn't find a trio with better anti-union, anti-teacher bona fides anywhere on the planet.

Let's start by making something crystal clear: the fiscal problems of the Philadelphia schools were created because Pennsylvania has one of the least fair school funding systems in the nation. I imagine the rest of Fordham's staff is afraid to look at Bruce Baker's posts on this (seeing as how Baker smacks around Andy Smarick so badly that the wonk must be lying on the floor in the fetal position). But they and this op-ed's writer, Eric Boehm, had best get themselves up to speed:

For the past few decades, Philly public schools have been systematically financial deprived as a function of one the nation’s most inequitable state school finance systems. To review:

1. Pennsylvania has among the least equitable state school finance systems in the country, and Philly bears the brunt of that system.
2. Pennsylvania’s school finance system is actually designed in ways that divert needed funding away from higher need districts like Philadelphia.
3. And Pennsylvania’s school finance system has created numerous perverse incentives regarding charter school funding, also to Philly’s disadvantage. (see here also)

The Philly area remains among the most racially and economically segregated areas in the nation.

Let's me add a few more relevant facts:

In terms of salaries, Philadelphia teachers, on average, earn 15 to 20 percent less than teachers in surrounding counties. (Although starting salaries are comparable, salaries for experienced teachers top out at a much lower level.) If the salary reductions being proposed are put into effect, then Philadelphia teachers' pay will be even less competitive with their suburban counterparts.

Then consider that working conditions in Philadelphia schools are more difficult than those in the suburbs. Class size is larger; there are fewer books, materials, and supplies; computer access is scarcer; instructional, social, and health supports are insufficient. And these working conditions will be significantly worse if other proposed reductions go into effect, including the elimination of counselors, nurses, assistant principals, and sports programs. [emphasis mine]

So Pennsylvania has a school funding system designed to screw cities like Philadelphia. And Philly has seen its revenues drained from the public school system by rampant, unjustified privatization. Which has forced the city to pay its teachers less than the surrounding suburbs, even though teaching in Philly is more challenging.

Here's one of the ironies in all this: Fordham is the progenitor of the National Council on Teacher Quality, which states that it exists to advocate "... for reforms in a broad range of teacher policies at the federal, state and local levels in order to increase the number of effective teachers." How will slashing the compensation for Philadelphia's teachers relative to surrounding districts help "increase the number of effective teachers" in the city?

It's always amused me to watch "conservative" organizations like Fordham and Franklin and NCTQ twist themselves into incoherent knots over labor market effects when it comes to teachers. These stalwart defenders of free-market principles when it comes to school "choice" never care to acknowledge that maybe the free market comes into play when addressing teacher quality. They pretend that districts can simply cut compensation and gut workplace protections and make teaching less attractive (by, among other things, increasing class size and tying compensation to testing) and fire "bad" teachers... and yet, still, crowds of highly-qualified, well-trained, better-performing candidates will still magically line up to take jobs teaching in America's most benighted cities!

What are these people smoking? Do they really think that Economics 101 does not apply to the teacher labor supply? That the evil unions have so distorted the market that poor urban schools will attract better teachers when they renege on their pension promises?

I actually don't think they believe this fairy tale at all -- and you can find evidence in the language used in a report like Fordham's The Big Squeeze: Retirement Costs And School District Budgets. In the report, pensions aren't cut; they're "reformed." Pensions aren't compensation; they're "promises." Slashing pension payouts and retiree health care isn't reneging; it's "flexibility." This is an Orwellian warping of language in an attempt to deny the truth:

Pensions are not really "benefits"; they are deferred compensation. And they can actually save the taxpayers money if they are managed correctly. I've quoted David Cay Johnston on this many times before:

The fact is that all of the money going into these plans belongs to the workers because it is part of the compensation of the state workers. The fact is that the state workers negotiate their total compensation, which they then divvy up between cash wages, paid vacations, health insurance and, yes, pensions. Since the Wisconsin government workers collectively bargained for their compensation, all of the compensation they have bargained for is part of their pay and thus only the workers contribute to the pension plan. This is an indisputable fact. 

Not every news report gets it wrong, but the narrative of the journalistic herd has now been set and is slowly hardening into a concrete falsehood that will distort public understanding of the issue for years to come unless journalists en masse correct their mistakes. From the Associated Press and The New York Times to Wisconsin's biggest newspaper, and every broadcast report I have heard, reporters again and again and again have written as fact what is nonsense. 

Compared to tax, this economic issue that reporters have been mishandling is simple. But if journalists cannot grasp the economics of this issue, then how can we hope to have an intelligent debate about tax policy?  [emphasis mine]

So you can't "reform" pensions without cutting teacher compensation, and you can't cut that compensation without affecting the teacher labor market. But Fordham and Franklin and all the other reformy right groups out there don't want to make this connection. Want to know why?

This is from the good folks at the Institute for Taxation and Economic Policy. In addition to having one of the least-fair school financing systems, Pennsylvania also has extremely regressive taxes.

I can't say for sure that raising taxes on the wealthy would completely solve the underfunding problem -- but it sure would help. Of course, making the funding problem out to be insurmountable is what the reformy right is paid to do: that way, they can make the argument that raising taxes on the wealthy just won't matter very much, because we are in such a big hole.

However, as both Paul Krugman and Dean Baker have argued, the pension crisis is manageable, if state and local governments start making their contributions (something they have required from teachers, by the way -- our mandatory contributions are the only thing keeping the system afloat). At least some of the money is there; all the governments have to do is start collecting taxes from the wealthy at a rate equivalent to what they collect from the poor.

But this, of course, will never be allowed. The end game here has always been to defund and eventually destroy the public education system. If neglecting pension obligations helps move school systems toward collapse and screws unionized workers out of the compensation they earned...

Well, that's a twofer!

Tom says: "Screw union retirees AND destroy Philly's schools? What a deal!"

ADDING: Am I the only one who thinks it is more than a little deceptive to state pension liabilities in terms of past, current, and future "costs per pupil"? Today's pension costs aren't only for current pupils: they are for every previous pupil who was taught in the system by a retired teacher. I understand paying off the liabilities comes from the current budget, but this reflects a generational shift: if the bill had been paid previously, costs wouldn't be rising as fast.

In other words: that growth to a high cost in the future wouldn't be nearly as great if the district and the state paid in more money now. So making a comparison between past, present, and future costs per pupil is not particularly germane to the discussion.

I'm no expert on this stuff, so someone please tell me what I'm missing here.

ADDING MORE: Oh, my:

About 40,000 public school teachers are supplementing their salaries with a night gig likely too scandalous for your average PTA. The job title? Sugar baby.

That's according to dating site SeekingArrangement.com, the go-to Internet marketplace for "mutually-beneficial relationships" almost exclusively between older men and younger women. The site's "successful and generous" sugar daddies provide pre-determined allowances to "attractive, ambitious and goal oriented" sugar babies, in exchange for their young, nubile companionship.

The website recently announced it noticed an "influx of teachers signing up last month, prior to heading back to school."

[...]

The top school district where teachers are seeking out the sugar baby night shift? Philadelphia. (Incidentally, that school district announced 646 teacher layoffs this past June alone, according to the New York Times. Now, the district is pushing its teachers' union to accept $133 million dollars in cuts.)

Along with Philadelphia, other popular school districts for teacher/sugar babies include Miami-Dade, Los Angeles and New York.[emphasis mine]

Talk about a "defined contribution"...

Define "Charity"

This time of year affords me the luxury of taking a few days to just read, write, and think. And it's at times like this that I can pull a few threads together to make a larger tapestry (yeah, the summer seems to be straining my metaphors a bit...).

Yesterday, I detailed the rapper Pitbull's foray into the wacky world of for-profit charter schooling. One thing that struck me, as I watched him make his little pre-scripted appearances in the media, is how badly the corporatized media wants us to believe that the rich and famous are doing good through their "charitable" works. Katie Couric, for instance, gushed over "Mr. 305" for "giving back to the community" by opening a charter school.

Well, how exactly is he "giving back"? His charter school, SLAM, will receive taxpayers funds for its operation. A hefty portion of those funds will go to a company called Academica, which has a documented history of enriching itself at the public trough by controlling charter school real estate, which is exempt from property taxes, while collecting rental fees from the charters it manages.

Maybe Pitbull is actually donating some money for the start-up or operations; I wouldn't know, because it appears no one in the press has bothered to ask - that includes "journalists" like Katie Couric. But if SLAM follows the pattern of other Academica schools, someone is going to make money off of this thing.

Does this sound like "charity" to you?

Today, Pitbull gave a speech at the 2013 National Charter Schools Conference, extolling the virtues of "choice" in education. At the same conference, the Walton Family Foundation is scheduled to be inducted into the National Charter Schools Hall of Fame:

Walton Family Foundation

The Arkansas-based Walton Family Foundation has provided an unprecedented level of financial support to schools and education organizations across the country over the past decade. Founded and run by the family of billionaire businessman Sam Walton, the foundation supports a wide range of causes but education organizations are its top funding priority and received over $158 million in grants in 2012 alone.
The foundation’s core strategy is “to infuse competitive pressure into America’s K-12 education system by increasing the quantity and quality of school choices available to parents, especially in low-income communities.” To do so, it spreads its education funding across three distinct initiatives: shaping public policy, creating quality schools, and improving existing schools. Charter schools have especially benefitted from the second initiative: to date, the Walton Family Foundation has invested over $300 million in start-up schools and is now the largest single funder of new charters. Additionally, the foundation has funded state charter organizations, local charter networks, national advocacy groups, teacher training programs, and research initiatives.[emphasis mine]

OK, look: maybe you're for charter schools, and maybe you aren't. Personally, I'd feel a lot better about their proliferation if the charter cheerleaders were at least somewhat honest about how they get their "successes." Regardless, I've always felt that there could be a place for charters if they are tightly regulated and if the communities they serve agree to have them.

But that's my opinion. You can agree or disagree, and that's fine; let's have the debate. But when did we decide that spreading the Walton family's opinion about charters was a "charitable" act? 

If the Waltons want to fund credentialed scholars to engage in high-quality, peer-reviewed research in public policy, I can see that as charitable giving. But giving money to think tanks and advocacy groups - like NAPCS - isn't funding policy development; it's promoting an agenda. It's really no different from giving money to a political candidate because he or she will vote the way you want on issues - and, yes, that includes issues that may not affect you directly. It really doesn't matter much why the Waltons want more charters; it's enough that they do, and that they are willing to spend gobs of money to get them.

Does this sound like "charity" to you?

Bill Gates has been pouring millions of dollars into educational "research." One of the groups he funds is the National Council on Teacher Quality, which, amazingly, put out a report recently that shores up his belief that something is very wrong with the way we train teachers. It doesn't much matter that just about every prominent education policy scholar has ripped this piece of hack-junk to shreds, including:

• Jack Hassard*
• Bruce Baker
• Linda Darling-Hammond
• Richard Allington
• Aaron Pallas
• Jon Eckert
• Sherman Dorn
• Michael Feuer
• Diane Ravitch
• Ed Fuller

I could go on, but I can already hear the objection that all of these people have a vested interest in keeping the "status quo" in college-based teacher preparation programs. Well, that's a really stupid point, but if you want to make it, OK. But how in the world does spending millions of dollars to prop up an organization that produces universally poorly-received "research" like this qualify as "philanthropy"?

Does this sound like "charity" to you?

So here's why I bring this up: Lauryn Hill, the singer, apparently didn't pay her taxes and is going to jail. She also recently put out music some find homophobic, which stirred up a bit of controversy. In the wake of all this, she went on to Tumblr (huh, I guess people do use it...) and posted a long essay about racism and her sentencing.

I'm sympathetic to some of her thoughts; less so to others (I guess I'm old-fahsioned, but it seems to me that everybody's got to pay their taxes). But this part of Hill's post really struck me:

The prosecutor, who was a woman, made a statement during sentencing about me not doing any charity work for a number of years during my ‘exile.’ A) Charity work is not a requirement, but something done because someone wants to. I was clearly doing charitable works way before other people were even thinking about it. And B) Even the judge had to comment that she, meaning I, was both having and raising children during this period. As if that was not challenging enough to do. She sounded like the echo of the grotesque slave master, who expected women to give birth while in the field, scoop the Baby up, and then continue to work. Disgusting.

Yeah, OK, that last part's a bit over the top for me - your mileage may vary. But I think Hill's got a serious point here: since when did "charity" work become a "Get Out of Jail Free" card (either actual or metaphorical) for the rich and famous?

We see this all the time: athletes, pop stars, actors, business titans - all happily smiling in front of the camera as they go about their "charitable" works. Except we rarely, if ever, get a true look at what exactly those "works" are. Yes, we get some nice press releases and photo-ops that, as Hill points out, can be quite valuable when the star is in trouble with the law or building his brand. But is what they are doing properly defined as "charity"?

When Mark Zuckerberg fell victim to an embarrassing movie, he rushed on to Oprah Winfrey's show and declared that his new "charity" would be education "reform" in Newark.

It turns out his COO at Facebook, Sheryl Sandberg, was intimately involved in managing the optics of the giving - most likely because she knew an IPO was in Facebook's future, and she wanted lots of happy-happy press about Zuckerberg before the offering. But what happened to the money? Turns out a good deal of it was used to settle a contract with Newark's teachers that introduced merit pay for the first time into this high-performing state with strong teacher unions.

Maybe you think merit pay is a good idea; I certainly don't, but that's because I read. Whatever: no matter how you come down on the issue, how can anyone seriously suggest that pushing teachers into this direction was an act of "charity"? That Zuckerberg should be lauded for putting up the dough for the deal - especially when there's no reason to believe he will do so again in the future - strikes me as more than a little bizarre.

I understand that there's a continuum here, and that many celebrities and magnates have done good work with their fame and fortune. I'm sure one of you will send me something to burst my bubble, but I always thought Paul Newman was a good guy and doing the right thing (and I like the salad dressing). And there's nothing wrong with these people advocating for their causes: I freely admit Ted Nugent has the right to advocate for whatever crazy idea pops into his head. It's a free country, he earned his money, and he can say whatever he wants. God Bless America.

But let's not pretend that a lot of what the wealthy try to sell to us as "charity" is just that. Pitbull, the Waltons, Bill Gates, Mark Zuckerberg, and many others are pushing their points of view on to the public stage, gaming the political system to create policies that match their ideological proclivities, and setting up non-profits that are little more than shells that funnel public money to for-profit corporations.

Does this sound like "charity" to you?

Yes!

* This is actually a critique of a 2012 report on teacher preparation put out by NCTQ. I regret the error.

Born to Teach?

Via Twitter, Leonie Haimson points us to a post by Gene Glass about every refomyist's favorite academic mercenary, Eric Hanushek. This piece mostly concentrates on Hanushek's absurd contention that money doesn't matter in determining school success (I'll leave it to Bruce Baker to explain that it most certainly does). That's important; but what really intrigued me about Glass's piece was this:

Hanushek is nothing if not a believer in the unconditional truth emanating from his regression equations. But of course, those equations have not always been as clear cut in their implications as some might believe. In 1997, Hanushek published an article in which he argued that a summary of dozens and dozens of correlation studies proved that teacher experience is unrelated to their students’ achievement—the financial implications being obvious. He presented the following summary of studies that investigated the relationship (in terms of regression coefficients) between student achievement and their teachers’ “years of experience.” 

 

Although a statistically significant regression coefficient for “teacher experience” was six times more likely to be positive than negative, Hanushek nonetheless read the results as negative for the effects of teacher experience on achievement. “Importantly, … 71% [of the regression coefficients] still indicate worsening performance with experience or less confidence in any positive effect,” he wrote. 

The logic of this conclusion is illusive—no; it makes no sense. Of results that reach statistical significance, 85% (60/70) are positive; students of more experienced teachers achieve at higher levels. Of the statistically non-significant results that can be determined, 55% are positive, but fail to reach conventional levels of significance. Hanushek creates an impression of no effect of teacher experience by lumping together the category (1) “indicative of worsening performance or less confidence of beneficial performance” all significant but negative coefficients (5%), (2) all nonsignificant coefficients whether positive or negative (30% + 24%) and, (3) remarkably, the 12% of the coefficients that were so incompletely reported that it could not be determined whether they were positive or negative. The treatment of these data is hardly even handed. By such logic, ten “positive studies,” “no negative studies” and 100 studies so poorly reported that the results could not be discerned would lead Hanushek to a conclusion of no confidence in a positive result. My reading of these results is much different from Hanushek's. Regression studies have generally shown a positive relationship between teacher experience and student achievement. Period. [emphasis mine]

This is, of course, one of the reformies' favorite arguments: after the first few years, experience doesn't make much difference. It's the basis of coming up with illogical and unworkable schemes like NCTQ's teacher pay "restructuring." The problem is that it is a house built on statistical sand.

But it also brings me to a larger point: if Hanushek and NCTQ and all the other reformyists think experience doesn't make a great teacher, what does? Bill Gates says it's not advanced degrees. TFA is predicated on the idea that it's not extensive training. What's left?

Talent, I guess. Which never changes with experience, training, or education. Either you're a good teacher after a couple of years, or you're not, and nothing can change that.

Folks, does that make any sense to you?

I'm not saying that talent doesn't matter. I'm not saying that some people won't show a natural ability at teaching, just like any other field. But it's absurd on its face to think that experience doesn't matter. It's silly to think good education and good training won't make anyone better at what they do.

The notion of an innate super-teacher, however, fits nicely with the corporate CEO/hedge fund manager worldview. They see themselves as extraordinary individuals, imbued with special powers. Their mirrors reflect the image of Nietzschean supermen who deserve their obscene wealth due to their inherent superiority.  No wonder they believe a great teacher is simply imbued with top-rate pedagogical abilities. It's a Ayn Randian fantasy: Howard Roark as Mr. Chips. And it plays directly to their own vanities.

The problem, of course, is that it has nothing - nothing - to do with reality.

ADDING: Via Miss Katie, this was always the plan:

As one of its deficit-reduction strategies, the District 65 administration plans to replace all teachers who retire or leave the District with teachers who have three years of experience and a lower salary level. Before that proposal becomes policy, some Board members would like to know its potential impact on student learning. 

In a proposal to the District 65 School Board discussed at the May 29 Board Policy Committee meeting, Human Resources Director Beth Sagett-Flores said, "There is a large body of research that has examined the relationship between teacher experience and teacher effectiveness. There is evidence that teachers in their earliest years of experience have a significant impact upon student achievement." 

The proposal also stated that this research, "like much of the research in the field of education, does not lend itself to deriving definitive conclusions." Because of that she said, the District is implementing a "rigorous screening process to review each application, using decision-making criteria and standards consistent with best practices. …" to ensure that the District hires high-quality teachers. [emphasis mine]

We're data-driven! Except when we're not. Or something. It's confusing...

Someone send Ms. Sagett-Flores this post, would ya? Before her bizarre dismissal of the value of experience destroys Evanston's schools.

High-Performing Students? Meh...

So says the Gates-funded, ultra-reformy National Council on Teacher Quality:

New Jersey’s report card from a group that seeks to improve standards for the nation’s teachers is dismal: D-plus, 36th in the U.S. and making less progress than most states.

The report, scheduled to be published today by the National Council on Teacher Quality, could bolster parts of Gov. Chris Christie’s education overhaul agenda — though his critics say it shouldn’t.

The analysis considers how teachers are trained, evaluated, rewarded and fired.

But it does not assess the overall state of teaching and learning. That’s an area where, on average, New Jersey is among the highest-performing states — despite being home to low-performing schools, particularly in its most impoverished cities like Camden.

Some of the areas Christie wants to fix are the same ones the Washington-based research and policy group says are broken.

“What the governor has proposed with evaluation and tenure would put New Jersey among the trailblazer states,” said Sandi Jacobs, vice president of the teacher quality organization.

New Jersey’s grade barely budged from the “D” it received from the council two years ago.

Florida, a state where standardized test scores are far short of New Jersey’s, received the highest mark this year — and it got just a “B.” [emphasis mine]

Wait a minute: New Jersey has HIGHER student scores than Florida, but gets a worse grade from NCTQ? Are we sure Florida isn't doing a better job at teaching kids?

Yep:

New Jersey beats Florida in every NAEP assessment listed on the state comparison tool. And New Jersey beats the national average in all of its assessments; Florida beats the national average in only five.

How is this possible? Because - as both Susan Ohanian and I have repeatedly pointed out - these think tanky rankings have nothing to do with student achievement. They are based solely on whether the state in question is doing whatever the think tank making the rankings wants the state to do. But here's the problem...

Florida is doing a good job following NCTQ's prescriptions, and New Jersey is doing a poor job. But New Jersey's students do better than Flroida's. Logic suggests only two possibilities:

1) NCTQ's policies have no effect on student achievement.
2) NCTQ's policies have a negative effect on student achievement.

It's got to be one or the other, folks. But in either case, if you want a state's kids to do well on the NAEP, ignore NCTQ.

ADDING: Trolls, please don't bring up the ridiculous argument that if the NAEP is good for comparing states, standardized tests must be good for identifying good teachers. Just because screwdrivers are great for driving screws doesn't mean they're also good at beating eggs.

ADDING MORE: Even the wingers funded by the Koch brothers can't deny that New Jersey's kids kick some serious academic ass:

New Jersey moved from 10th place to third in student performance based on the 2011 scores for low-income students on the National Assessment of Educational Progress, or NAEP, given to fourth- and eighth-graders. Massachusetts and Vermont were the top two states based on student performance, and both also scored poorly on teacher quality.

Maybe "teacher quality" means something different when you're living in a think-tanky, hermetically sealed bubble. And dig this:

New Jersey gets an A for home school regulations, because the state has none. Parents who choose to teach their children at home are not required to follow any state guidelines or curriculum.

Oy. These people get to spew this stuff all over the press; me, I can't get a simple reply from Tom Moran...

Who Gets a "Superstar" Teacher?

The NY Times writes about merit pay in post-Rhee Washington, D.C.:

“The most important role for incentives is in shaping who enters the teaching profession and who stays,” said Eric A. Hanushek, a professor of economics at the Hoover Institution at Stanford University. “Washington’s incentive system will attract talented teachers, and it’ll help keep the best ones.” 

Under the system, known as Impact Plus, teachers rated “highly effective” earn bonuses ranging from $2,400 to $25,000. Teachers who get that rating two years in a row are eligible for a large permanent pay increase to make their salary equivalent to that of a colleague with five more years of experience and a more advanced degree.

Those rewards come with risk: to receive the bonuses and raises, teachers must sign away some job security provisions outlined in their union contract. About 20 percent of the teachers eligible for the raises this year and 30 percent of those eligible for bonuses turned them down rather than give up those protections.

Two thoughts:

1) If Hanushek is right, and better teachers come into the profession, won't more and more of them earn bonuses? Won't that increase the teacher payroll for Washington? How will they pay for that?

(NCTQ has said that they'd rather pay good teachers more up front and keep salaries stable throughout a teacher's career.  Well, if the lifetime earnings of a teacher remain the same, where's the incentive for "better" candidates to come into the field? You get what you pay for, folks.)

2) How will Washington decide the class lists and building assignments next year for these "superstar" teachers? How would you feel if your kid had the new TFAer - whose training is a 5-week summer seminar - while down the hall your neighbor's kid has last year's big bonus winner?

I've said this many times: the people who push this stuff just don't think it all the way through.

"Who Cares If Good Teachers Are Fired?"

Stop your whining, already:

Jacobs listed what the NCTQ considers the key "early lessons" for states embarking on these sorts of policy changes. Among other things, she said:

• "Teacher effectiveness measures don't have to be perfect to be useful. Some people are concerned that not every i is dotted and they're rolling out these systems. But keep in mind how unsatisfactory the [previous] systems have been." She added that an evaluation system "doesn't have to grind to a halt to be fine-tuned." (See my recent story on Tennessee's teacher-evaluation system, which is causing an uproar among teachers because it is far from fine-tuned.)

So what if we fire Mr. Chips and Mr. Holland but keep Cameron Diaz? Their lives are just an "i" to be dotted, right?

Hey, all you bright and talent young people! Doesn't this sound like a GREAT career choice? The pay may stink and you won't have good benefits anymore, but at least you can be fired arbitrarily based on admittedly flawed evaluation systems!

Don't all push to get in...

B4K Decides To Engage Me

I must admit, I was quite surprised to see B4K take issue with anything I write on this blog: I thought any mention of me was banned over there (again, I think we're just skipping right past the "then they laugh at you..." part).

I want to apologize to Mike Lilley, who wrote the post: I would have responded earlier, but I really didn't know you had written about me. I haven't had a chance to check B4K's site lately, as I am a working teacher and this is the beginning of the year. Well, better late than never:

You can use B4K’s “facts” – which, as you note, are unembellished and unspun – but please don’t presume to speak for us.   As the person who built B4K’s “facts” section, I take issue with a number of your points:

I would never presume to speak for you: I will, however, quote you all day long. And I always provide links so people can check things out for themselves.

1)      Questioning the independence of the sources.  Are they not “independent” simply because they don’t agree with you?  Are you suggesting that Editorial Projects in Education (EPE), the publishers ofEducationWeek, aren’t independent?  Are you suggesting that the Center for American Progress (CAP) isn’t independent?  I note you still manage to use one of CAP’s ratings when it suits your agenda.  I went out and tried to gather every rating of New Jersey’s education system I could find, and I made sure that they were all from reputable sources, which they are.  Rather than engage the facts, you seek to denigrate the sources wholesale.  

Mike, I don't know if you're new to the education policy world, but those of us old dogs who've been around the track a few times have a very different perspective on this. CAP, NCTQ, and Fordham have an agenda: everyone knows it, it's not a surprise to say it, and any assessment they make of New Jersey's schools is going to be colored by it.

The studies that you cite primarily measure whether a state is following the policy prescriptions each organization favors. That's fine, but it doesn't say much about student achievement, teacher effectiveness, or whether the policies they advocate actually work.

So, no, I don't find this data to be "independent" - more accurately, I don't find it objective. This is not peer-reviewed, academic research that is subject to the oversight that you find in serious journals and publications. It is think-tanky policy work that, again, is designed to promote an agenda. And I'm not alone here: serious researchers do not consider these objective studies.

So, no, I'm not "suggesting" CAP isn't independent - I'm stating it clearly and upfront. They have an agenda and they should not be presented as independent or objective, any more than I or B4K or the NEA should. They have every right to promote their agenda, but let's be honest about what this stuff really is.

2)      Selectively picking facts I: You cherry-pick the one “A” grade CAP gives NJ while ignoring the one “B,” one “C” and five “Ds” we received.   If CAP ratings merit mention for the one “A,” why not for the 5 “Ds”?  Perhaps because it suits your agenda?  And of course if you took the time to read CAP’s report, you would know that NJ got its “A” for removing ineffective teachers because only 38% of principals said personnel policies were a barrier (10% below the national average of 48%). But you would also see that 70% of NJ principals say that teachers unions are a barrier.  I don’t presume to question CAP’s methodology, but taking their data at face value, there really is a “problem” when it comes to removing ineffective teachers: the teachers union.    

Mike, what I did - and readers, check it out for yourselves - was point out that even in these clearly subjective assessments, New Jersey didn't do poorly on all of them. You're the one who pointed out that "A" Mike, not me. If you don't think New Jersey really didn't deserve it, why did you publish it?

Don't expect me to make your case for you, Mike - I linked to your work and did not mischaracterize what you wrote. Do, however, expect me to point out that your case is pretty weak - especially when some of the subjective indicators you use contradict one of your organization's core messages.

Further: of course teachers unions are a barrier to removing teachers: that's their job! The question is not whether the unions are barriers - it's whether the barriers are so strong that large numbers of ineffective teachers are not being dismissed. As I have pointed out repeatedly, ineffective teachers are removed from schools at a far greater rate than B4K's ads suggest.

I keep asking - begging - for some serious research about how many "bad" teachers are actually in the schools due to unionization or tenure. Post some, Mike, please. You guys keep saying this is such a huge problem; well, it's up to you to prove that it is. But understand that asking principals whether the union makes it more difficult to dismiss a teacher is a completely different point, and not the data we need to see if what B4K is calling for will actually make a real difference in our schools.

3)      Selectively picking facts II:  You cite EPE’s ranking of NJ as 2^nd in the nation in terms of student achievement on the NAEP tests, which is indeed true.   You neglect to mention NJ’s overall ranking by EPE – for all six of its rating categories, not just one as 7^th in the nation.  Not bad, actually, except when you consider that we are 2^nd in the nation in spending per student at $17,000, a full 70% above the national average of $10,000.   Moreover, if NAEP is the “gold standard” as you say, I urge you and your readers to go to our “fact” section entitled “Performance on National Tests.” There you will see how NJ students actually did on ALL the NAEP tests: in eight out of nine test groups, fewer than half of NJ students were proficient.  For example, in 12^th grade math, only 31% of NJ students were proficient: 69% were not.  Regardless of how this ranks against other states, is this sufficient?  

Mike, let's get you up to speed, shall we?

- As I pointed out in my original post, almost every item here does NOT measure student achievement. For an organization that focuses so much on teacher quality, you seem to give great weight to things in the EPE report that have NOTHING to do with teachers, their effectiveness, or actual student achievement: financing, standards, transitions to employment, etc.

Notice I said this in my original post. Perhaps if B4K focused more of its energies on curriculum and fiscal policy, rather than unionization, you could affect some changes in these areas and improve education. But you folks don't seem much interested in that.

- The NAEP is the gold standard in educational research to discern trends, but you cannot use it to judge things like "proficiency" against other measures; in other words, don't mistake 31% proficiency on the NAEP with 31% proficiency on the NJASK, or any of a number of assessments. 31% proficiency on the NAEP does not mean 69% of kids can't do math.

From Diane Ravitch's Death and Life of the Great American School System (which, Mike, you really should read):

The term "proficiency" - which is the goal of the law - is not the same as "minimal literacy." The term "proficiency" has been used since the early 1990s by the federal testing program, the National Assessment of Education Progress, where it connotes a very high level of academic achievement. (p. 102)

So let's not imply a universally accepted criterion-based standard where none exists.

- I am not an expert on school finance; luckily for us both, we happen to have a nationally renowned one right here in New Jersey! And he has a blog! Start here for Dr. Baker's post on state spending comparisons, and understand any comparison you will make with other states must take into account the relative differences in wages and cost-of-living, as well as the commitment of the state to bring resources to the neediest children. I'd also recommend you read this post by Dr. Baker. Maybe we're spending more than other states, but you really didn't think that 70% more money (NOT regionally adjusted) would lead to 70% better performance on tests, did you?

In any case, since you like state ranking so much, how about this? New Jersey gets two "As" for funding  fairness. Maybe that's where the money is going. Is that a bad thing? I thought you folks wanted ALL kids to succeed: "zip code is not destiny," yes?

Lilley continues:

I urge you to drop the cant and spin and ad hominem attacks and embrace the facts.  The stakes are too high.

I urge your readers to see the facts – unembellished and unspun for themselves on our website:www.b4njkids.org.   Then they can make up their own minds.

Respectfully,

Mike Lilley – B4K

Mike, I'm all about the facts. But you guys really have got some nerve talking about ad hominem attacks when you go around bad-mouthing my union. Want me to back off? You first.

Oh, look, a postscript:

One last related point: JerseyJazzman criticizes the independent ratings because they do “NOT measure student achievement“ and cites the NAEP tests as “the ‘gold standard’ of education research.”  Do we then take it that he agrees with B4K that student achievement and standardized tests are useful measures of educational attainment? Should we then also assume he supports the use of student achievement as measured by standardized tests as part of a new teacher evaluation system?  And thus would be comfortable basing tenure and other personnel policies based on such evaluations?

Do I agree that standardized tests are useful measures? Of course.

Would I be comfortable using them to evaluate teachers? No way in hell.

Mike, the NAEP is NOT used to evaluate teachers; it is a research tool. No one pays a penalty when students do poorly on the NAEP (yet), which is one of the reasons it's so effective for formulating educational policy.

The NJASK is currently a valuable tool for gathering information on a student's progress, a school's outcomes, and a district's curriculum. It also - and I am the very first to say this - has value in helping a teacher to reflect on his or her practice. No one thinks we should stop administering standardized tests (although we should stop trying to do so on the cheap and open up the tests to scrutiny after every administration).

What we have to do is stop MISUSING them and putting faith in evaluation systems that everyone - even the VAM cheerleaders - knows contain high rates of error. It is foolish beyond belief to use instruments to do a task that they were never designed to do. I honestly do not understand where this passion for judging teachers by how well a kid fills in bubble sheets comes from.

I will say one last thing, and I am sincere here: I'm glad you're reading the blog. You should also check out Ed Reform 101 if you haven't already. You folks need to hear from your critics - particularly those of us who are on the job -  and you need to start responding. I'm happy to engage you or Derrell any time, but understand - I am a teacher with no wealthy backers, no fundraisers, no network, no clout, and little extra time. This isn't my full-time gig. So you may have to wait a bit.

But when I do respond, don't expect me to pull punches. This is my life's work; mine and over 3 million others. We're the ones who made the commitment to actually do the job.

Keep that in mind next time you blog.

ADDING: Here's a terrific piece by Linda Darling-Hammond with more on why using test scores won't work to evaluate teachers, and what will work.