NJ Teacher Evaluation: Based On a Mistruth

The new proposals for changes in teacher evaluation in New Jersey - code name: Operation Hindenberg - have been made public by Education Commissioner Chris Cerf. And you know what's really amazing?

The first sentence in Cerf's proposal - the very first sentence - is a mistruth, slickly packaged to give the NJDOE plausible deniability for misleading the public about the need for a revision in teacher evaluations.

Here it is:

In schools, teachers and leaders have the greatest influence on student learning.

Let me start by turning to the always excellent Matt DiCarlo at Shanker Blog:

Specific wordings vary, but if you follow education even casually, you hear some version of this argument with incredible frequency. In fact, most Americans are hearing it – I’d be surprised if many days pass when some approximation of it isn’t made in a newspaper, magazine, or high-traffic blog. It is the shorthand justification – the talking point, if you will – for the current efforts to base teachers’ hiring, firing, evaluation, and compensation on students’ test scores and other “performance” measures. 

Now, anyone outside of the education research/policy arena who reads the sentence above might very well walk away thinking that teachers are the silver bullet, more important than everything else, perhaps everything else combined. I cannot prove it, but I suspect that many Americans actually believe that. It is false. 

As is so often the case with this argument, the sentence is carefully worded with the qualifier “at every level of our education system” so as to be essentially in line with the research. This is critical because it signals (very poorly in this case) that teachers are the most influential schooling factor in student achievement (which the blueprint calls “success”). And, indeed, this is the current empirical consensus. It means teachers have a larger effect (far larger, actually) than principals, facilities, textbooks, class size, technology, and all other school-related factors than can be measured. 

But in the big picture, roughly 60 percent of achievement outcomes is explained by student and family background characteristics (most are unobserved, but likely pertain to income/poverty). Observable and unobservable schooling factors explain roughly 20 percent, most of this (10-15 percent) being teacher effects. The rest of the variation (about 20 percent) is unexplained (error). In other words, though precise estimates vary, the preponderance of evidence shows that achievement differences between students are overwhelmingly attributable to factors outside of schools and classrooms (see Hanushek et al. 1998; Rockoff 2003; Goldhaber et al. 1999; Rowan et al. 2002; Nye et al. 2004). [emphasis mine]

How about a picture?

Now, some of you may have noticed that I cast things a little differently than DiCarlo. The first difference is that I'm changing "teacher effects" to "teacher and classroom effects." Why? Because I don't believe (and I'm not saying Matt does, but he doesn't broach the topic here) that teacher effects in a classroom can be disentangled from peer effects. In other words:  a teacher's "effectiveness" depends, in part, on the students who are assigned to her room. More on this later when we talk about SGPs.

The other difference, however, is more important: I believe that all of these things are "in-school" factors, including the student's background.

Students are in school, right? That makes them an "in-school" factor; you can't separate their effect from the school's effect, because they are in school. So the largest "in-school factor" affecting student achievement isn't the teacher - it's the student!

Yes, I'm being a little facetious, but only to make a point. DiCarlo makes it more seriously:

So, in my humble view, those who make the teacher effects argument to non-technical audiences should consider making it more clearly. Admittedly, the blueprint’s wording is unusually fuzzy. More commonly, the argument will use the phrase “schooling factor” or something similar. This is much clearer, and it is not misleading per se, but it is still probably insufficient for many people to get the distinction (a disturbing proportion of people making the argument don’t give any qualifier at all).

But I contend that this is exactly why people like Chris Cerf make the assertion in the first place. Cerf is not a stupid man; he knows exactly what he is saying, and exactly what he means to convey to his audience. I know some people get squeamish when folks like me ascribe ulterior motives, but there comes a point where you have to be honest with yourself about what's going on:

This mistruth is part of a deliberate attempt on the part of the NJDOE to convince the public that the biggest problem with New Jersey's schools is poorly performing teachers.

Keep this in mind as we continue to explore these proposals, which include the application of an evaluation tool that cannot be used, by design, to evaluate teachers. Yes, you heard me right.

NJ's New Teacher Evaluation System: Operation Hindenberg