Big Brother, Teacher Evaluator

Smile!

The meticulous scoring of videotaped lessons for this project is unfolding on a scale never undertaken in educational research, said Catherine A. McClellan, a director for theEducational Testing Service who is overseeing the process.

By next June, researchers will have about 24,000 videotaped lessons. Because some must be scored using more than one protocol, the research will eventually involve reviewing some 64,000 hours of classroom video. Early next year, Dr. McClellan expects to recruit hundreds of educators and train them to score lessons.

The goal is to help researchers look for possible correlations between certain teaching practices and high student achievement, measured by value-added scores. Thomas J. Kane, a Harvard economist who is leading the research, is scheduled to announce some preliminary results in Washington next Friday. More definitive conclusions are expected in about a year.

Notice how we just assume VAM based on test scores is an accurate measure of student achievement?

Look, I'm all for best practices. But I'm going to be shocked if this turns up anything we didn't already know. Looking at larger and larger sets of qualitative data doesn't necessarily get you to induce new patterns of behavior that can then be taught to educators.

I may be wrong, but this looks like recipe for a bunch of bromides strung together and presented as research-based teacher training.

In any case, there is plenty that can go wrong in this sort of research:

Cynthia M. Tocci, director of a research center at the Educational Testing Service, used the framework to critique a vocabulary lesson taped last month by a fifth-grade teacher in Charlotte.

Half an hour into the video, Dr. Tocci noticed that a boy with his hand up had grown impatient after the teacher failed repeatedly to call on him. Eventually the boy threw up his hands in frustration. The teacher had not noticed.

“That’s poor on respect and rapport,” Dr. Tocci said, scoring the lesson with 2 points, out of a possible 4, in that category. (Only egregious disrespect — an open exchange of classroom insults — would rate a 1 in the respect category, she said.)

It may well be as it seems on its face in this case. But those of you who are teachers probably read that and thought just what I did: that teacher may well have ignored that boy because he raises his hand inappropriately all the time and she's not about to give him the satisfaction of controlling the class or stopping other students' learning.

Every teacher has students like this: they have to give an opinion on EVERYTHING, and they want to control the class with their incessant questioning. We can have a legitimate discussion about how to deal with these students, but the fact here is that Dr. Tocci is making a judgment without all of the facts, AND she's missing something obvious a veteran teacher would almost certainly pick up on.

I have long been an advocate of peer-review: it's useful, it's non-threatening, and you're getting feedback from the folks who are down in the trenches with you. I'm not NBPTS certified, but I've known teachers from back in Florida who did the program and found it very useful. Videotapes are used as a portfolio that you get feedback on, and not as a substitute for walk-in evaluations.

I'm much less in favor of this. Nit-picking by "experts" isn't really going to do much to improve teacher quality.