Well, at least that's what Star-Ledger columnist Tom Moran says:
Tom, maybe I've been wrong about VAM - I'll be the first to admit it if I am. I'd like to evaluate the evidence, however. Maybe you can help; I don't understand this from page 10:But the teachers union, and many obedient Democrats in the Legislature, are resisting all use of test scores in teacher evaluations.Perhaps this study can help change their minds. It finds that the kids who attended classes with effective teachers were more likely to avoid teen pregnancy, attend college and to earn more money later in life.
I have two masters degrees, a reasonable amount of training and experience in research methods, and decent amateur statistics skills. With time, patience, luck, and a little help (OK, a lot), I'll be able to get through this paper eventually.
But I would never venture an opinion about whether it informs actual policy making unless and until others with far better chops than mine go through and gave it a reading. To come out and demand policy changes based on one study of one limited sample in one city is foolish beyond belief.
I blame think tanks for this. They produce "studies" with a patina of academic rigor: references in ABA style, charts, executive summaries. Gullible policy makers and journalists look at this stuff and think, "Oh, this is easy! I can get this!"
But think-tanky stuff is not serious academic research, and no one should pretend that it is. What we have here is a serious study by people with serious academic credentials; it's not a policy brief from AEI or the Gates Foundation. It should be properly vetted and debated by people with the skills to do so before anyone tries to draw conclusions from it.
I'll be reading it during the commercials of the Giants game this Sunday...
So, you're telling me that those equations can determine whether or not I'm an 'effective' teacher? Where in that equation is the work I've done with an emotionally disturbed second grader who talks incessantly about guns, gangs and pornographic websites (I am NOT kidding)? How do all those letters and numbers explain how my colleagues and I are helping him gain some semblance of normalcy for 6 hours a day in between the time he spends at home, unsupervised in front of a computer? Oh wait, they don't. That kind of thing doesn't matter to education 'reformers'.
ReplyDeleteOh, and how did they figure out that only 'poor' students struggle in school? The student I'm talking about is not poor.
Does that formula prove, statistically, the validity of the standardized tests given? In fact, where in this model is the test accountable for accurately measuring student learning? Have these tests been revised to measure teachers instead of students? Where is that statistic in this model?
ReplyDeleteBecause when the NTSB (National Transportation Safety Board)assesses the safety of cars it is totally the assembly line workers that have caused these flaws in the cars and they had better improve or they will lose their job, correct?
Because we all know the ground floor instruction on how to take these tests is what's all important. The new testing genre is the one ring to rule them all... and yes, according to Marie Clay, testing is now a genre. This is what defines us from other successful countries in education more than anything else... as they detach from testing we hammer it more firmly into place with all high stakes decisions based upon them. Again, where is that test accountability and validity to measure students AND teachers?!
Anon #1: You are, of course, absolutely right that econometrics cannot capture any of this value. Although I wouldn't dismiss it outright - I think there's a place for some quantitative analysis. But my background as a musician comes directly into play here: I know of pieces that can be "justified" through music theory - the problem is, no one likes to listen to them. Same with teaching - there are teachers who may get high VA scores, but no one would want to be in their class. So the study probably says something important, but it needs to be taken in context.
ReplyDeleteAnd I don't believe many corporate reformers or their cheerleaders in the media have the ability to do just that.
Anon #2: No, it does not. This is huge problem. I would venture to say that the authors would argue that since the test is given to everyone, and it measures SOMETHING, it is valid for this study. Maybe; I need to think and read about that some more.
ReplyDeleteBut your point about tests measuring students - not teachers - is perfectly valid. What we have here is a study that, if correct, says teachers who have high VA scores are better at teaching their kids to pass a test. And that there seems to be some - limited, but some - benefit to being able to pass a test.
But we know for a fact there are many other factors that lead to "success" in life besides a teacher's ability to get their students to fill in the right bubbles. My worry continues to be that we focus so much on the teacher that we use that as an excuse not to address the others, which are undoubtedly more important.
And not for nothing, and I know we all know this, but there is a lot more to a person's success and happiness in life than how they do on standardized tests in their youth. I was a notoriously bad tester, but good grades, and I have a PhD funded by NSF and Fulbright. Thank goodness they did not see my test scores. It breaks my heart to see us (not us here, but society us) reduce accomplishment to this limited sphere.
ReplyDeleteI do understand there is a place for the tests, but the people who advocate for them as a major tool in student and teacher evaluation have dehumanized us all so far that they clearly have different visions and goals about the outcome of successful education.
It bothers me when people cite "credentials" without any reference and claim they hold weight. Duke, what are your masters degrees IN (I won't even ask where they are from, largely irrelevant in my mind). People with masters in English Lit seem to thing they can diagnose cancer because they have a "masters"...
ReplyDeleteAnon:
ReplyDeleteIf you think I'm a hack, why do you troll here?
So a study on economics tells us the truth on nuances of teaching?
ReplyDeleteAnd we're trusting economists in this country to tell us this?
Am I the only one who notices the irony of that as we enter year 3 of the worse recession this country has seen in nearly 100 years?
Baker posted his analysis. I look forward to reading your thoughts on it.
ReplyDeleteDuke, you mention two masters degrees you have as an argument against your not comprehending the above formula. If you are going to cite that, what are your madsters degrees in? Music?
ReplyDeleteOh please. Gaining a masters degree in ANY subject requires a level of intellectual curiosity that would lend itself to deciphering a well written analysis. I have a masters in music myself, and if you passed graduate level music theory, you can parse any study. Go have a look at Schenker's method of music analysis and come back and tell me that someone with a masters in music isn't smart enough to read and understand a statistics paper.
ReplyDeleterofl.....Duke's comment was that he has two masters degrees and CAN'T understand the paper.
ReplyDeleteSometimes his cloying fans are so kneejerk their legs get out from under them.
Dude, you're getting a little obsessive about me and my background. How about we leave it at this:
ReplyDeleteI have two masters from two nationally known colleges/universities. My first is in music; my second is in education. The second required grad level courses in research methods and statistics; they were not music ed courses.
My thesis required I run several ANCOVAs - analyses of co-variance, a moderately sophisticated statistical procedure.So I'm not a total neophyte when it comes to this stuff.
I do not have the background to judge the mathematical models in this paper; I do have enough background to understand them at a little more than a superficial level. I certainly have enough training to recognize that this is a serious piece of real - not think tanky - research, and I know that there is very little chance that Tom Moran or most other op-ed writers have the background in education theory, econometrics, or statistics to judge this paper on its merits.
I also have enough knowledge to know that Bruce Baker and Matt DiCarlo CAN make a substantive assessment. I read their writing on this with great interest, and you should as well, if you are really interested in the topic.
Are you?
Cloying? Please. Stop impugning other people's intelligence to make your arguments look more convincing. Duke has two masters' degrees and finds the statistics difficult, and that means that your average journalist likely does as well.
ReplyDeleteCloying? Please. Stop impugning other people's intelligence to make your arguments look more convincing.
ReplyDeleteThere was a mistaken word in my initial post. "I have a masters in music myself, and if you passed graduate level music theory, you can parse almost any study." Duke has two masters' degrees and finds the statistics difficult, and that means that your average journalist (and politician) likely does as well. Not that it is stopping them, or you, from pushing a wrong-headed agenda.