I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Saturday, October 12, 2013

Stupid SGO Tricks

I notice that the NJDOE has slapped up a "Top 5 Frequently Asked Questions" about Student Growth Objectives on their teacher evaluation website. SGOs are a mandatory part of AchieveNJ -- known around here as Operation Hindenburg -- that will count for 15% of a teacher's evaluation rating.

As I mentioned before: there is no research base to back up the use of SGOs in evaluating teachers. And it's rather obvious, looking through this "Top 5," that the NJDOE never bothered to field test the concept; otherwise, they wouldn't be making statements like this:

Q: How can teachers who instruct in semester blocks or nine-week cycles set SGOs?
A: These teachers should set SGOs as early in the semester as possible. If the instructional period is less than nine weeks (e.g. 30-day cycles), teachers should set goals for several of these short cycles and then aggregate performance on these goals into their SGOs when possible.
But what if the content changes each cycle? And what if the "growth" for each cycle is not comparable to the others? Why didn't you folks think about this stuff before you rolled this out? If you had field tested this, maybe you would have had time to come up with answers before teachers all over New Jersey were being held responsible.
Q: How should SGOs be handled for a teacher on a leave of absence or otherwise away from teaching for a period of time?
A: Teachers should be present for at least a nine-week continuous period of time during the year to set an SGO. A teacher who has not set an SGO before the November 15 deadline due to an extended leave should set an SGO as early as possible after his or her return to the classroom.
Well, that will be fun to think about during a family medical or maternity leave. And, of course, there is no research base to back up the nine-week minimal time frame; like everything else here, NJDOE is going with its gut.
Q: I am a school counselor. Do I need to set SGOs? Do you have exemplars for SGOs for that I could take a look at?
A: SGOs are not required for school counselors, but only for teachers with an assigned class roster. Educational specialists such as academic coaches, CST professionals, librarians/media specialists, paraprofessionals, athletic trainers, health workers and counselors, etc. may set SGOs at the discretion of local district leadership. However, the Department encourages all educators to set SGOs to help improve their performance by goal setting.  Office of Evaluation staff members have worked with several professional organizations that represent these educational specialists to develop SGO samples that may be useful reference documents. Please contact your professional organization for these samples.

So here we go: just like test-based evaluations, some educators have to deal with this system, and some don't. And administrators and educators now have to contact the staff at NJDOE (think they be able to get someone with actual experience in schools on the line?) and spend their time finding out how to make an SGO for the athletic trainer or librarian, despite the fact that there is no evidence SGOs will improve those educators' performance or lead to a better evaluation.

Q: I teach Special Education. How will my SGOs be different than a general education teacher's?
A: Like a general education teacher, your SGO should be tailored to meet the needs of your students. As much as possible, your SGOs should encompass the curriculum and students for which you are responsible. The IEPs of the students you teach contain added guidance to help you provide the type of instructional support that will help your students succeed. You may use this guidance to help inform your SGOs. For example, you may set an SGO that includes standards for reading comprehension. The IEP of a student may include strategies to improve the comprehension skills that the individual student has not yet learned. In this case, the IEP and SGO for that student are aligned.
Let's disabuse ourselves once and for all of this notion: SGOs are not at all about "meeting the needs of students." They are being used for teacher evaluation, not student evaluation. And I find it more than a little ethically dangerous to set an SGO using an IEP (Individualized Education Program: the formal plan all students classified as needing special education services must have).

Why? Because that SpecEd teacher now has an incentive to have her students show "growth," whether those students actually "grow" or not. If the IEP is not working and the child does not reach his or her goals, the teacher will pay a penalty for honestly saying so. No rational, ethical teacher evaluation system would ever force a teacher to choose between an honest assessment of a SpecEd student  -- or, for that matter, any student -- and their own effectiveness rating, especially since teachers do not have sole control over IEPs or curriculum.

But this is exactly what Operation Hindenburg requires of teachers. Which leads us to...
Q: Are teachers required to use a pre-assessment?
A: No. Pre-assessments may be used to collect baseline data, but there are several other ways to determine students' starting points as mentioned in the preceding question. Additionally, unless they are carefully constructed, pre-assessments may not provide the type of information that will be useful for setting appropriate goals. For instance, a pre-assessment that is identical to a post-assessment may be so challenging for students that most score very poorly and some become discouraged by their test performance early in the school year. Additionally, a pre-assessment on content in a subject area to which students have never been exposed (e.g. German 1) may provide little useful information. At the beginning of the school year, it may be challenging to motivate students to do their best on a pre-assessment. However, some teachers are using carefully crafted and rigorous pre-assessments effectively and have worked to overcome some of the problems associated with pre-assessments noted above. In consultation with their supervisors, these teachers may opt to continue using such pre-assessments.
Of course it's dumb to test kids on things they haven't yet learned -- but this is happening right now all over New Jersey. In addition to my own work and the younger Jazzboy's own school, I get lots of updates from teachers all over the state (thanks for that, folks - keep it coming!). Because the guidance has been so last-minute from NJDOE, and because curricular directors at the district level are swamped with implementing a system that was never field tested, teachers rushed into doing pre-assessments that were inappropriate for the simple reason that the material on the test had never been taught.

The NJDOE might fool themselves into thinking these pre-assessments were "carefully crafted and rigorous," but there's little reason to believe that: they haven't done a program evaluation, so how would they know?

Let's lay our manipulatives on the table, shall we? The only reason teachers across New Jersey are being forced to do SGOs this year is because Education Commissioner Chris Cerf thinks their implementation will make the use of NJASK test scores easier to swallow.

Only teachers of math and language arts in the 4th through 8th grades will be subjected to evaluation through standardized tests via Student Growth Percentiles (SGPs). Leave aside the obvious truth that SGPs are wholly inappropriate for use in rating teachers; what really gets under these teachers' skins is that they have to worry about NJASK scores while their colleagues don't.

What better way to make SGPs more palatable than to subject every teacher to a quantitative system like SGOs -- a system that has even less research to back it up than SGPs?

Again: this is not about the students. It is about satisfying the ideological predilections of a group of people running the NJDOE who clearly have no education, experience, or training in teacher evaluation. They are creating a disaster that is wholly avoidable simply because they can.

AchieveNJ, aka Operation Hinderburg


Teacher Mom said...

Yes, it would appear that every teacher in NJ is pulling their hair out trying to figure out something that as of yet has no real meaning and nobody can give us guidance on how to do it. Let alone the fact that we know they are complete BS.

Our district was TOLD what to use as a SGO and they chose the DORA reading assessment. Even I as a special education teacher will be using this assessment. Nobody can tell me if I need to individual SGOs for each students based on personal growth or average growth of all of my students or the average growth of grade level groups. Nobody knows anything!!ANd I'm not a freaking statistician!!

Teacher Mom said...

Sorry for the typos

Bruce Taterka said...

You ask, "Why didn't you folks think about this stuff before you rolled this out?"

Here's the thing: they DID think of all these problems, but went ahead and made this mess anyway. I was a member of the NJDOE Evaluation Pilot Advisory Committee, and at meeting after meeting we identified problems such as evaluating teachers in semester courses, how to allocate "growth" in collaborative teaching arrangements, teachers whose students don't come to class, homebound, etc. But they just glossed over it all and rammed this program through.

Based on everything presented to the EPAC it seemed pretty clear that using student achievement to measure teacher effectiveness is a fool's errand, but the results were preordained.


Duke said...

Thx for that, TM. Never worry about typos here.

Bruce, great stuff, and you are absolutely right. Unlike the EE Task Force, the EPAC advisors knew what they were talking about. I'm just egocentric enough to believe my little rants may have made a difference there.

Will follow up soon...

Lori said...

I teach in New York. It sounds like started how the whole testing debacle started here. In the beginning, some teachers were held responsible (4-8) and then that started the whole school grading thing. I remember a school-wide meeting where we told that all the information was being uploaded into computers but not to worry it was only to help the teachers grow professionally as individuals. Then, teachers names and grades got published in the papers. Apparentently, Klein and company urged reporters to file to see them under the Freedom of Information Act because those teacher "grades" were part of the public record now. It was all part of a plan to cow us and demean us and turn the public against us to point where we are all being graded starting at K using really bogus tests and not being told how much growth is enough because they want to play with the numbers to figure out how many people will get fired. The "growth" we are supposed to show is a moving target. NYC teachers have been working without a contract for years longer than any others. Don't let Cerf and company to dominate the conversation. Demand that your union put out ads about how the NEW ranks NJ schools number one regarding quality, #1 for 8th grade in writing, #4 in reading, #3 in math... It's all their website. No more "teacher cares" ads because all you will get back is that the teachers are lazy and greedy. And that sounds a lot catchier than we care. You gotta fight fire with fire. Use your own stats against them. Yeah, you care but look you have great schools to back it up. Make the union work and not cave. Our union gave in and gave in and then we had no goodwill built up for us. As teachers, we told ourselves it wasn't so bad. It is only certain teachers. Now, it is the worst year ever in my teaching career. It CAN get worse. Mobilize and stick together. Don't be afraid.. Christie touts those stats in the web even as he demonizes teachers. Flaunt It Right Back.