Thoughts on the Graduation Exam Mess in New Jersey

Let me recap where we are first -- skip down if you know the background.

Way back in 1979, New Jersey passed a law which required high school students to take an exam before graduating (the Education Law Center has some background on that law here and here). The law says the exam must "...measure those basic skills all students must possess to function politically, economically and socially in a democratic society." (N.J.S.A. 18A:7C-6.1.) Keep in mind that New Jersey is only one of only 12 states to require require an exit exam for high school graduates.

In 2014, the NJ Department of Education, under then-Governor Chris Christie, decided to replace the exam, known as the HSPA, with two tests from the Partnership for Assessment of Readiness for College and Career (PARCC): the Grade 10 English Language Arts and multi-grade Algebra I exams.

In 2016, ELC, the American Civil Liberties Union, and several other groups filed suit against the state, challenging the use of the PARCC tests as exit exams. Late last year, the state appellate court ruled in favor of the plaintiffs and struck down the regulations that led to the change in tests. The court found several problems with the change: the law calls for a single exam, not multiple ones, and the exam has to be in Grade 11, not in multiple grades like the Algebra I exam. 

The ruling left New Jersey's current juniors and seniors without a clear path to graduation. Some advocates suggested the exams should be scrapped altogether. But the plaintiffs and the state reached a settlement that allowed these students to take multiple testing pathways to graduation, including getting a passing score on the SAT, ACT, or military placement exams. The agreement would give the NJDOE time to develop new regulations on exit exams for current sophomores and freshman.

At the same time, however, State Senator Teresa Ruiz introduced a bill that changes the original statute to align with the current regulations -- the same regulations the court declared illegal. As I wrote previously, I find it highly problematic that the law is being changed to meet the regulations, and not vice versa.

But the NJ Senate apparently has no such qualms: they passed the bill 21 to 7, even after the settlement had been reached. The bill will now move on to the Assembly, where it will be voted on this Monday.

* * *

So that's where we are. Now let me add, in no particular order, my thoughts on all this:

- There is no evidence exit exams, by themselves, improve learning outcomes for students.

Again: only 12 states require exit exams. There is no evidence I've seen that these states have improved their outcomes as a result of having a graduation test.

There is some limited evidence accountability testing improves student outcomes; however, that evidence has been challenged in recent years. But that type of testing is not the same as testing which imposes a high-stakes consequence -- such as not receiving a diploma -- on students. We simply do not have evidence that imposing this requirement will lead to better student outcomes.

- A core principle of testing is that you cannot use an assessment for a specific purpose unless you make an argument it is valid for that purpose -- and no one has made that validity argument for using the PARCC tests as a graduation exam.

I keep a copy of Standards for Educational and Psychological Testing within reach when I'm writing about this stuff. Standard 1.0 -- the very first one -- states:

Clear articulation of each intended test score interpretation for a specific use should be set forth, and appropriate validity evidence in support of each intended interpretation should be provided. (p. 23)

The PARCC Algebra I exam purports to be an assessment of a student's ability in algebra. We can argue all day about whether it is... but no one has yet put forward that it is an appropriate assessment of "...those basic skills all students must possess to function politically, economically and socially in a democratic society" -- the clear language of the law.

Ruiz's bill, by the way, pulls off neat trick: it pulls the word "basic" from the original statute. This strikes me as a way of avoiding what should be at the core of any debate about the law: what exactly are these "skills," basic or otherwise?

Here, for example, is a sample question from the PARCC Algebra I exam:

Would you say that someone who can't answer this question cannot "function politically, economically and socially in a democratic society"? How many members of the NJ Legislature, for example, do you think could answer this question? How many of the senators who voted in favor of Ruiz's bill could? If they can't, does that mean they are unable to "function"?

- When you use a test to make a high-stakes decision, your validity argument must be especially strong.

Denying a young person a high school diploma is a serious matter -- especially if they've been attending school and doing their required work. You are closing off all sorts of opportunities to them to participate meaningfully in the workforce and in civic life. 

If you are going to deny them those opportunities based on a couple of tests, you'd better be sure the consequences of that denial are justified. You better spend time listening to experts about the validity of the test for the purposes stated in the law. You'd better be sure the law itself is a good idea.

Nothing even remotely like this happened in the NJ Senate. This bill was quickly rammed through without any real discussion, informed by expert opinion, of whether the PARCC is a valid instrument for the purposes of the law.

- The "passing score" for the PARCC is completely arbitrary -- it can be set anywhere the state wants.

As I pointed out previously, only 46 percent of test takers last year passed the PARCC Algebra I exam. There is simply no way that the public is going to put up with a graduation rate that aligns with that passing rate. Which means there will be enormous political pressure to change the proficiency rate on the exam.

And, contrary to the opinions of the ill-informed, changing proficiency levels is a rather simple matter.

As I've noted previously, tests like the PARCC are designed to yield normal, bell-curved distributions of scores. Within that distribution, anyone with the power to do so can set a passing or proficiency rate wherever they like.

The notion that "proficiency" is some objective standard, therefore, is simply not true. Proficiency is a social construct and, in my opinion, an artifact of political struggles; look at the history of New York's proficiency rates if you doubt me.

As Anne Hyslop points out, one of the consequences of using the PARCC as an exit exam is that there will be great pressure put on the state to set the standard for "proficiency" much lower than it currently is. That likely will affect the rates on the other exams throughout the various grade levels where PARCC is administered. Ironically, the push to set a high proficiency level for graduation may, in fact, lead to lower proficiency levels on all the PARCC exams.*

It really comes down to this: either the proficiency rate on the PARCC is going to come down, or lots of students are going to be denied diplomas. Which do you think is more likely to occur?

- The argument that community college remediation rates are so bad that we need graduation exams is weak.

In a truly awful editorial, the reliably reformy Star-Ledger makes this claim:

At Essex County College in Newark, 85 percent of incoming freshmen need to take remedial math. In 2017, only 13 percent graduated. While social promotion also happens in wealthier districts, those kids have a deeper safety net.

This is why we need an objective test. Yet because the PARCC is such a powerful diagnostic tool that can trace a student’s learning problem right down to a particular teacher’s lesson, it ran afoul of the union.

I posted that last paragraph on Twitter, and howls of derision from educators and testing experts ensued. It is, of course, impossible for any standardized test to pinpoint a teacher's particular lesson as the cause of a learning deficit; in fact, standardized test results, by themselves, cannot attribute the cause of any student learning outcome. If the S-L editorial board ever bothered to listen to experts in the field, maybe they wouldn't make such embarrassing claims.

Again: testing, by itself, cannot improve instruction. But the S-L appears to believe that if the PARCC was implemented as an exit exam, the remediation rates for math at Essex County College would improve. As I've pointed out in the past, however, the majority of community college students are older than 22, which means they haven't had a high school math course in years.

Further, if you click through on the link the S-L gives, you'll find that ECC is struggling mightily to provide remedial instruction, which of course raises the question: if better math instruction is simply the result of more testing, why are the community colleges having such a hard time teaching self-selected students the same material?

Maybe it's because, as Sara Goldrick-Rab has been finding in her invaluable work, many college students are suffering from food and housing insecurity. Of course, so are many high school students. Is denying them a diploma because they can't pass the PARCC really the best solution to this crisis?

College remediation rates are a club that reformy types regularly use to beat up on public education and public school teachers. But "ready for college work" is not an objective standard, and the research around this topic has many limitations. In addition: if legislators are going to insist that all students must show they are ready to meet the high standard of being ready for college work, then...

- No legislator should vote for this bill unless they are prepared to spend a lot more money to give students a real opportunity to pass the exam.

Again, from Standards for Educational and Psychological Testing:

Standard 12.8 : When test results contribute substantially to decisions about student promotion or graduation, evidence should be provided that students have had an opportunity to learn the content and skills measured by the test.

"Opportunity to learn" is a core concept in educational testing. The Standards define it as:

The extent to which test takers have been exposed to the tested constructs through their educational program and/or have had exposure to or experience with the language or the majority culture required to understand the test. (p. 221)

I actually have a bit of a problem with this definition, because "exposed" comes across as a passive conception: leave an algebra textbook in front of a kid and arguably you've "exposed" them to the material covered by the PARCC. But that's clearly not what we're talking about here; the student needs to be given an opportunity to be meaningfully engaged with the material.

That means a qualified teacher, in a well-resourced school with a well-constructed curriculum, with all the support necessary -- both in and out of school -- for the student to be able to thrive.

All the way back in 2008, the NJ Legislature passed a bill, the School Funding Reform Act, which codified the amount of money necessary for students to achieve a minimally adequate education. But then the Legislature repeatedly refused to fully fund what the state's own law says schools need to educate students.

Worse, as Bruce Baker pointed out in real time, there's plenty of reason to believe the amount set was not enough. And the standards back in 2008 were lower than what the PARCC sets today. There is no doubt that sufficient funding is the necessary pre-condition to achieve desired educational outcomes. There is also no doubt the state has not come close to providing that funding.

If the New Jersey Legislature is prepared to set a much higher bar for graduation, they have a moral and legal obligation to provide the extra resources needed to clear that bar. If they can't or won't, they have no business imposing an onerous testing regime on students who came to school, did their work, and passed their classes.

One final thought:

New Jersey is home to some of the nation's foremost experts on psychometrics. I can't claim to know every conversation every legislator has had regarding this issue, but from the news reports I've seen, it seems these experts have largely been left out of the debate over the use of PARCC that's been going for the last year.

Why?

I understand lots of people have a stake in this issue, and we should hear from parents, educators, and other interested parties. But expertise has got to count for something. The Assembly would be well-advised to withdraw this bill from a vote on Monday and, instead, convene a panel of experts in testing and its uses to study the issue.

Time is no longer an issue: the settlement ensures that current upperclass high school students have a clear path to graduation. The state should take its time and get this right. As Senator Ruiz herself said:

“Even though the agreement came through, there has to be a statute change,” Ruiz said. “Either you do this today or we do it in two years when we come to this crossroads again.”

That's exactly right: we have two years to study this and get it right. Why wouldn't we take advantage of that?

More to come...

* One aspect of setting proficiency rates I rarely see discussed is whether it helps or hurts a child to hear that they didn't "pass" an exam. Some doctors have suggested ADHD rates have risen in the era of testing accountability. High-stakes testing has been shown to induce negative feelings in children and stress in students and teachers. Undoubtedly the fear of failure is the primary cause.

Is the push to raise standards worth this cost? Shouldn't we at least have a conversation about it?