We Don't Need Standardized Testing In a Pandemic

UPDATE: Just as I was finishing this post, I caught a glimpse of this news:

The U.S. Department of Education extended flexibility to states today in how and when they administer mandated end-of-the-year assessments, including allowing shorter tests that can be given remotely and, in the summer, or even in the fall. The federal agency advised states to blunt the impact of the tests, suggesting the scores not be used in final grades and grade promotion decisions.

However, despite the disruption to schools from the pandemic, the federal agency did not liberate states from administering standardized tests; it will continue to require statewide assessments. Some states, including Georgia, requested waivers that would allow them to forgo standardized testing altogether this year.

As I argue below, this is a bad idea. I really hope the Biden administration can be persuaded to reconsider.

* * *

A notable bit of education policy news in New Jersey last week:

New Jersey will apply to the federal government to waive standardized testing for the current school year as districts across the state continue to cope with the constraints of the coronavirus pandemic, Gov. Phil Murphy announced Friday.

Murphy said the state has not yet received word from President Joe Biden’s administration as to whether it will accept its application to waive the federal testing requirements.

“We also recognize the importance of statewide assessments to gauge where our students’ learning,” the governor said during his latest COVID-19 briefing in Trenton. “But given the need to ensure our students’ instructional time is maximized and the levels of stress on them, our educators, our school administrators, and our families are minimized, we are putting forward this waiver request.”

The state is hardly alone, as several others have also applied for waivers from the federal mandate for testing. But there's plenty of pushback, as it's become an article of faith among certain education reformers that we must have tests, and that in a time of crisis testing is more important than ever. In Florida, for example, students will be required to show up to take their tests in person, even if their parents have been keeping them home during the pandemic. Why?

State Education Commissioner Richard Corcoran signed an order this week requiring the tests. The order said the tests are more important than ever because many struggling students are learning at home and falling behind.

Test results will “ensure that each student is given the services and supports they need to succeed in life,” the order says.

If you take Commissioner Corcoran at his word, he's going to use the tests to make sure students get what they need. But that's coming from a state that chronically underfunds its schools. If students weren't getting the supports they needed before the pandemic, what would make anyone think they'll get them now -- but only if they take a test?

Corcoran is engaging in the typically facile thinking that has become the hallmark of many who espouse the virtues of standardized testing. I have a set of questions I like to ask whenever anyone says students must take a standardized test:

- What are you going to do with the test results? 

A core concept of assessment is that tests must be shown to be valid for the purposes in which they will be used. In other words: you should make a separate argument for every proposed use of a test. A test that may be valid to use for, say, determining whether there are enough overall resources in the education system isn't necessarily valid for the purpose of determining whether a student should pass to the next grade.

Psyshometricians often speak of making a validity argument in favor of the use of a test for a particular purpose. That argument should touch upon the relevancy of the outcomes to a specific use, the consequences of making decisions based on these outcomes, the opportunity to learn the content in the test, and other factors.

One of the biggest failings in our current testing system is that we use statewide standardized tests for many purposes -- even if no one has presented an argument for those uses. Some lawmakers have argued that these tests should be used as a graduation exam or to determine grade promotion, even though most have never put forward an argument that the test is valid for that purpose. Some say test outcomes run through a statistical model should be used to evaluate teachers, again without making an argument against the many reasons that's a bad idea.

That's especially true this year. The fact is that too much of what is happening in schools is out of the control of teachers, administrators, or students. Students have had wildly uneven opportunities to learn during the pandemic, and it's not fair to visit consequences on them or their teachers based on outcomes in this year's tests.

One implication those who argue for standardized testing make is that teachers need the data to help students make up for the learning they've lost over the past year. But these tests provide little meaningful information for teachers. First, the results take a long time to come back, so they aren't useful in real time. Second, teachers aren't allowed to see the questions, which makes them mostly useless in informing instruction. Third, there usually aren't enough items in each area of content knowledge to provide reliable data. Teachers need to assess their students and get to work quickly; these tests weren't designed to help with that task.

So, if the tests aren't going to be used for teacher evaluation, or promotion/graduation, or allocating resources, or informing instruction... why do we need to give them? For what purpose, exactly, should they be used? No one should advocate for administering tests unless they can give a specific answer to this question.

- What is the cost of administering the tests?

I'm not just talking about dollar costs, although those can be significant. But the truth is that we have known for a good long while that administering standardized tests in math and English has extracted a price on students, educators, and the K-12 education system aside from mere dollars. 

Testing has narrowed the curriculum, especially in districts with high levels of student poverty and, consequently, relatively low average test scores. The pressure on students to perform on these tests has been well documented. Several major cheating scandals highlight the pressure adults also feel, leading to moral compromises that would likely be absent in an environment without such high-stakes attached to the tests.  

Add this to the current worries about contracting COVID-19 in school and it becomes clear that the cost of administering these tests this spring will be very, very high. Which leads me to ask...

- Is the cost of testing worth it?

What, exactly, will statewide standardized test outcomes tell us that we didn't already know, or that we couldn't find out some other way? That students didn't gain as much learning as they would have without the pandemic? We already know this; and again, it's not like the tests will give educators data they couldn't get other ways that are much faster and more detailed.

Will we learn that students who are economically disadvantaged need more resources to equalize their educational opportunity? We already knew that before the pandemic. And does anyone really think that otherwise reluctant politicians will be persuaded to dole out more funds when they see this year's test scores? Really?

It's worth noting that all tests are subject to construct irrelevant variation, a fancy term that means the outcomes can vary based on things other than a student's knowledge. Students who take tests in less-than-optimal conditions, for example, are more likely to do poorly than they would if they were in better circumstances. 

This year, the differences in the testing environments will be more stark than ever. Students in districts that remain fully remote will take their tests in widely-ranging home environments. Students who take their tests in school may be in smaller cohorts, or crammed together in full classes. The COVID mitigations they face during the tests varies widely, as does the technology available to take the tests. 

Variations in test-taking conditions has always been great, but now those conditions will very even more than before. How, then, can we trust that the test is measuring what it's supposed to actually measure?

Normally, I believe that test data can be useful for research and policymaking purposes -- although I think we could get data just as good as we have now for a lot less cost and bother by cutting back the amount of testing and removing unvalidated attachments to high-stakes decisions. 

But the more I think about it, I the more I come to the conclusion that the data this year isn't going to be of much use. Why, then, go through the bother of testing kids when the cost is so high and the value of the results is so low?

The tests can wait a year. Give everyone a break.

ADDING: Honestly, this makes no sense to me:

“To be successful once schools have re-opened, we need to understand the impact COVID-19 has had on learning and identify what resources and supports students need,” Ian Rosenblum, acting assistant education secretary, wrote in a letter to state education leaders.

What decisions exactly are you going to make based on those scores? If you can't answer that, you don't need the data.

Identifying the resources and supports students need is the job of schools. The government's job is to get schools the extra money they're going to need so schools can provide resources and supports. Go work on that and leave the educational decisions to educators.