Standardized Tests: Symptoms, Not Causes

Let's talk about tests for a bit:

All Standardized tests, by design, yield "normal" or bell curve distributions:

You sometimes hear quanty folks say: "God loves the normal distribution." In this case, however, man is creating God in his own image, because the tests are designed to yield this distribution of scores.

In a normal distribution, most students get average scores: that's why the highest point in the curve is in the middle. As we move to the right from the middle, the test scores go up, but fewer and fewer students get those higher scores. Likewise, as we move to the left from the center, fewer students get the lowest scores.

How does this happen? Well, to start, and as testing expert Daniel Koretz has explained, we only test students on what we reasonably expect them to be able to do. We don't test fourth graders on trigonometry because we know most of them can't solve those kind of problems and wouldn't be able to even if we drilled them on it repeatedly. Similarly, we don't test high school juniors on adding one-digit numbers because we know that would be too easy.

Are there fourth graders that know the difference between a sine and a cosine? Are there juniors who can't add 5 and 4? Of course there are, but we know they are far away from the norm; we don't set our standards based on these outliers. Because we like to see ourselves as normally distributed, we create standards and tests that give us normal distributions. 

This can't be stressed enough in the testing debates: we design tests not based on objective criteria, but on socially constructed frameworks that assume some of us are above average, some of us are below, and most of us are in the middle.

Now, sometimes we design a test that is "too hard" or "too easy." What does that mean?

Here's a test where many students got high grades; it's skewed negative, meaning many students got grades close to the top of the test's scale. In fact, the "right tail" of the curve is cut off: a good number of students got the highest grade possible. This is a "ceiling effect," and many people, including the NJDOE and other authorities in education, do not like this distribution. They think the test is "too easy," because they assume that students must be normally distributed.

Hence the shift to tests like the PARCC, which reformy folk say is testing "higher order thinking" and "real world problem solving" and so on. Frankly, if you can find a real world situation where people use phrases like "constant of proportionality," I'd be surprised. What's really going on is that the questions are more difficult so the test can lose its skewness and return a normal distribution. 

The irony here is that the promoters of standardized testing are using an argument against inequity to insist on the continued use of these tests (if not their expansion), and to shame those who are opting their children out of these tests as perpetuators of race and class injustices. 

We'll leave aside the point for the moment that these same people also refuse to actually provide the funds the law -- based on a large and growing body of empirical research -- says are needed to equalize the test-based goals they've set out. 

Instead, let's look at the logic of their argument. Somehow, everyone has to be "college and career ready" (as if that is an objective criterion), and we're going to insist that everyone perform at the highest possible levels. But we're going to use a test that forces a normal distribution; and if that test actually shows that many students are meeting a high standard, we'll declare the test "too easy," and redesign it so we get back the bell curve we crave.

Does everyone see the problem here? We're insisting that all children demonstrate high performance on a test that, by design, only allows a few children to demonstrate high performance.

This is where "proficiency rates" enter the conversation:* 

All a proficiency rate does is set the cut point along the normal distribution. Where the rate is set is entirely up to whomever has the power to set it; it can be as low or as high as they like.

But what usually happens is that proficiency is determined by another test that -- surprise! -- yields a normal distribution. And that test -- say, the SAT -- is tied to some other normalized standard, like college freshman GPA. Why do I say that's normalized? Because even the reformiest of the reformy admit not everyone can or should go to college, and college standards are determined, like those in K-12 schools, by what we reasonably expect the average college student do be able to do. It's all normalized.

This is a fundamental contradiction inherent in the arguments for standardized testing as necessary prerequisites for addressing inequitable outcomes in education. Standardized tests, by design, give us bell curves, and reformy types insist we change them if too many students are getting high scores. But then they moan that not enough students are above average!

Further: they fail to see what the tests are really measuring:

The correlation between socio-economic status and test scores is absolutely iron-clad. Does anyone think eliminating the ceiling effect is going to change this? Granted, there is likely a ceiling on how income effects test scores: a kid in a family making $300K a year probably isn't at much, if any, disadvantage compared to a kid in a family making $500K.

But the wealthy have always enjoyed an advantage in our false meritocracy. The biases in the tests themselves, coupled with the inequitable distribution of resources available for schools, all but guarantee the majority of the variation in test scores will be explained by class.

The neo-liberal view appears to be that this is inevitable and just, so long as we decouple these inequities from race. If we can get some more students of color into elite schools, and create a few more black and brown millionaires and billionaires, everything will be "fair." The owners of the country can then sleep soundly at night, content that they may be classists, but they aren't racists.

I'm all for social mobility, but increasing it isn't the same as decreasing inequity. There are millions of people in this country doing difficult, necessary jobs. It's wrong to consign people of color to these jobs through a system of social reproduction in our schools. But it's also wrong to pretend that we have a system where everybody can be above average, and in doing so can make a better life for themselves.

So long as we keep making bell curves, somebody has to be on the left side. Somebody has to do the work that needs to get done. But there's no reason those decent, hardworking people shouldn't have good wages and good medical care and good housing and disposable income and workplace rights and time to spend raising their children.

Yes, their children should have the same opportunities to move to the right side of the bell curve. But if they do, somebody is going to have to take their place. Maybe if the consequences for being on the left side of the bell curve weren't so dire, affluent people wouldn't be as obsessed with maintaining the advantages they enjoy in keeping their children on the right side. Maybe they would stop pushing their children to the breaking point just to stay ahead of the pack:



Maybe we'd allow children to become themselves and realize their full potentials, free of the fear that their "failure" will inevitably banish them to a life of toil and misery. Maybe we'd start to see schooling not as preparation for a life of stepping on our fellow citizens, and instead as a process by which we become a people who balance our own self-interest with caring for our fellow citizens.

And then maybe we wouldn't feel the need to make these bell curves at all.


* UPDATE: Hey, I got it to loop!

** UPDATE 2: Sometimes you can look at something a hundred times and you never really see it. Thanks to MFortun in the comments for pointing out my axis titles were backwards!