I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Tuesday, December 27, 2011

Poverty, Shmoverty: Part IV

This is a little wonky, but I think the payoff is worth it. Bear with me...


So I've been going on a bit about international comparisons of both student achievement and poverty (go here, here, and here to follow). The impetus for all this was Amanda Ripley's contention that low-poverty schools in the US do a lousy job compared to low-poverty schools around the world.

Twice, Ripley compares the 75th percentile in socio-economic status (What the PISA test calls ESCS) in the US to the 75th percentile in the rest of the world. I have said that:

  1. The top quarter are hardly what I'd call "rich kids," the term Ripley uses.
  2. The nature of income distribution in the US may make our top quarter a lot less homogenous than other countries' top quarter.
  3. That aside, ranking countries doesn't tell us much about how much further ahead or behind the US may be; we have to look at the scores themselves.
Now, Ripley does a whole schtick on how poorly we are doing in math and science, using 2003 results. To her credit, she includes a graph that at least shows more than rankings:

But if you look at the fine print, the problem remains the same: we are dealing with the top quarter, and NOT with the "rich kids." (And, again: what do 50 points actually mean in a comparison like this?) Let's instead look at the data disaggregated for the real "rich kids": the top 5%.

I don't have the data for the 2003 math exam, but I do have it for the 2009 reading exam: Table II.1.1 on p.152 of PISA 2009 Results: Overcoming Social Background: Equity in Learning Opportunities and Outcomes (Volume II). I showed this table before:


OK, our rich kids are #8; but only one point behind Belgium, and only 2 points behind Korea. Is this really a huge problem?

Now let's look again at the differential between the 95th percentile kids and the 75th percentile kids:


So the gap is further apart between 75th and 95th percentile kids in SES than any other country but Israel; in other words, kids in the US benefit more from being higher up the SES ladder than in most other countries. (And look who has the smallest differentials: envied Finland and Korea. Huh.)

Let me show this another way. Let's compare the US score for kids at different SES percentiles to the top scoring country in that percentile. How far behind do US kids fall?


In the 10th percentile for SES - the 10% of the poorest kids in each country - Korea's kids do the best in the world. And they beat the US's poorest kids by over 60 points.

But in the top 95th percentile for SES - the 5% of the richest kids in each country - New Zealand's richest kids do the best in the world. And they beat the US's richest kids by only 22 points.

In other words: wealth alone cuts the gap between our kids and the top performers in the world by two-thirds.

But hold on! This really isn't a good comparison; it doesn't account for outliers. Go up to my first graph: New Zealand beats the next best scorer in the 95th percentile by 10 points. This suggests there may be some anomaly that's creating statistical noise and giving NZ an unfair advantage; maybe the test had a bunch of hard questions about counting kiwis or something...

Let's do this: take the top eleven overall scorers for all kids (the US ties with Poland and Iceland at #12, but we'll keep it simple and exclude them). Average the scores for those top eleven at each percentile for SES. Then look at how the US compares; that should mitigate against outliers a bit. How are we doing?


And look at that: the 20-point differential at the 10th percentile disappears to next to nothing at the 95th percentile. What does this mean?

It means kids in the US are penalized MORE for being poor or even middle class than kids in other high-scoring countries.

Now, you can conjecture this is due to a lot of things. Maybe the schools get a lot worse the lower you go on the SES ladder in the US than in other countries. Maybe the real difference in the life a poor child and a rich child is greater in the US than in other places. This is a big, serious topic and it deserves real debate.

What you can't say is that schools for rich kids in the US aren't pulling their weight. Because the richer a US school's population is, the more it's like the schools in the rest of the world. In fact, it's tempting to want to disaggregate the data even further: how would our top 1% fare compared to the rest of the world's top 1%? I'm not sure it would tell us anything, but I'd be curious.

In any case, this PISA data shows two things:
  1. Socio-economic status matters, and it matters more in the US than in other countries.
  2. Our "rich kids" do pretty well compared to the rest of the world.
Last point: I've given up a lot of premises making this argument. I'm giving up that the PISA is a valid international comparison. I'm giving up it's measuring real learning in 15-year-olds, just as they are on the cusp of true abstract thinking. I'm giving up that the PISA measure of socio-economic status is valid and reliable across countries. 

I think my point here is valid, but I'm not prepared to keep operating on premises for which I have real doubts. I have big reservations about whether standardized tests do a very good job measuring how well-educated a child is - particularly an older child. I don't know enough about PISA to say it allows for good international comparisons, and I really don't know anything about the validity of the PISA SES measures.

I just don't care to continue argue this on others' terms. So this will be my last post on this for a while.

Unless someone says something really stupid....

ADDING: The spell checker changed "disaggregated" to "desegregated." Funny in a weird, wonky way.

17 comments:

Stuart Buck said...

But wouldn't you bet that the top 5% in the U.S. -- given our greater wealth and our greater inequality -- actually have way more money and resources than the top 5% in Australia, Belgium, etc.?

So doesn't your top 5% chart actually suggest that education here totally stinks compared to elsewhere? Alternatively, maybe income doesn't matter as much as you'd like to think?

KatieO said...

Jersey, great post (yet again!). Let's not forget that having huge economic inequality brings down the WHOLE society. See this great Ted Talk "Richard Wilkinson: How economic inequality harms societies": http://www.ted.com/talks/lang/en/richard_wilkinson.html

I too dove into that PISA document after reading Amanda Ripley's ridiculous assertion. This one line in particular stood out to me: "With the exception of Turkey, Slovenia, Israel and the United States, where socio-economically DISADVANTAGED SCHOOLS ALSO TEND TO BE DEPRIVED IN TERMS OF BASIC RESOURCES, such as larger student staff ratios..."(p. 13 Emphasis mine.) Notice what company the US is in there. We UNDERFUND the schools which need the most help! Geez...

But back to the topic at hand, I too have little faith in the test itself as a measure of...well, anything. I taught in a Japanese high school for many years. My students sure were great test takers, but don't ever ask them to do anything which required critical thinking (i.e. debating a topic or talking in a foreign language class.)

KatieO said...

Oh and Stuart, as someone who has actually spent significant time in one of the "high-achieving" nation's classrooms, I can safely say that I was NOT impressed by what I saw. I know first-hand the actual experience of students in classrooms and those classrooms were simply not extraordinary.

In fact, I would put my Alma mater, New Trier High School located in a north suburb of Chicago (one of the top 5% of US schools) against ANY school in the world. The amazing teaching, the ridiculous variety in courses offered (How about classes like Russian, Chinese, Hebrew, every possible AP course and don't forget rock-climbing, FOUR wind ensembles, FOUR jazz bands, FOUR orchestras, sculpture, photography, dance...And that doesn't even include after-school activities!) Students wrote and directed full-length comedy shows and musicals, ran both a radio station and a public access cable station. We won state in almost every sport. We kicked every other school in Illinois with our test scores (excluding selective enrollment). And we were a public school.

Go ahead. Tell me my old school sucks.

But all that was at the expense of other schools like the inner-city schools which I work with now. The savage inequalities will always kick America's butt.

Duke said...

Thx Katie.

Stuart, I will not concede your point on the top 5% at all. Those countries have single-payer health care, pensions, subsidized housing, strong public transportation, large public social infrastructure, free/cheap college, and a host of others things six-figure earners in the US have to pay out of pocket even after paying taxes.

So, no, income doesn't matter as much as SES and quality of life do.

I find your insistence on living in denial of this to be astonishing. These other countries' schools - with their unionized teacher workforces and lack of parental "choice" - are the primary difference between measures of student achievement? When we know at least 60% of that achievement is determined by factors outside of the school?

I admire your tenaciousness, Stuart. You're like a pit bull. But you really need to step back a little and think about your position.

Stuart Buck said...

So far, Ravitch is the only one who has said anything that is completely stupid. One can argue all day about how to compare international test scores and poverty rates, but one thing that no intelligent person would ever do is make international comparisons (which are hard enough) that are based on completely different definitions of poverty.

Stuart Buck said...

" These other countries' schools - with their unionized teacher workforces and lack of parental "choice" - are the primary difference between measures of student achievement? "

Other countries tend to have much more parental choice than America. Several European countries actually have parental school choice as a constitutional right.

Stuart Buck said...

You don't concede the point on the top 5%? I thought the usual argument was that the top 5% here have it much better themselves because they aren't paying the tax rates that would support the social services that Europe has. But you evidently think that European countries with much lower GDP-per-person than the U.S. nonetheless manage to have poor people who are better off AND rich people who are better off? What are they, perpetual motion machines, manufacturing social welfare out of nothing?

Duke said...

Stuart, as usual, Krugman says it all:

http://www.nytimes.com/2011/11/25/opinion/we-are-the-99-9.html

Hey, do you have a blog? So I can leave comments there...

Anonymous said...

"Comparisons are odorous," saith Dogberry. (Who was his English teacher?)

Singapore does a good job with math--taught to the tune of rattan cane.

Could we contrast cultures now?


Happy New Year, J.J.!

Stuart Buck said...

Sure, leave comments there. That would be great.

Duke said...

Just need the URL. I always appreciate a sincere, well0informed debate.

Duke said...

Anon: Interesting point, and poetically put.

Happy New Year to you as well!

Duke said...

Stuart: ah, stupid me - just click on your name. Got it.

Anonymous said...

It's the screaming "reformers" and the hacks in the media who are always bringing up how much better the foreign schools are doing than the US. The foreign schools, which have unionized work forces, no charter schools and no home schooling. Home schooling is illegal in Germany and does not exist in Finland. The notion of school choice would be ridiculous in most of the other advanced democracies because it would undermine their national school systems. The US child poverty rate is 21% while in most of Finland, Denmark, Norway and Sweden the child poverty rate is well below 10% (more like the 3% to 5% range).

Stuart Buck said...

"The notion of school choice would be ridiculous in most of the other advanced democracies because it would undermine their national school systems."

Completely false. As none other than Diane Ravitch said in 2001 (she doesn't contradict this statement now; she just ignores it):

"Among the modern industrialized nations of the world, the United States is in a minority on this issue. Of the thirty nations that are part of the Organization for Economic Cooperation and Development, only seven do not permit any government funding of K-12 private schools; in addition to the United States, they include Greece, Ireland, Japan, Mexico, Portugal, and Turkey. The proportion of students in government-funded private schools is sizable in countries such as Australia (25 percent), Belgium (58 percent), Denmark (11 percent), France (16.8 percent), South Korea (21 percent), the Netherlands (76 percent), Spain (24 percent), and the United Kingdom (30 percent). The remaining sixteen nations subsidize private education, but their enrollments are smaller. In this category are countries such as Austria (7 percent), Canada (2 percent), Finland (4 percent), Luxembourg (6 percent), Sweden (2 percent), and Switzerland (2 percent). Yet even these small enrollments are significant: in the United States, 2 percent would translate into about one million children."

In fact, as I pointed out above, school choice is a constitutional right in some European countries. The Netherlands Constitution, Article 23, provides, "Private primary schools that satisfy the conditions laid down by Act of Parliament shall be financed from public funds according to the same standards as public-authority schools."

Similarly, the Belgian Constitution, Article 24, provides, "The community offers free choice to parents. . . . All pupils of school age have the right to moral or religious education at the community’s expense." Sweden also has a universal voucher system, although under law rather than by constitution: http://news.bbc.co.uk/2/hi/uk_news/education/3717744.stm

Duke said...

Stuart, why the obsession with plying gotcha with Diane Ravitch? Is the woman not allowed to change her mind?

From your link:

"Like the Conservatives' proposals in England, the Swedish voucher cannot be "topped up". In other words, any private school participating in the scheme cannot charge any additional fees.

Nor can the private schools select pupils on any basis other than first-come-first-served."

Yeah, little bit different than what we're talking about here. As the report says:

"But there is one big difference between the Swedish and the English school systems. Whereas Sweden had virtually no pre-existing private sector, independent schools are long-established in England.

Would leading English independent schools be willing to provide places at the same cost as state schools when their own fees are currently two to three times this level?

And, just as tricky, would they be ready to give up their right to select pupils by entrance test or interview?"

I have an answer when it comes to America: no.

Duke said...

Also:

http://www.theatlantic.com/national/archive/2011/12/what-americans-keep-ignoring-about-finlands-school-success/250564/

"Finally, in Finland, school choice is noticeably not a priority, nor is engaging the private sector at all. Which brings us back to the silence after Sahlberg's comment at the Dwight School that schools like Dwight don't exist in Finland.

"Here in America," Sahlberg said at the Teachers College, "parents can choose to take their kids to private schools. It's the same idea of a marketplace that applies to, say, shops. Schools are a shop and parents can buy what ever they want. In Finland parents can also choose. But the options are all the same."

Herein lay the real shocker. As Sahlberg continued, his core message emerged, whether or not anyone in his American audience heard it.

Decades ago, when the Finnish school system was badly in need of reform, the goal of the program that Finland instituted, resulting in so much success today, was never excellence. It was equity.
"