Who You Gonna Believe: PARCC Cheerleaders, Or Your Own Lying Eyes?

There was a debate here in New Jersey this week, sponsored by radio station NJ 101.5, about the new Common Core-aligned tests known as the PARCC (Partnership for Assessment of Readiness for College and Careers).

The PARCC-skeptical side was represented by Wendell Steinhauer, president of the New Jersey Education Association, the state's largest teachers union. The pro-PARCC side was argued by Dr. Sandra Alberti, former director of Academic Standards for the NJ Department of Education.

I thought Alberti's answers were quite representative of the pro-PARCC arguments I hear from a variety of sources. So I'd like to focus in on one particular statement she made at 49:50 into the debate. Steinhauer had made the point that the over-emphasis on standardized testing was leading to a narrowing of the curriculum, where test prep had taken over too much time and focus. Alberti wasn't buying it:

(49:50) DR. SANDRA ALBERTI: When it's an assessment [the PARCC] that actually asks you to think, test prep looks like we're thinking in schools every day, and those are the schools I want for my children, those are the schools I want for this nation. 

Test prep, the way it's designed for PARCC, that kind of drill is ineffective; it won't work for PARCC. There are lots of high-performing charter schools who are in a panic right now because their typical systems of give a standard drill, test, drill, test, doesn't work. 

The whole design of this assessment is to be an assessment worthy, worthy of instructional imitation and that's what we want. I don't want one minute wasted on test prep if it's a test not worth knowing.

These are tests that are worth it, and I get it that we don't have enough transparency about this. I get it that I'm selling like this fairy tale of what a test could be. But I think it's too early to pass this judgment based on some released items that will never make it on a real test, they weren't field tested like the real items were. They're a communication piece. They're not the real test. 

This is a test that I can tell you, as somebody who has read every single NJASK and HSPA item that we have given, generations better than what we've ever seen for our kids.  

So if our argument is no standardized testing, that's one thing. But if it is standardized testing with a test that is of quality, we have never had a better opportunity in our state or in our country than we have right now.

As I said: I think Alberti's presentation here is very typical of the pro-PARCC side -- both in content and in tone. So I think it's well worth breaking down:

- If the PARCC isn't really like the practice test items released on the parcconline.org website, what in the hell is the point of releasing them?!

Over and over, the pro-PARCC side told us that these tests were going to be the most transparent assessments we've ever had. The PARCC consortium has bragged about how many items they are going to release after the tests. They themselves said the sample items they released have been thoroughly vetted:

Washington, D.C. - August 19, 2013 - The Partnership for Assessment of Readiness for College and Careers (PARCC), a 19-state consortium working together to create next generation assessments, today released additional sample items for both English language arts/literacy and mathematics. The sample items show how PARCC is developing tasks to measure the critical content and skills found in the Common Core State Standards (CCSS).  The sample items have undergone PARCC's rigorous review process to ensure quality and demonstrate the content that will be on the assessments in 2014-2015. 

"We are developing a high quality next-generation assessment system to measure the content and skills students need to succeed at the next level and in college and career," said Massachusetts Elementary and Secondary Education Commissioner Mitchell Chester, chair of the PARCC Governing Board. "The sample items we are releasing today will help to provide a clear signal to educators, parents, and others about the rigor of the test." [emphasis mine]

Again: this is PARCC themselves saying the sample items are representative. Now, all of a sudden, we're supposed to believe they're not?

At this point, we've had lots of parents and teachers go to "Take the PARCC" events at their schools. Many more have gone to the PARCC website to view the items. And many, many stakeholders -- on the basis of samples the PARCC people themselves say are representative -- have come away genuinely wondering how anyone could think these tests were in any way "better" than previous standardized tests.

If Alberti or anyone else would like to make the case that the PARCC is really, truly better, they'd do well to start with explaining how the items we all can see and judge for ourselves provide any evidence for their claims.

- There is no evidence test prep for the PARCC is a superior form of instruction compared to test prep for any other standardized test.

I won't claim special expertise in learning and cognition theory, but I've had enough training and experience to know that what we're really dealing with on standardized tests like the PARCC is what is often referred to in education research as "transfer."

When a student transfers learning, they are finding ways to take their previous learning and apply it to new contexts. When a student takes the PARCC, for example, we are looking to see that student take their previous knowledge of language arts or math and use it to solve a novel problem. In other words (and this is a gross simplification): if we've taught a kid that 5 bags with 3 apples each is 15 apples, will he know how to divide those 15 apples into only 3 bags? Can we take a concept the child has learned and get him to use in in a unique way?

Again, I claim no authority in this area, but I do know this for sure: transfer is really, really tough. The context in which learning takes place matters enormously; further, the proposition that the measure of a student's ability to transfer knowledge is reflective of his teacher's or school's ability to deliver instruction is highly dubious.

One of the claims about the "failure" of public education that always cracks me up is when some corporate stooge gets up in front of a crowd and tells some story about how he can't find workers to do his particular task in his particular way. It never occurs to these guys that the schools can't possibly anticipate every context in which learning may need to be transferred. They want workers who can walk into their businesses immediately and start doing their jobs; sorry, but that's just not reasonable. You have to train people in the context in which you expect them to perform.

Which gets us back to test prep. If you really want your students to pass a test, you're going to instruct them on how to pass it. You're not going to waste time instructing them in different contexts; the only context you care about is the one that matters.

The PARCC is a test; it is its own context and, as such, its results are subject to limitations on transfer as much as any other test. Pretending it mirrors the "real world" better than any other standardized test flies in the face of everything we know about human learning and common sense. Which bring me to my last point (for now):

- We don't even know if the PARCC is measuring the effectiveness of instruction.

Give the PARCC folks credit: they published a brief that admits they really don't know if the PARCC is "instructionally sensitive" or not:

One area of interest as the PARCC assessments are developed and implemented is instructional sensitivity. In particular, PARCC is interested in considering studies to examine the instructional sensitivity of items and assessment components as the CCSS become more widely and effectively implemented over time. 

At one level, instructional sensitivity has a relatively straightforward definition. According to Popham (2007), “A test’s instructional sensitivity represents the degree to which students’ performances on that test accurately reflect the quality of the instruction that was provided specifically to promote students’ mastery of whatever is being assessed” (p. 146). In a recent in- depth article on instructional sensitivity in Educational Measurement: Issues and Practice, Polikoff (2010) traced the meaning and origins of instructional sensitivity, connecting it to instructional alignment, opportunity to learn, and the criterion-referenced testing movement that began in the 1960s. Chen (2012) also summarized the relevant literature, and based on that review, provided a graphical depiction of the relationship between instructional sensitivity, instructional validity and curricular validity shown in Figure 1. (emphasis mine)

You can click through to see the graph (and welcome to my world these days). But I'll summarize: the relevant question is whether the PARCC is reflecting school-based effects or something else. And the answer is: "we don't know."

Morgan Polikoff and I have tangled a bit on social media (he didn't much care for my bluntness in calling institutional racism what it is). But he is a smart guy and he makes a very good point here:

The success of schools under standards-based reform depends on the performance of students on state assessments of student achievement. The reform’s theory of change suggests that, with high-quality content standards and aligned assessments, teachers will modify their instruction to align to the standards and assessments, and achievement will rise (Smith & O’Day, 1990). Dissatisfied with the performance of the No Child Left Behind Act (NCLB) to this point, researchers and policymakers have begun to advocate for and work toward common standards and assessments that would, presumably, be better aligned and more likely to encourage teachers to modify their instruction. However, there has been little discussion of the last part of the theory of change—that changes in instruction will result in changes in performance on assessments of student learning (Popham, 2007). Indeed, though millions of dollars are spent on developing and administering assessments of student learning each year across the 50 states (Goertz, 2005), there is little evidence that these assessments can detect the effects of high-quality instruction. Instead, it is assumed that assessments aligned to standards through one of the various alignment procedures will necessarily be able to detect instructional quality. This key omitted property of an assessment is called its instructional sensitivity, and it is the focus of this paper. [emphasis mine]

This is an interesting read and well worth some quality nerding-out time. Polikoff's point is that if we're going to use the test to judge instruction, we ought to have some idea as to whether the results vary as instruction varies.

What this PARCC memo above is tacitly admitting, however, is that we have no evidence that the PARCC measures the efficacy of instruction. In discussing avenues for future research, the memo is in effect admitting we have a gap in our knowledge about the PARCC: we don't know whether its results are sensitive to instruction. And yet the pro-PARCC folks keep making claims as to its superiority as an accountability assessment. Wouldn't it make sense to verify this claim before making it?

Look, I'm not trying to single out Alberti; she's only repeating what all the pro-PRACC types have been saying. Over and over they tell us PARCC is a "better" test, measuring "real" learning. But the truth is there is little to no evidence to back up the claim that the PARCC is a "better" test.

When a parent goes to the PARCC website or a "Take The PARCC" and comes away with doubts as to whether these tests are appropriate and necessary, she should be met with a better response than "Trust us!" The burden of proof is on the pro-PARCC faction. Either back up your claims, or stop making them.

First, we need to test the test to see if it's ready to test you.

How The Teachers Union in Jersey City May Have Started To Win

Every once in a while, a group of teachers comes together and says: "Enough."

It happened a few couple of years ago in Chicago, when Karen Lewis rallied her fellow teachers and took on Rahm Emanuel and his rubber-stamping board of education. And now it appears to be happening again...

In, of all places, Jersey City -- thanks, in part, to the leadership of Ronnie Greco.

Greco is the president of the Jersey City Education Association, the local teachers union and the largest local affiliate of the New Jersey Education Association. I reached out to Greco earlier this month; we eventually had a lengthy conversation, followed by email exchanges, about what is happening in his city's schools.

The size of the JCEA alone is reason enough to pay attention what is happening in the district. But I believe teachers and union leaders all over the country can learn lessons by studying how Greco and his fellow Jersey City teachers are fighting back against the political forces that have sought to undermine their association.

Let's get some context first:

The Jersey City Public Schools have been under state control since 1989, making it the first city in the country where local citizens lost control over their schools. As I reported back in 2012, then-Education Commissioner Chris Cerf colluded with then-Councilman -- and now Mayor -- Steve Fulop to install Marcia Lyles as the state superintendent of schools.

Cerf pushed the ethics envelope (once again) when he met with in private with board members who had been elected, but had not yet taken office, to push for Lyles to get the job. Both Lyles and Cerf are graduates of the infamous Broad Academy, a pipeline for getting corporate-styled "reformers" into key education leadership positions. There's little doubt that Broad considers the leadership of New Jersey's second-largest school district to be a valuable trophy.

But Broad graduates are notorious for heavy-handed, top-down, and often failed leadership. Does Lyles fit this pattern? Is she, according to Greco, qualified to lead the district?

"Absolutely not," says Greco. "She came from failure; that’s what she knows. She came from what was designated as a failing school district in Delaware. Do I believe in the term 'failing school district'? No, that’s a label that the government drops on us based upon benchmarks that they set and they set them higher and higher every year and keep changing the method in which you have to achieve those benchmarks. 

"But as a person? No. She has no personality, she has no social skills, she has no people skills, from the folks I speak to up in the central office she has no managerial skills. She’s a puppet of [NJ Governor] Chris Christie. She was a puppet of [former schools chancellor] Joel Klein over in New York City. She’s one of these people… she’s a suitcase traveler, bouncing from state to state, tacking on a nice pension. She has no connection here."

Unlike Lyles, Greco has deep roots in his community. His father was a teacher in Jersey City for 40 years; Greco taught for 17 years himself before he took over the leadership of JCEA. And it's Greco's connection to that long institutional memory that gives him a heightened insight into what is happening to Jersey City's schools, and what it takes to fight back. For example:

The JCEA has been working without a contract for two years. According to Greco, the last offer from the board was "three years of zero increment," which, in the lingo of teachers contracts, translates to no raises. That may have been defensible back in the recession; however, New Jersey's teachers have been traditionally underpaid compared to the rest of the region. And it's not as if Jersey City's teachers, living in one of the most expensive labor markets in the world, are asking for anything more than a fair increase.

After repeatedly being rebuffed by both the board of education and the state superintendent, and after repeated delays from the Public Employment Relations Commission (PERC), things finally reached a tipping point this past fall. The issue was "Report Card Night," a three-times-a-year event similar to many districts' back-to-school nights.

"In my 17 years as a teacher, Report Card Nights were always conducted from 6:30 to 8:00 PM," says Greco. "And there was one in September at back to school night, and one in November, and one in April. It was always 6:30 to 8:00 PM. And in terms of working an hour-and-a-half in the evening, that gave us specific days in the calendar where we would get out an hour-and-a-half early.

"We're in contract negotiations as you know. One of the board's proposals reads as such: 'Change report card sessions from 6:30 to 8:00 to 6:00 to 8:00.' That's a proposal. So what the superintendent did was on Friday, October 18, 2013, she calls me. And she said: 'I am changing it to 6:00 PM.'

"And I said, 'Dr. Lyles, first of all, it's negotiable. Second of all, that's one of your proposals. And that's how we get an hour-and-a-half off on other days; it's a trade off with the time. It's an established past practice here from my forefathers. It's a long-standing tradition.'"

But Lyles refused to negotiate with Greco, and she refused to budge on changing the time. "I met with the superintendent five different times," says Greco, "and her response was always very simply: 'I am not discussing it.' End of story, didn't want to entertain me, that was it."

Again, this is a hallmark of Broad-style leadership, which draws from the "Cult of the CEO" and creates an autocratic and dismissive atmosphere in school districts. Lyles wanted her Report Card Nights when she wanted them, past practices and her teachers' ability to negotiate be damned.

So Greco filed a grievance in October of 2013. PERC eventually confirmed the meeting start time was negotiable. Even still, in an effort to show good faith, Greco and his team initially decided to comply, and report at 6:00.

"November 2013 came along and we complied: the law in NJ says you comply, then you grieve. So we complied. We came in at 6:00 PM in November. We came in at 6:00 PM in April because PERC never came back with a decision. Finally, in the summer of 2014, this past summer, we got an answer from PERC: 'Since you are in negotiations, you should negotiate it.' So that was enough for me; I decided, 'We're coming in a 6:30 and we will negotiate it.'

"I hadn't spoken to the superintendent since August. I said to one of her chiefs we were coming in at 6:30 this time, and he said, 'Oh, no, it's 6:00.' And I said, "No, it's 6:30." And she said, "Oh, no, Dr. Lyles wants 6:00 PM." In my head I thought, 'Screw this. PERC came back and said negotiate it. And the superintendent still refuses to negotiate.' So, we printed up 4,000 signs, and I thought: 'This is a test run for a strike.'"

I had to stop and ask Greco at this point why he didn't just give in again. It seems like a trivial issue; why fight over something so small?

"It’s such a minor thing," admits Greco. "I mean, 6:00 PM was something in good-faith negotiations. We could have agreed to come in at 6:00 PM, just because we could get something as a trade-off that we want. The argument with most of the teachers that I speak with is and my union reps is: 'We’re already here.' People are already staying after school until 6:30. Why not just start the session at 6:00 PM because we’re already here?

"But it’s the manner in which it was done. 'We’re not doing it. We’re not honoring the negotiations. Forget that it was one of our proposals. To hell with what the PERC says. To heck with a state agency. To heck with the union.' So that’s why people are just fed up with this superintendent. She just does whatever she wants."

And so, in November of 2014, the teachers stood their ground and came in at 6:30, as they had done for years. And that's when Lyles really began to lose control of her district:

"The report card nights were in November," says Greco. "The superintendent conducts regular meetings with all her principals. Whenever there's a principal's meeting, there's always a dozen or so principals that give me a call or text and tell me what was just discussed in the meeting. So the PSA (Principals and Supervisors Association), the principal's union, their president was adamant that the teachers be reprimanded, and that we should be written up and we should be docked pay. Because the principal's union president, who was a principal, is up the superintendent's backside.

"But many of the people in the room, the principals, were saying: 'We're not writing the teachers up.' The superintendent got very angry because now these are her soldiers who are going against here. And on the report card night -- because it was a cold night, the temperature had dipped into the 30s -- numerous schools had coffee and hot chocolate, and the parent councils were out there with food and bullhorns to show their support for us. So it was kind of a setback to the superintendent.

"And then two or three weeks went by and nothing was done yet, and she told the principals: 'I want them written up!' And the principals were just refusing to do it. So finally, in December, an edict was issued by Maryann Dickar -- she's the superintendent's right-hand lady, a transplant from New York, who came to Jersey City with a $90,000 raise -- and she told the principals: 'By December 12 you must write these people up.' So the principals were really upset, the superintendent provided them with a fill in the blank form letter. It was a letter of insubordination. The principals had to sign it.

"Now most principals that I speak to... those principals gave out the letters, put them in one folder, so technically they're on file at the office. But most principals have not put them in the teachers' individual files. What it was was a letter of reprimand. I mean big deal."

Understand that is how the state-run districts in New Jersey are managed these days. As in Newark, a war has broken out between the state superintendent (and her closest lieutenants) and the rest of the district. And, as in Newark, both the principals and the teachers are standing up and fighting back.

"Most teachers have had it," explains Greco. "People have all the pressures with the standardized testing. And the dilapidated facilitates and the superintendent. And we have a RAC [Regional Achievement Center] monitor, Cathy Coyle [assigned by the state to oversee the management of JCPS], she's constantly fighting with people. Coyle is collecting a full pension from the state as well as a full salary from the State Department of Education, to monitor Jersey City.

"So people just don't care anymore. The superintendent is so disrespectful, she doesn't speak to principals, she doesn't speak to teachers, she doesn't speak to parents."

And here's where the story gets most interesting. Because Greco and JCEA decided it wasn't enough to simply stand it opposition to the state; they had to get people on to the board of education who would be willing to negotiate in good faith. 

One of the reasons the board had become so intractable was that candidates were being funded by billionaire "reformers," including hedge fund manager David Tepper, who stood behind Chris Christie's education agenda. Greco knew he couldn't match Tepper's money; he could, however, rally his teachers and parents, many of whom were his long-time neighbors. And, thanks to the union's work, a slate of three candidates supported by the JCEA clobbered the opposing slate in this past fall's election.

"We were fortunate to team up with NJEA and Garden State Forward," says Greco. "Obviously we didn't have a big bank account. I was looking at the ELEC [New Jersey's campaign finance reporting system] report today. Parents for Excellence [the opposing slate] has donors from all over the country: pharmaceutical companies and bankers and so on.

"But we did a good job -- I keep telling my members, we have to pat ourselves on the back. The community really came out and helped us. I think that's telling of the community's attitude, because we had so many parents that helped us, volunteered on election day, and they knew we didn't have the funds to pay people. And they said, 'Give me that t-shirt, I want to campaign, I want to help put these people in.' And then teachers, paraprofessionals, secretaries, custodians -- everyone who helped us.

"That's why we believe we were victorious, because we had the boots on the ground, despite the state RAC monitor Cathy Coyle openly campaigning for Parents for Excellence, donning a shirt and handing out literature.”

Greco's ability to mobilize the community in favor of his teachers has not gone unnoticed -- particularly by Mayor Fulop. Even though Fulop was involved in bringing Lyles to Jersey City, and even though he has previously backed anti-union candidates, Fulop has started to open up to the legitimate concerns of the JCEA.

"I’m going to say we both got an education," says Greco. "He got an education by hanging out with me, and I got an education from being with him and grasping what his viewpoint was, what his perception of what the Jersey City Schools were. And he got a bit of an education as to how I was looking at him and viewed him. I thought he was very radical and wanted to come in here and just turn the school system upside down. And he believed I was here to just keep what he thought was the status quo. 

"Mayor Fulop has some very progressive, yet traditional ideas on how to approach addressing the needs of Jersey City’s Public School system. He has certainly included the JCEA in those conversations.”

"The board president, Sangeeta Ranade, and Marcia Lyles just seemed to have gone rogue. I have observed how they just do not listen to or entertain what the Mayor has to say. It is clear they do not listen to or heed the Mayor’s advice. And then I would scratch my head," laughed Greco.

"But then I saw some public displays of what was going on, but even then I would say, 'Well, this is just a smokescreen.' And I even told the mayor that. And he said, 'Ronnie, it’s not a smokescreen. They don’t listen to me.'

"The mayor has said: 'I want both sides to sit down. This has gone on long enough. Settle the contract.' But they don’t listen to the mayor. Since then the board president has blasted him in the newspapers. 'I don’t listen to you. You’re the mayor; I’m the board president.' So there’s no respect there or collegiality among these people.

"I mean, when you insult parents at board meetings, you refuse to answer them at board meetings, when parents come up to speak and you’re on your phone, they really are just disrespectful people."

Is this disrespect due, in part, to the lack of local control in Jersey City over its schools? Would ending the state's two-decade reign change things for the better?

"Yes, I do believe it would change," says Greco. "I believe that the operation of the system would be more orderly. I think that the number of employees in the management would shrink. When we were locally controlled, the BOE office was in a small building on Erie Street. Now it’s in an old factory building. It’s 8 floors with hundreds and hundreds of people there. So as the state came in, it’s grown over the years.

"Every governor’s friends get sent up here. The law firms get contracts up here. I’ve discussed this with Mayor Fulop. We wouldn’t have 11 or 12 law firms on retainer. We wouldn’t have all these contractors and consultants and you name it."

Greco and the JCEA may have helped to elect a less stubborn slate of board candidates, but their work is far from over. No one in the local ever thought electing a less radical school board would immediately lead to a contract. But Greco believes there is now hope for a settlement, and a chance for real reform in Jersey City's schools:

"I’m a little optimistic. The new board was seated last night, they have a big task ahead of them, but I’m optimistic we’ll make some progress. Because I know Jersey City. And I know they have money squirreled away and their offer of three years still being frozen… they can do better than that. There’s where I’ll leave it at."

And perhaps that's the best lesson that Jersey City's teacher union and Ronnie Greco have to teach other locals in this state and the nation: sometimes you need to stand tough, and sometimes you need to make alliances. It's a tricky business; again, few did it better than Karen Lewis in Chicago.

But the JCEA may well be a source of hope: a new, shrewd unionism -- one that knows when to open a hand, and when to clench fist -- may be replicable. If JCEA gets a new contract this year, Greco may, indeed, offer us proof that the victory of the Chicago Teachers Union wasn't a fluke.

And that would be good news for teachers, good news for students, and good news for our public schools -- in Jersey City, and everywhere else.

Ronnie Greco, JCEA President.

This blog proudly supports all of the local associations of the NJEA and AFTNJ.

Why Does the Public Hate Standardized Tests?

The New Jersey Education Association, the state's largest teachers union, released a bombshell of a poll yesterday; click through to see the raw results for all voters and just for parents. My guess is that once we clear away the snow, we're going to be talking about these results for a long time.

nj.com has a breakdown:

In conjunction with D.C. based Mellman Group, NJEA administered two polls in December asking a range of questions related to standardized testing. One poll surveyed 800 likely voters, including 200 parents. A second poll added 200 more parents to the original 200, creating a 400-parent poll. 

Both polls reflected the same general opinions on testing, though by different percentages. The poll of 400 parents found: 

• 81 percent of parents are concerned that “teachers are forced to teach to the test.” 

• 80 percent are concerned that “too much of the school year is spent preparing for standardized tests.” 

• 78 percent want to limit the number of hours of testing. 

• 78 percent say testing “causes stress for students.” 

• 77 percent are concerned that testing “takes time and money from other educational priorities.” 

A majority of parents polled, 82 percent, said they want legislators to pass a testing "Bill of Rights," requiring transparency on high-stakes testing, including how much they cost taxpayers and how student data will be used. Parents also want the ability to opt their students out of tests — 66 percent said they support having a parental right of refusal.

I've noticed some folks already picking nits at the questions, trying to show they're biased (for the most part, I don't agree that they are). But even if you see a slant here, the resulting numbers are still astonishingly high. When four our of five parents are telling us that testing has become a pernicious force in their children's lives, something is clearly amiss.

Of course, this has been bubbling up for a long time and all over the country. Parents are increasingly worried that testing has taken over the lives of their children -- so much so that politicians who have traditionally relied on standardized tests to push their particular brand of "reform" are now forced to deal with the backlash against those tests.

So what's going on here? Why such a pronounced and forceful rejection of a testing regime that admittedly has been expanding, yet has actually been in place for a good numbers of years?

Part of what's going on, I think, is that the tests are a proxy for general discontentment with how the lives of our children are being lived these days.



Race To Nowhere remains, for me, one of the best indictments of the "achievement culture" that has consumed the lives of our children. I think there's a strong sense out there that kids are being run through a gauntlet, and if they don't survive, they aren't deemed worthy of entering the middle class and leading decent lives.

Standardized tests don't measure learning so much as they rank and order children according to a social construction that is based on the notion that we must have winners and losers in our society. 

Neo-liberals have warped our sense of social justice: we now believe in "level playing fields," rather than making sure everyone has a chance to contribute to our society and be guaranteed a life of basic human dignity. So long as poor people of color have an equal chance to work on Wall Street and pull down an obscene amount of money at the expense of everyone else, we can call our society "just."

As I've said now a thousand times: we have millions of people in this country doing hard, sometimes dirty, sometimes dangerous, often backbreaking and monotonous, but necessary work. These folks are working hard and playing by the rules, but they can't even afford to live in decent housing, let alone take a vacation every now and then.

Ranking and sorting students does nothing to address this core problem in today's America. All it does is ratchet up the pressure on children, whose parents understand that a life without a college degree increasingly means a life of misery. It's become clear to many (even if they don't articulate it in the same way I am doing here) that the standardized test is the lynchpin for this system.

And it's also become clear that the defenders of the real status quo refuse to acknowledge the truth about this state of affairs. Here in New Jersey, our Department of Education (and their willing mouthpieces) keeps telling us that standardized tests like the PARCC are really for the benefit of the students. Education Commissioner David Hespe, for example, swears the tests are really all about helping the wee ones:

Hespe said that state and federal regulations require at least 95 percent of students take the exams, or districts could potentially lose some undefined funding. Most of all, Hespe said, he wanted to point to the importance and benefits of the state assessments for individual students and their schools.

“The PARCC assessments will, for the first time, provide detailed diagnostic information about each individual student’s performance that educators, parents and students can utilize to enhance foundational knowledge and student achievement,” Hespe wrote. [emphasis mine]

 Too bad his own Assistant Commissioner, Bari Erlichson, disagrees:

ERLICHSON: In terms of testing the full breadth and depth of the standards in every grade level, yes, these are going to be tests that in fact are reliable and valid at multiple cluster scores, which is not true today in our NJASK. But there’s absolutely a… the word "diagnostic" here is also very important. As Jean sort of spoke to earlier: these are not intended to be the kind of through-course — what we’re talking about here, the PARCC end-of-year/end-of-course assessments — are not intended to be sort of the through-course diagnostic form of assessments, the benchmark assessments, that most of us are used to, that would diagnose and be able to inform instruction in the middle of the year.

These are in fact summative test scores that have a different purpose than the one that we’re talking about here in terms of diagnosis.

Kudos to Erlichson, because that is exactly right: standardized tests like the PARCC are accountability measures; they are not designed to help inform instruction for students.

One of the more ignorant arguments I've been hearing lately about standardized tests runs along the lines of: "But we've always had tests! These are just better tests, because they're better aligned to 'real-world' learning objectives! So they can help teachers create better instruction for their students!"

This is a poor argument for several reasons. First, a standardized test is of little use to a teacher who is instructing non-standardized students. 

Any teacher who went through a decent preparation program has seen some sort of variation of this graphic:

This is from Carnegie Melon's Eberly Center for Teacher Excellence and Learning Innovation, but it could have come from any number of sources, because it's such a basic idea. A teacher sets a learning objective, creates instructional activities, and designs assessments to see if the students actually learned what they are supposed to learn. It's a dynamic system: the assessment provides feedback on whether the objectives were well-designed and the instruction was well-executed. But the objectives and instruction also inform the assessment. Everything works together, and a good teacher is constantly adjusting and refining all parts of the system.

But a standardized test can't be changed; it gives very little useful information as to how a teacher should adjust instruction and objectives. As a practical matter, it's next to useless because by the time the test results get back to the teacher the kids have usually moved on to another grade. But even if the results came back instantaneously, the assessment wouldn't help inform instruction much because the test can't change as instruction and objectives change.

That's not to say that these tests don't have a role. I'm probably going to piss off some folks who normally agree with me, but I actually do think these tests are necessary as accountability measures. We should be able to judge at some level whether or not the system is working; certainly, I and others who attempt to judge the effectiveness of education policies benefit from having some sort of metric of student learning to help us make our determinations (so long as we use them appropriately).

But it's not really useful to anyone to have an accountability measure so overwhelm the system that it takes on unwarranted importance. Which brings us to the second reason why the pro-PARCC argument doesn't hold up: we don't know whether these tests are truly "better" or not.

Last year, we found that the New York State standardized exams were loosely aligned, in a convoluted way, to a vague learning outcome: getting a "C+" in a freshman course in math or a "B-" in language arts at a number of selective colleges. Getting good grades is certainly better than getting poor ones, but is anyone prepared to argue that it is any more of a "real world" outcome than the tests themselves?

One claim made about PARCC is that it measures "higher order thinking." From what I've seen, however, what's really being measured is whether a student can see past the tricks and distractors thrown up by the test to pick the correct answer. Test items like these are excellent for teasing out a normal distribution of scores:

But it's quite a stretch to claim this is "better" than the standardized tests we had before, which yielded the same distribution. Do we really believe the NJASK was giving a distorted view of how students were performing relative to each other? 

I don't have a problem with standards, and I don't have a problem with setting them high. I'm fine with testing, and I'm fine with the appropriate use of standardized tests. We certainly can use tests to inform decisions -- not compel, but inform -- about teachers and schools and policies. The great thing is that we can do this without an overly onerous or expensive testing regime simply by understanding a few basic statistical principles.

But even if we make our standards and tests truly "better," we're still going to have to solve this:

Nearly 70 percent of the variation in New Jersey school-level SAT scores can be explained by  levels of student economic disadvantage. Putting pressure on kids to perform better on "better" standardized tests isn't going to change this. Setting higher standards isn't going to change this.

And maybe that's why people are getting fed up with standardized tests: they are a distraction that keeps us from doing what really needs to be done.

Testing, testing, 1, 2, 3...

Sunday Night Music: The Dry Cleaner From Des Moines

Why are we getting socked by the latest in a series of "Blizzards of the Century" on a Monday? Why did Northwestern drop its third Big Ten squeaker? Why is my right hamstring twitching yet again?

It's just luck -- right, Joni?



Sadly, all three in the band here -- Don Alias, Jaco Pastorious, and the great Michael Brecker -- left us too early. Pastorious's story is especially sad.

This is from Mitchell's seminal album, Mingus, a collaboration with the great bassist & composer who I have been rediscovering this past month.



Stay safe this week, folks.

One of the many reasons I never moved into administration.

Chartery AWESOMENESS!

Diane Ravitch points us to a recent paper by Atila Abdulkadiroğlu, Joshua Angrist, Peter Hull, and Parag Pathak that finds positive effects from charter school takeovers in New Orleans and Boston. No doubt, this will be presented as more evidence that "relinquishment"* is the new model for success in urban school systems.

I look at it, however, as more evidence that we are rushing headlong into a series of "reforms" without thinking fully about the consequences of our actions.

Let me start by saying this is a smart study written by some very smart people. I've just started a book by Angrist on econometrics, and he is obviously an accomplished economist whose work is and should be taken seriously.

In Angrist's telling, econometrics pretty much boils down to one central idea: ceteris paribus, or "hold other things constant." In other words, if you want to find out if a particular policy is going to work or not, you have to make that policy the only thing that changes between a treatment and control group. Econometrics is largely about using statistical tools to do just that.

In the case of this paper, the authors use a student matching technique, combined with a basic statistics technique called regression, to hold constant the differences in students who went to charter schools that took over public schools in New Orleans and Boston, and those who stayed in the public system. The paper finds there are test score gains for those who went to the charters.

It's been my experience that charter cheerleaders often rush to research like this and immediately claim victory. "See?" they exclaim. "We need to let charter proliferation thrive! The children can't wait another minute!" 

But it's also been my experience that if you press them as to why the charters in these studies are performing better on tests -- which is the central policy question in the debate over charter expansion -- they suddenly get all wishy-washy. "We do more for less!" is often the best you can get out of them. "We aren't tied to the bureaucracy!" is another "explanation." Sometimes you get a vague smack down of teachers unions.

But I have yet to hear a charter cheerleader give a cogent explanation as to why a gain found in any number of studies that is attributable to charter schools is replicable on more than a limited scale. And I'm afraid that studies like this one, as interesting and ingenious as it is, do little to get to this fundamental question.

I actually have a few nits to pick with this paper. First, like so much econometrically-based research in education, there isn't much context here for understanding the effect sizes in question. The outcomes are based standardized tests scores, which are measures with their own inherent limitations and flaws. The effect sizes are expressed in standard deviations; thankfully, the authors here avoid the indefensible practice of converting those into "x days of learning." But, as Kevin Welner has noted, showing a "statistically significant" gain isn't the same as showing a practical gain.

Abdulkadiroğlu et. al. report that "Attendance at RSD takeover charters is estimated to increase math and ELA scores by an average of 0.21σ and 0.14σ, respectively, per year enrolled." (p. 14)** Translating from  geek-speak, that's equivalent to moving from the 50th percentile to the 58th in math, and the 50th to the 56th in English language arts. 

In the conclusions, they state: "In practice, cleaning up the non-charter counterfactual substantially boosts our estimates of RSD takeover effects on math, from about 0.21σ to 0.36σ, while leaving the smaller ELA estimates largely unchanged." That raises the math percentile to the 64th percentile, up from the 50th. 

Sure, that's not anything to ignore. But it's hardly enough to close the "achievement gap" we hear so much about. Are we really prepared to rush into radically changing how we structure our schools on this basis alone? Hardly seems prudent to me.

Second, this is yet another paper that doesn't disaggregate its measures of student characteristics. Put another  way: it shoves kids who are probably quite different into the same box. For example, students are classified as qualifying for free or reduced-price lunch, or not. But as both Bruce Baker and I have harped on over and over, the distinction between free lunch eligible (130% of the poverty line or below) and reduced-price lunch eligible (130% to 185% of the poverty line) matters, especially when you're in a community like New Orleans where nearly all the students are in economic disadvantage. 

Same with special education: a student with a speech impediment is not equivalent to one with a profound cognitive impairment. I understand it's often hard for researchers to get data to do this, but at the very least it should be acknowledged as a methodological limitation.

Third: the very first thing I did when I opened up the paper, after reading the abstract, was look for any description of the student populations of the schools, and, critically, whether that population had changed after the charter school team took it over. I couldn't find it.

The authors acknowledge that students in economically disadvantaged communities are often quite mobile, and that's certainly true. But why these students might leave one school or another matters, and what the school looks like after they leave matters. Why?

We know that peer effect is real. We know that there are "successful" charter schools that engage in patterns of significant student attrition. And if charters are shedding students in an effort to improve the peer climate of the school, that's a problem: there aren't enough "non-disabled, non-poor, fluent English speaking females" to make charter proliferation a viable large-scale strategy for urban school improvement.

Which brings me to my biggest problem with this paper and, for that matter, much of the econometric education research I read these days. Because, too often, I see our brilliant economist friends jump to conclusions based on qualitative evidence that is, to be charitable, incomplete. Take page 5 of this paper:

The No Excuses model for urban education is characterized by extensive use of tutoring and targeted remedial support, reliance on data and teacher feedback, a curriculum focused on basic skills, high expectations from students and staff, and an emphasis on discipline and comportment.

Is it? Just say "No Excuses," and the Chartery AWESOMENESS kicks in, and everybody starts learning? 

That's certainly what the charter sector would like us to believe: if only we stop accepting "excuses," and set "high expectations," and get back to "basic skills," (even if stuff like the arts is especially good for low-income children), urban education will be fixed.  

This sort of thing reminds me of Peter Pan telling the audience to clap harder when Tinkerbell lays dying. Sure, believing that you can change the lives of urban students is important, but the obvious question is: now that you "believe," what's next?

Let me put this another way:

Here's a very much incomplete graph showing some of the reasons why the charters in this study, and the CREDO studies, and in other charter school studies, might get the effects they show. In blue I've got a host of reasons why "successful" charter schools might "succeed" that have nothing to do with their "charteriness."

If, for example, a charter gets a peer effect because its student population is different from the school to which it is compared, that's not particularly "chartery": we could conceivably set up public schools that did the same thing, and they would enjoy those same advantages when comparing test-based outcomes. Likewise, a test-prep curriculum is not the exclusive province of charter schools; we could drill all kids on bubble computer test prep, regardless of whether they're in charter schools or not, and likely get better test scores.

We know many "successful" charters spend more on their students, allowing for longer school days, smaller class sizes, and wrap around services. To their credit, Abdulkadiroğlu et. al. include some data on this in the paper -- but it's not really helpful unless we adjust for differences in student populations in the schools we're comparing.

My point here is that in a real ceteris paribus analysis that tests the proposition that "charter schools do more with less," we'd be holding everything I've got in blue constant so we could isolate the independent variable we really care about: Chartery AWESOMENESS. Then we could get a better sense if the effect of Chartery AWESOMENESS is like this:

Or, perhaps, more like this:

This is where we have to put our focus next. Because if the charter effects we're seeing are due to factors that have nothing to do with the Chartery AWESOMENESS of a school...

Why don't we just implement them, if we can, in all public schools?

Again: this is a smart study and we should pay attention to it. But I'd respectfully suggest to the economists who continue to produce these pieces that it's well past time they began to shift their research focus. Yes, some "successful" charter schools get gains.

The critical question now is: "Why?"

You mean "Chartery AWESOMENESS!" isn't explicit enough?

* Isn't it funny that, if your'e an academic, it's OK to state your academic credentials if you produce research that favors the charter sector's cheerleading; however, if you put your credentials on a report that stands opposite to the company line, you can be brought up on ethics charges. Golly, you'd almost think there was a double-standard or something...

** Don't make the mistake of thinking this "yearly" gain can be in any way extrapolated across the entire time a student is enrolled at a charter school; in other words, there's no reason to believe those gains will always be the same each year, so that in five years the math gains are over one standard deviation. That's just not how these things work; as the paper points out numerous times, gains are usually stronger in the first year, then start to plateau.

Shorter @NYGovCuomo: "Go Along With My Reformy Nonsense, Ignore My Funding Failures!"

I finally sat through New York Governor Andrew Cuomo's combination budget/State Of The State speech from this past week. There were a few huge groaners that need to be debunked, especially the lie -- yes, I said it -- that 32 percent of New York's prospective teachers failed a "bar exam."

But before I get to any of that, let's get right to the heart of the matter:

Governor Cuomo proposed a deal that is disturbingly close to an act of extortion. If everyone goes along with his education agenda, he'll add some more state aid for New York's schools. What's his program?

- An expansion of test-based teacher evaluations, even though everyone who knows anything about this topic says it is fraught with danger and will not work.

- An expansion of charter schools, even though everyone who looks at the charter sector honestly knows the gains they lay claim to are highly suspect.

- "Anti-creaming" charter school legislation, even though everyone knows charter schools have calcified patterns of segregation that will almost certainly be impossible to break.

- Merit pay, even though everyone who has studied it knows it has never worked before and certainly will not work now.

- The destruction of tenure, even though everyone who bothers to look at the empirical evidence comes to the conclusion that tenure it is certainly not an impediment to student learning, and is actually a benefit to the taxpayers and students that keeps our schools from becoming patronage mills.

- "Turnaround" plans for "failing" schools, even though everyone who knows the history of the school closure model knows it doesn't work.

- Mayoral control of schools, even though everyone who knows anything about New York City's schools understands that pinning hopes on mayoral control is a joke.

"Go along with all my unsupported, invalid malarky," barks the governor, "and I'll pitch in some more dough for the schools!" How much, exactly, Governor?

The original budget, according to Cuomo, called for an increase of $377 million. But if, and only if, New York goes along with this reformy nonsense, Cuomo will raise state aid by $1.1 billion. Which might sound great at first, until you realize one thing:

Cuomo's proposed $1.1 billion increase in state aid is nothing compared to the $5.9 billion New York State is behind on its own funding formula!

Let's have the good folks at the Alliance for Quality Education explain why Cuomo's proposal is a sick joke:

“The findings are clear and shocking. Governor Cuomo has failed to provide the leadership to uphold the state’s constitutional responsibility to provide every student with a sound basic public education. The state has a $6.2 billion surplus heading into 2015 there is no excuse to continue to make our students lose out,” said Billy Easton, Executive Director, Alliance for Quality Education. 

Currently, the state is behind $4.9 billion in Foundation Aid and $1 billion in Gap Elimination Adjustment (the GEA is the result of cuts made in 2010 and 2011).  Multiple court cases are now being brought against the state for its failure to fulfill its constitutional obligation to students, including the Small Cities Case for which David Sciarra, Executive Director of the Education Law Center is now serving as co-counsel. 

“This report demonstrates why students and parents from small cities across the state are suing Governor Cuomo and the State of New York for violating students’ constitutional rights,” said Sciarra. “We are prepared to demonstrate at the trial in December the educational harm being caused by the state’s continuing failure to deliver adequate funding to high needs schools.” [emphasis mine]

That would be the Maisto case, brought by many of New York's small cities against the state. These cities' schools have been grossly underfunded to the point that New York's poorest districts are suffering a funding gap of historic proportions.

As I wrote previously: New York has gone through a painstaking, decades-long process to bring school funding equity to the state. Panels of experts carefully constructed a formula the state itself said was necessary to provide New York's students with a "sound basic education."

The New York Legislature passed the law that determined how the state and local governments would divide the costs of adequately funding schools, taking into account both the characteristics of the students and the ability of the local governments to raise the necessary revenues. The state carefully worked out the system that would deliver enough money where it was needed most.

And then, as Bruce Baker puts it: "...they simply failed to fund it."

Now Cuomo comes along and offers a small fraction of what the state itself said is needed to properly educate New York's children. He pats himself on the back for his supposed bravery in taking on the "education bureaucracy," all the while blithely ignoring his duty to the most deserving children in his state.

Andrew Cuomo should not, in good conscience, demand that teachers step up and solve New York's terrible problems with inequality when he can't even muster up the political courage to give schools what the state itself says they need to adequately educate their students.

For Cuomo to hold hostage funds that the state itself says are necessary for New York's schools to properly do their job is cynical beyond belief. How craven must this man be that he can't even look the political leadership of New York in the face, following a decisive victory in the last election, and demand they do what the state itself says must be done for its students?

I am quickly finding Andrew Cuomo to be the most exasperating politician in America. He can be so correct on things like equal pay for equal work and universal preschool and several other issues. But he is massively wrong on education -- likely because he has adopted a pro-corporate neo-liberalism that his father wisely (if not entirely) resisted.

If Andrew Cuomo can't or won't do his job, he'd best not wag his finger at the many hardworking teachers of New York, who -- despite the failure of their governor -- serve the state's children far better than the cynics in Albany ever could.

On education, two peas in a pod.

(via Drew Sheneman)

@GovChristie's Casual Relationship With The Truth

Over at Blue Jersey, "interested observer" finds proof that Chris Christie wasn't telling the truth when he said he only met Cowboys owner Jerry Jones in 2013. It turns out this past December Christie claimed he had been friends with Jones for far longer:

And that would also seem to back up what he said with Steve Adubato this past December:

However, in an interview with Steve Adubato on PBS in late December, Christie was far less specific, telling his interviewer, "I've become friends with Jerry over the last five years."Adubato, looking a bit surprised, took note that the two men were apparently on a first-name basis, interjecting, "'Jerry'?" To which Christie said, " "Yeah. Jerry.' He allows me to call him 'Jerry'. I don't call him, 'Mr. Jones.' I call him 'Jerry.' And I've become friends with Jerry over the last five years."

So if "Jerry" didn't meet Chris until 2013, why have they been friends for 5 years with Christie saying he got his first call way back in 2009?

We'll have to see how all this impacts the investigation into whether Christie influenced the awarding of a contract to Jones's firm by the Port Authority. But we don't have to wait to put this incident into a larger context that explains the character of New Jersey' governor.

Because for a long, long time, Chris Christie has had a casual relationship with the truth:

- Chris Christie, in the 2009 campaign, told teachers and cops: "I will protect your pensions. Nothing about your pension is going to change when I am governor." We all know what happened next.

- When confronted on breaking this explicit promise in 2012, Christie said: "When I wrote that letter, I had no idea the pension system was about to go bankrupt." But everyone had been talking about New Jersey's pension time bomb for years before he wrote that letter, meaning Christie was either brazenly lying or frighteningly clueless (or, perhaps, both).

- When the expansion of charter schools appeared to be tied to political favors back in 2011, Christie said he didn't know one of the most controversial beneficiaries of that expansion, Amir Khan. Yet Khan sat directly behind Christie during at least two "town halls," and reports put Khan backstage with Christie before one event.

- Back in 2011, Christie claimed that New Jersey was the highest-taxed state, and that wealthy people were leaving New Jersey because of high taxes. Neither claim was true.

- In 2010, Christie told school districts across the state they would have their aid cut by 15%; he then proceeded to cut it all, and tried to blame the about face on his then education commissioner, Brett Schundler.

- Speaking of Schundler: Christie blamed him for the botched Race To The Top application in 2010. But Schundler testified that Christie himself insisted on the changes that scuttled the application.

- middle girl at DailyKos has a nice roundup of some of Christie's whoppers, including misstating the costs of the ARC tunnel, blaming the feds for screw-ups on Sandy aid, downplaying his relationship with David Wildstein, and, of course, Bridgegate.

- Christie's blatant disregard for the facts related to Bridgegate include making up a claim about the number of lanes available only to Fort Lee.

- Christie makes claims that he has slowed the growth of taxes, but he doesn't account for slashing property tax rebates. His response to being caught in this weasel wording? Hiding the data.

- Christie said he was the first governor to endorse Mitt Romney. He wasn't. (Personally, if Romney does run, I'm looking forward to seeing how Christie lies his way out of telling this lie.)

- Christie publicly misstated the medical condition of Kaci Hickox and threw the region into an unnecessary panic over Ebola.

- In 2011, Christie said that if teachers had taken a pay freeze, there would have been no teacher layoffs because the money saved would have made up for his cuts in state aid. The Office of Legislative Services later proved this was not true, and even the teachers union-bashing Star-Ledger Editorial Board chastised the governor for promoting this falsehood.

If Christie had only lied about his relationship with Jerry Jones, that would be bad enough. But this latest mistruth is just one more example of Chris Christie's long established pattern of deception, misstatements, and straight up lying.

Why, then, would anyone believe anything the man has to say now?

It all comes down to this.