I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Saturday, February 23, 2019

Thoughts on the Graduation Exam Mess in New Jersey

Let me recap where we are first -- skip down if you know the background.

Way back in 1979, New Jersey passed a law which required high school students to take an exam before graduating (the Education Law Center has some background on that law here and here). The law says the exam must "...measure those basic skills all students must possess to function politically, economically and socially in a democratic society." (N.J.S.A. 18A:7C-6.1.) Keep in mind that New Jersey is only one of only 12 states to require require an exit exam for high school graduates.

In 2014, the NJ Department of Education, under then-Governor Chris Christie, decided to replace the exam, known as the HSPA, with two tests from the Partnership for Assessment of Readiness for College and Career (PARCC): the Grade 10 English Language Arts and multi-grade Algebra I exams.

In 2016, ELC, the American Civil Liberties Union, and several other groups filed suit against the state, challenging the use of the PARCC tests as exit exams. Late last year, the state appellate court ruled in favor of the plaintiffs and struck down the regulations that led to the change in tests. The court found several problems with the change: the law calls for a single exam, not multiple ones, and the exam has to be in Grade 11, not in multiple grades like the Algebra I exam. 

The ruling left New Jersey's current juniors and seniors without a clear path to graduation. Some advocates suggested the exams should be scrapped altogether. But the plaintiffs and the state reached a settlement that allowed these students to take multiple testing pathways to graduation, including getting a passing score on the SAT, ACT, or military placement exams. The agreement would give the NJDOE time to develop new regulations on exit exams for current sophomores and freshman.

At the same time, however, State Senator Teresa Ruiz introduced a bill that changes the original statute to align with the current regulations -- the same regulations the court declared illegal. As I wrote previously, I find it highly problematic that the law is being changed to meet the regulations, and not vice versa.

But the NJ Senate apparently has no such qualms: they passed the bill 21 to 7, even after the settlement had been reached. The bill will now move on to the Assembly, where it will be voted on this Monday.

* * *

So that's where we are. Now let me add, in no particular order, my thoughts on all this:

- There is no evidence exit exams, by themselves, improve learning outcomes for students.

Again: only 12 states require exit exams. There is no evidence I've seen that these states have improved their outcomes as a result of having a graduation test.

There is some limited evidence accountability testing improves student outcomes; however, that evidence has been challenged in recent years. But that type of testing is not the same as testing which imposes a high-stakes consequence -- such as not receiving a diploma -- on students. We simply do not have evidence that imposing this requirement will lead to better student outcomes.

- A core principle of testing is that you cannot use an assessment for a specific purpose unless you make an argument it is valid for that purpose -- and no one has made that validity argument for using the PARCC tests as a graduation exam.

I keep a copy of Standards for Educational and Psychological Testing within reach when I'm writing about this stuff. Standard 1.0 -- the very first one -- states:
Clear articulation of each intended test score interpretation for a specific use should be set forth, and appropriate validity evidence in support of each intended interpretation should be provided. (p. 23)
The PARCC Algebra I exam purports to be an assessment of a student's ability in algebra. We can argue all day about whether it is... but no one has yet put forward that it is an appropriate assessment of "...those basic skills all students must possess to function politically, economically and socially in a democratic society" -- the clear language of the law.

Ruiz's bill, by the way, pulls off neat trick: it pulls the word "basic" from the original statute. This strikes me as a way of avoiding what should be at the core of any debate about the law: what exactly are these "skills," basic or otherwise?

Here, for example, is a sample question from the PARCC Algebra I exam:

Would you say that someone who can't answer this question cannot "function politically, economically and socially in a democratic society"? How many members of the NJ Legislature, for example, do you think could answer this question? How many of the senators who voted in favor of Ruiz's bill could? If they can't, does that mean they are unable to "function"?

- When you use a test to make a high-stakes decision, your validity argument must be especially strong.

Denying a young person a high school diploma is a serious matter -- especially if they've been attending school and doing their required work. You are closing off all sorts of opportunities to them to participate meaningfully in the workforce and in civic life. 

If you are going to deny them those opportunities based on a couple of tests, you'd better be sure the consequences of that denial are justified. You better spend time listening to experts about the validity of the test for the purposes stated in the law. You'd better be sure the law itself is a good idea.

Nothing even remotely like this happened in the NJ Senate. This bill was quickly rammed through without any real discussion, informed by expert opinion, of whether the PARCC is a valid instrument for the purposes of the law.

- The "passing score" for the PARCC is completely arbitrary -- it can be set anywhere the state wants.

As I pointed out previously, only 46 percent of test takers last year passed the PARCC Algebra I exam. There is simply no way that the public is going to put up with a graduation rate that aligns with that passing rate. Which means there will be enormous political pressure to change the proficiency rate on the exam.

And, contrary to the opinions of the ill-informed, changing proficiency levels is a rather simple matter.


As I've noted previously, tests like the PARCC are designed to yield normal, bell-curved distributions of scores. Within that distribution, anyone with the power to do so can set a passing or proficiency rate wherever they like.

The notion that "proficiency" is some objective standard, therefore, is simply not true. Proficiency is a social construct and, in my opinion, an artifact of political struggles; look at the history of New York's proficiency rates if you doubt me.

As Anne Hyslop points out, one of the consequences of using the PARCC as an exit exam is that there will be great pressure put on the state to set the standard for "proficiency" much lower than it currently is. That likely will affect the rates on the other exams throughout the various grade levels where PARCC is administered. Ironically, the push to set a high proficiency level for graduation may, in fact, lead to lower proficiency levels on all the PARCC exams.*

It really comes down to this: either the proficiency rate on the PARCC is going to come down, or lots of students are going to be denied diplomas. Which do you think is more likely to occur?

- The argument that community college remediation rates are so bad that we need graduation exams is weak.

In a truly awful editorial, the reliably reformy Star-Ledger makes this claim:
At Essex County College in Newark, 85 percent of incoming freshmen need to take remedial math. In 2017, only 13 percent graduated. While social promotion also happens in wealthier districts, those kids have a deeper safety net.
This is why we need an objective test. Yet because the PARCC is such a powerful diagnostic tool that can trace a student’s learning problem right down to a particular teacher’s lesson, it ran afoul of the union.
I posted that last paragraph on Twitter, and howls of derision from educators and testing experts ensued. It is, of course, impossible for any standardized test to pinpoint a teacher's particular lesson as the cause of a learning deficit; in fact, standardized test results, by themselves, cannot attribute the cause of any student learning outcome. If the S-L editorial board ever bothered to listen to experts in the field, maybe they wouldn't make such embarrassing claims.

Again: testing, by itself, cannot improve instruction. But the S-L appears to believe that if the PARCC was implemented as an exit exam, the remediation rates for math at Essex County College would improve. As I've pointed out in the past, however, the majority of community college students are older than 22, which means they haven't had a high school math course in years.

Further, if you click through on the link the S-L gives, you'll find that ECC is struggling mightily to provide remedial instruction, which of course raises the question: if better math instruction is simply the result of more testing, why are the community colleges having such a hard time teaching self-selected students the same material?

Maybe it's because, as Sara Goldrick-Rab has been finding in her invaluable work, many college students are suffering from food and housing insecurity. Of course, so are many high school students. Is denying them a diploma because they can't pass the PARCC really the best solution to this crisis?

College remediation rates are a club that reformy types regularly use to beat up on public education and public school teachers. But "ready for college work" is not an objective standard, and the research around this topic has many limitations. In addition: if legislators are going to insist that all students must show they are ready to meet the high standard of being ready for college work, then...

- No legislator should vote for this bill unless they are prepared to spend a lot more money to give students a real opportunity to pass the exam.

Standard 12.8 : When test results contribute substantially to decisions about student promotion or graduation, evidence should be provided that students have had an opportunity to learn the content and skills measured by the test.
"Opportunity to learn" is a core concept in educational testing. The Standards define it as:
The extent to which test takers have been exposed to the tested constructs through their educational program and/or have had exposure to or experience with the language or the majority culture required to understand the test. (p. 221)
I actually have a bit of a problem with this definition, because "exposed" comes across as a passive conception: leave an algebra textbook in front of a kid and arguably you've "exposed" them to the material covered by the PARCC. But that's clearly not what we're talking about here; the student needs to be given an opportunity to be meaningfully engaged with the material.

That means a qualified teacher, in a well-resourced school with a well-constructed curriculum, with all the support necessary -- both in and out of school -- for the student to be able to thrive.

All the way back in 2008, the NJ Legislature passed a bill, the School Funding Reform Act, which codified the amount of money necessary for students to achieve a minimally adequate education. But then the Legislature repeatedly refused to fully fund what the state's own law says schools need to educate students.

Worse, as Bruce Baker pointed out in real time, there's plenty of reason to believe the amount set was not enough. And the standards back in 2008 were lower than what the PARCC sets today. There is no doubt that sufficient funding is the necessary pre-condition to achieve desired educational outcomes. There is also no doubt the state has not come close to providing that funding.

If the New Jersey Legislature is prepared to set a much higher bar for graduation, they have a moral and legal obligation to provide the extra resources needed to clear that bar. If they can't or won't, they have no business imposing an onerous testing regime on students who came to school, did their work, and passed their classes.

One final thought:

New Jersey is home to some of the nation's foremost experts on psychometrics. I can't claim to know every conversation every legislator has had regarding this issue, but from the news reports I've seen, it seems these experts have largely been left out of the debate over the use of PARCC that's been going for the last year.

Why?

I understand lots of people have a stake in this issue, and we should hear from parents, educators, and other interested parties. But expertise has got to count for something. The Assembly would be well-advised to withdraw this bill from a vote on Monday and, instead, convene a panel of experts in testing and its uses to study the issue.

Time is no longer an issue: the settlement ensures that current upperclass high school students have a clear path to graduation. The state should take its time and get this right. As Senator Ruiz herself said:
“Even though the agreement came through, there has to be a statute change,” Ruiz said. “Either you do this today or we do it in two years when we come to this crossroads again.”
That's exactly right: we have two years to study this and get it right. Why wouldn't we take advantage of that?

More to come...




* One aspect of setting proficiency rates I rarely see discussed is whether it helps or hurts a child to hear that they didn't "pass" an exam. Some doctors have suggested ADHD rates have risen in the era of testing accountability. High-stakes testing has been shown to induce negative feelings in children and stress in students and teachers. Undoubtedly the fear of failure is the primary cause.

Is the push to raise standards worth this cost? Shouldn't we at least have a conversation about it?

Monday, February 18, 2019

The True History of New Jersey Teachers' (and Other Public Workers') Sacrifices on Pensions And Benefits

I want to talk about a report that emerged a couple of weeks ago about New Jersey public employees' pensions and health care. But let me set the table first...

Because every time I come across a story like this, it's as if the Ministry of Information has decided, once again, to disappear history. But I'm not about to let that happen -- and if it means I have to keep coming back and repeating the timeline below, I will.

I posted the first version of this back in 2016. And I will keep updating it so that no one can make the claim that teachers and other public employees haven't already made large sacrifices in an effort to bring New Jersey back to fiscal health.

 * * *

THE TRUE HISTORY OF NEW JERSEY TEACHERS' (AND OTHER PUBLIC WORKERS') SACRIFICES ON PENSIONS AND BENEFITS

1995: Governor Jim Florio begins the modern era of New Jersey pension underfunding.

1997: Governor Christie Todd Whitman essentially pays for tax cuts by underfunding the pensions.

2001: Governor Donald DiFrancesco raises pension benefits, but he does so by using the same sort of revaluation tactics that Florio and Whitman had used.

2004: Teachers' mandatory contribution to the pension, which had been as low as 3 percent, is raised to 5 percent.

2007: By now, everyone (except Chris Christie) knows the pensions are in trouble and have to be fixed. Under Governor Jon Corzine, teachers and state workers now pay 5.5 percent into their pensions and see the retirement age go up 5 years. In addition, state workers now pay 1.5 percent of their income for health benefits.

These increases are part of an explicit deal: in exchange for these concessions the state will start making payments into the pension system. Those payments, however, last for only two years.





2008: The state raises the retirement age again.

2010: Governor Chris Christie makes significant changes to the pension for all new hires, and now requires all current school district employees to pay 1.5 percent of their salaries toward health benefits. At this time, a report is released from Labor and Employment Relations Professor Jeffrey Keefe of Rutgers University that shows: "... full-time state and local employees are under-compensated by 5.88% in New Jersey, in comparison to otherwise similar private-sector workers."

2011: After running a campaign in which he said explicitly: "Nothing about your pension is going to change when I am governor," Chris Christie, with the support of many Democrats in the Legislature, passes a sweeping pension and benefits overhaul law, known as Chapter 78. As the Communications Workers of America explains:
The plan for increased worker contributions is phased in over four years. At the end, in 2015, workers pay 25% more to get 30% less pension. Workers are required to pay an increasing amount of the health care premium, with a top rate of 35% of premium for the highest earners. The state budget includes a 1/7th payment for FY12. Christie makes the payment on the final day of the fiscal year, in June 2012. In response to the 2010 Supreme Court ruling, the legislature creates a contractual right to the funding of the pension.
In addition, cost of living adjustments (COLAs) are frozen under the new law.

2013: During the gubernatorial election, no one in the press cares to ask either Chris Christie or his opponent, Barbara Buono, how they plan to raise the revenues for a full pension payment by 2017.

2014: To the surprise of no one sentient, Christie refuses to make the payments his own law requires. Meanwhile, reports begin to surface about inordinately high management fees paid to Wall Street firms linked to Christie. The unions file suit.

2015: Reports of malfeasance in the management of the pensions continue. The fourth year of the 2011 Chapter 78 law starts: a teacher making $65K now pays at least 19% of her premium for family medical coverage. As NJ Spotlight notes:
Today, however, while the cost of New Jersey public employee health insurance coverage remains the third-highest in the nation, most New Jersey public employees are paying more than the national average for state government workers toward their health insurance costs, an NJ Spotlight analysis shows. 
In fact, the average New Jersey government employee is paying more for individual health insurance coverage than government workers in any other state and the 10th-highest average premium for family coverage in the country. 
Further, state and local government workers are paying a much higher percentage of the cost of their individual health insurance policies than private-sector employees in New Jersey have been paying, and not much less than the percentage paid by the state’s private-sector workers for family coverage. [emphasis mine]
Meanwhile, New Jersey's public employee pensions have devolved into one of the least generous in the nation, according to a New Jersey Policy Perspective analysis. And yet the state's highest court rules New Jersey doesn't have to fund the pensions, and can instead set the state up for a looming fiscal disaster.

2016: The Chapter 78 law sunsets; however, the NJ School Boards Association declares to its members that the much higher employee payments for health insurance have become the new normal:
As the sun sets on Chapter 78, boards need to remember three important points.
  • The fully phased in employee contributions are the status quo for negotiations purposes – in order for the employee contributions to be reduced, the board must agree to do so;
  • A reduction in employee health care contributions is not negotiable until the next contract executed after Chapter 78 is fully implemented; and
  • Once, employee contributions become negotiable, boards need to carefully consider the long term implications of moving away from the cost structure dictated by Chapter 78.
At the same time, Chris Christie's pension commission releases a report calling for public workers to be moved into less generous health care plans, requiring much greater out-of-pocket expenses. 

In June, the NJ Supreme Court upholds the frozen COLAs on pensions.

2017: A new report from Rutgers professor Jeffrey Keefe finds: 
New Jersey public school teachers are in fact undercompensated, not overcompensated. Using regression analysis to control for level of education and other factors that affect pay, we find that public school teachers earn 16.8 percent less in weekly wages and 12.5 percent less in weekly total compensation (wages and benefits) than other full-time workers in New Jersey. The percent by which teacher pay is less than pay of comparable workers is called the teacher pay penalty. An analysis of hourly compensation shows the teacher pay penalty at 13.7 percent for wages and 9.4 percent for total compensation.
Late in the year, NJEA, the state's largest teachers union, reports the teachers pension has a funding level of 47 percent. The level must be 80 percent to reinstate the COLA.

2018: NJEA agrees to changes in health coverage that reduce costs by raising the prices for out-of-network coverage. Meanwhile, reports continue to surface about high health care costs for New Jersey's teachers.

On July 1, the final pension rate hike under Chapter 78 is implemented: teachers now pay 7.5 percent of their salaries toward their pensions.

2019: In a report, The Office of the State Comptroller finds that New Jersey's program of tax incentives for businesses, tagged at a price of $11 billion, operated with little oversight:
The Office of the State Comptroller reviewed several of the state’s tax-break programs aimed at rewarding companies for either creating or retaining jobs in New Jersey over the last decade. 
The agency gave closest scrutiny to a sample group of a few dozen companies that were awarded incentives, and it found nearly 3,000 reported jobs were not substantiated as having been created or retained even though the companies in that group redeemed their incentives.
 * * *

Let's recap what's happened over the last 15 years or so:
  • Pensions: Payments have gone up from 5 to 7.5 percent, the COLA is still frozen, and the pension is judged to be one of the least generous in the nation.
  • Health insurance: Both the premium and the percentage employees pay has skyrocketed. Again, public employees are paying more than private employees for individual coverage, and not much less for family coverage.
  • Wages: By all indications, teachers continue to suffer from a wage penalty. In addition, while teacher salaries are growing slightly in the state, the growth appears to be nowhere near enough to make up for the losses from pension and health care contributions.
By any reasonable measure, New Jersey teachers -- and other public employees -- pay more for their retirement and health care benefits, get less back for those pay hikes, and still lag behind the private sector in total compensation.

This is beyond debate. Even as the state gave away billions in tax incentives that haven't been shown to do anything, teachers and other public employees have sacrificed -- over and over again -- to get New Jersey back to fiscal health.

Are we all clear on this? OK...

Then why is this even a thing?
A top aide to Gov. Phil Murphy, in a conference call with liberal activists, suggested ways to push back against state Senate President Stephen Sweeney’s big plan to fix New Jersey’s long-term fiscal problems, NJ Advance Media has learned.
The aide — Deborah Cornavaca, Murphy’s deputy chief of staff for outreach — said during the call Wednesday that Sweeney, a fellow Democrat but frequent Murphy rival, is pushing “a false narrative” against public-worker unions.
Sweeney, D-Gloucester, told NJ Advance Media he’s bothered by Cornavaca’s appearance on the call.
“You can imagine how disappointed I am that a deputy chief of staff is calling groups to basically attack,” Sweeney said Thursday.
No. No, no, no.

A disclaimer first: I've known Deb Cornavaca for a good long while. I wouldn't say we were close friends, but we've shared a few meals over the years and I like her a lot. She's smart and tough and doesn't back down from a fight.

Yes, there is a lot of bad blood between Sweeney and the NJEA. But I listened to this town hall and I didn't hear Deb tell anyone to go after Sweeney personally. Instead, I heard her tell activists -- people who work to protect the interests of middle-class public workers like me and many of my readers -- to push back on the story that it is up to teachers and cops and firefighters and state workers to fix New Jersey's fiscal mess.

Look at my timeline above again. Can anyone really make an argument that New Jersey's public workers haven't been doing their fair share -- in fact, way more than that -- to solve the state's budget crisis? Is it wrong to say to those who keep demanding more and more from public workers that it's past time for other interests to start pitching in as well?

The sacrifices of New Jersey's teachers are laid out above. Where, however, are the sacrifices of the wealthiest people in the state? The small hike on income over $5 million imposed last year? The temporary corporate surcharge? Is anyone prepared to argue that those are at all comparable to what I've outlined above? Especially since New Jersey corporations have been enjoying huge tax incentives with no accountability?

I will admit that New Jersey is better than most states when it comes to tax fairness, but we're hardly progressive.

The top 1 percent in New Jersey make family incomes of over $900,000 a year, and they pay less in taxes than people who have incomes in the middle -- people like teachers and the vast majority of public employees. Is anyone seriously suggesting the state's wealthiest folks can't put in some more to help New Jersey out of this mess, especially since public workers have been putting in more and more for years.

And what has been done to rein in the excessive fees collected by Wall Street to manage the pension funds under Christie? What has been done to bring the cost of public employee health care down? Where are the tough negotiations with the pharmaceutical companies and big health insurers?

What Deb was talking about in that meeting wasn't just about pushing back on a narrative -- it was about telling the truth. New Jersey's teachers and public employees have already made significant sacrifices to help the state out of its fiscal crunch.

It can't always fall to us to be the ones to fix a mess we didn't create. We can't be the only ones who are expected to ask our families once again to make even more sacrifices while plenty of others in the state enjoy tax giveaways.

I'm sorry that Senator Sweeney's feelings are bruised, but there's no way he should expect, given the history I've outlined above, that public employees are just going to keep taking it and taking it and taking it. Unless and until New Jersey starts demanding more from its wealthiest residents and corporations, public employees have every right to point at everything they've already given up and say: "Enough."



Saturday, February 16, 2019

Testing Chaos Continues in New Jersey

Here's an update from my last blog on the chaotic debate over New Jersey's high school exit exam:

Yesterday, after several delays, the state announced a settlement that at least clarifies the path to graduation for New Jersey's high school juniors and seniors:
New Jersey reached a settlement Friday that offers a clear path for juniors and seniors to graduate, after a court declared current public school standardized testing requirements illegal.
The settlement clears a path for about 170,000 juniors and seniors who had passed state exams and who the state said were "in limbo" after the court's decision.
Under the court-approved settlement, the state will allow high school juniors and seniors to graduate if they have passing scores on state PARCC exams or other approved standardized tests, such as the SAT, ACT or the military placement exam.
The agreement also provides the Department of Education with time to propose new graduation testing rules for the classes of 2021 and 2022, who are current freshman and sophomores. 
You'll recall that the appellate court declared that the PARCC Algebra I and English Language Arts (ELA) Grade 10 tests were, together, not an acceptable graduation test, because they're not a single test and they're not given in 11th grade, as the law says the graduation test should be.

But as I pointed out last time: a larger problem is that no one has ever shown these tests to be valid for the purpose laid out in the law. Psyshometricians will tell you this is a minimum requirement for any test: you have to show its outcomes are appropriate for a specific use.

The law says the test must "...measure those basic skills all students must possess to function politically, economically and socially in a democratic society." (N.J.S.A. 18A:7C-6.1.) But it's clear that the PARCC tests don't measure "basic skills." They set a much higher bar  -- a bar so high only 46 percent of test takers last year could meet it. There's simply no way to argue a test measures "basic skills" when its passing rate is so low.

I'll note here that the court did not rule on whether the content of the PARCC tests was valid for the purposes of an exit exam: the fact that the tests aren't a single exam and aren't necessarily given in Grade 11 was reason enough to strike down the regulations.

I suppose it was on this basis that State Senator Teresa Ruiz, long a proponent of the PARCC, introduced a bill that would have changed the law so that multiple tests given in any grade could be used as high school exit exams. But that still wouldn't have addressed the problem of determining whether the PARCC tests are valid for the purposes laid out in the law.

It also strikes me as highly problematic that Ruiz wanted to change a law, passed by the Legislature and signed by the Governor, to conform with administrative regulations. As Sarah Blaine pointed out in her excellent analysis of the court's ruling, regulations are supposed to follow from laws, not vice versa. Ruiz's bill, which was being rushed through in time to affect current high school seniors, was making changes to the law -- not the regulations -- so that the PARCC tests could be used in a way the court said was illegal.

Unfortunately, it seems like this agreement is doing the same thing: even after the court said, "Don't use these tests this way," the state is going to go ahead and do just that for the next two years.

I can understand that the plaintiffs in the case -- which was argued by the good folks at the Education Law Center and the American Civil Liberties Union -- wanted to quickly remove any uncertainty for the students affected by this ongoing mess. But the fact remains that even though the court ruled these tests are not appropriate for use as graduation exams, they remain in place as just that.

I think this is deeply unfair for at least one reason: The state has not been providing the resources necessary for the majority of students to meet this new, higher standard.

The usual suspects have, of course, been making their case that New Jersey must have these tests in place to ensure that high school diplomas "mean something." They worry that without a rigorous exit exam, New Jersey -- consistently one of the highest-performing states in a variety of educational outcome measures -- will dumb down its standards and leave its students less than "college- and career-ready."

First of all: if there is any empirical evidence that high school exit exams, by themselves, improve educational outcomes, I haven't seen it. After all, weighing the pig doesn't fatten it up. Plenty of states don't have exit exams; some, like Connecticut, perform well in national and international comparisons. Where, then, is the evidence exit exams lead to better outcomes?

Second: I often read op-eds like this and think the writers must believe that all we need to do to improve educational outcomes is just try a little harder. Those of us who actually work in schools, however, know it's never that simple. If a child shows up at the schoolhouse door hungry or ill or in stress, that child will have a disadvantage compared to others in academic outcomes. So if we want all of New Jersey's students to meet a "high" standard, we have to ask whether those students are arriving at school ready to meet that standard.

Further, we have to ask whether the school itself has the resources it needs to educate children to meet higher levels of achievement. Remember: New Jersey has not been providing its schools with what the state itself determined was necessary for children to achieve equal education opportunity.

Worse, that determination was made back when the standards were lower. Now, suddenly, "reformers" want to raise the bar, without the slightest thought as to whether schools might need even more resources to achieve even higher outcomes.

How can anyone possibly think it's fair to deny children a high school diploma when they've done everything that was asked of them -- stayed in school, went to class, passed their courses -- but the state hasn't done what it was supposed to do and fully fund their schools? 

Unfortunately, it's too late for the juniors and seniors: they'll have to soldier through the tests, or pay to take alternative assessments like the SAT or ACT, or slog through the onerous, expensive process of making a "portfolio." I don't see any benefit in forcing these young people and their families to go through all this; the argument about making their diplomas "mean something" is specious at best.

If one good thing comes from this agreement, however, it's that over the next several months New Jersey might finally get to have a real debate about testing, standards, and school funding. I'll be curious to see if those who've been pushing for harder tests and higher standards will fight just as hard for adequate and equitable funding for all schools.







ADDING: The op-ed in nj.com states:
"Twenty-two of our sister states require high school students to take an assessment in order to graduate."
Click through on the link, however, and you'll find that only 12 states require an exit exam. In 2017, Stan Karp, who knows as much as anyone about this stuff, put the number at 13.

Sunday, February 10, 2019

The Failure of State Control in Camden, NJ

In my last blog, I wrote about the NJ Auditor's report on Camden's "Renaissance" schools. These charter-district hybrids, run by three of the region's biggest charter operators -- Mastery, Uncommon, and KIPP -- were supposed to show definitively that charter schools could serve all of the children in a neighborhood. They weren't going to "skim the cream" any more; instead, they'd take every child, no matter their family background or educational need.

Well, it turns out the Renaissance promise was just a lot of hot air: according to the Auditor, fewer than half of neighborhood students are enrolled in their neighborhood renaissance school. Thanks to Camden's "universal enrollment" system, the Renaissance schools appear to be doing a completely different job than the public district schools.

How could this happen? Why didn't the Camden City Public Schools administration pick up on this problem? How could they have missed this? Weren't they paying attention?

As it so happens, the Auditor, Steven Eells, has been busy: not only did he and his staff examine the Renaissance schools -- they looked at the district as a whole. And what they found isn't very encouraging (all emphases below are mine):
The lack of continuity within and oversight of the district’s business office functions has resulted in a lack of control and accountability of the district’s finances. The lack of stability in administrative positions inhibited the development of long-term goals and the ability to establish and enforce internal controls to ensure district resources were expended in an efficient and effective manner and assets were properly safeguarded. The financial transactions included in our testing were related to the district’s programs and were reasonable; however, they were not always properly recorded in the accounting system, and there were many instances when requested documents could not be provided. We found programs lacking internal controls and proper oversight, significant deficiencies in the procurement process, and other issues requiring corrective action. Certain provisions of the Urban Hope Act included in our testing were complied with by the district with the exception of those related to enrollment.
Before we dive into this, let's step back and recall some history:

Way back in 2012 -- back when Chris Christie was making teacher bashing fashionable -- a couple of low-level bureaucrats in the NJ Department of Education came up with a plan for Camden's Schools. The idea was to take power away from the local school board -- which didn't have much power anyway as it had been subject to the direction of a state fiscal monitor since 2006 -- and shift control to the Christie administration and the State Board of Education. This would allow charter schools to flourish while CCPS schools were shuttered.

It's worth noting that the guys who came up with the plan were paid by California billionaire Eli Broad, who was the patron of then-Acting Commissioner of Education Chris Cerf. The next year, Christie went all-in on Camden and had the state take overt the district. The excuse was that Camden was such a failure, the state really had no choice. 

Christie proceeded to go out and get a very young fellow to be his new superintendent. Paymon Rouhanifard had, at best, six years of total experience in education, but apparently that's all he needed to take on arguably the toughest school leadership job in the state.

Rouhanifard left CCPS last year; when the Auditor discusses the state of Camden's schools, he's discussing Rouhanifard's legacy. I've already gone over the issues with the Renaissance schools' enrollments; let's look at what else the Auditor found in Camden:
  • The district failed to timely recover $2.5 million in utility costs, shared custodial and security services, and leased facilities and facility space provided to renaissance school operators. Additionally, actual custodial costs incurred by the district exceeded reimbursed amounts for fiscal years 2016 and 2017 by $245,000 and $217,000, respectively. 
  • Contracted preschool providers were overpaid $281,921 because the district did not make required payment adjustments during fiscal year 2017. 
  • Controls over expenditures need to be strengthened. The district’s failure to record all obligations promptly could result in unrecorded liabilities and cause the district to overspend budgeted funds. 
  • A vendor was paid a flat yearly fee of $1,638,104 to operate the district’s alternative school programs. The district failed to adequately monitor vendor payments, resulting in an overpayment of $151,300. Additionally, the district was not aware that contracted performance metrics and deliverables were not achieved. The district has taken over the alternative school programs, budgeting $280,000. 
  • Other areas of concern involve employee leave records, Camden County Technical Schools tuition, inventory controls, and various additional procurement deficiencies.
It's really no wonder that the Renaissance schools weren't being properly monitored -- The district itself was failing in its core responsibility to monitor its own finances.

Keep in mind that Rouhanifard, like Cerf, made his bones in the NYC school system under Mike Bloomberg and Joel Klein. This was the heyday of the "disruption" doctrine of school leadership -- and disruption meant kicking out people who had built their careers in the school system and replacing them with hotshots who would come in turn over a few art supply carts.

The problem with this mindset is that institutions like school districts really need stable, experienced leadership. For example:
The district’s business office has been negatively impacted by frequent turnover within managerial positions. The lack of employee continuity has disrupted the internal control system in place. During our 32-month audit period, the School Business Administrator position was held by three different individuals. Also, the Assistant Business Administrator, Comptroller, Payables Manager, and Senior Payroll Manager positions have been abolished. Additionally, 23 upper-management employees separated from employment with the district. The average length of time these employees worked for the district was three years. This turnover has contributed to the erosion of internal controls.
Again: stuff like this was the whole reason Christie insisted the state had to take over CCPS. But then he appointed a wet-behind-the-ears superintendent who, it turns out, was clearly out of his depth. No surprise -- Christie did the same thing in Newark, and we all saw how that went...

I've said this repeatedly, so you'll pardon me if this is getting stale, but I'm going to give it yet another go in the hopes that maybe it starts to stick: 

White parents in the leafy suburbs would never, ever, put up with state control of their schools -- especially if that meant having to deal with a superintendent who had no experience running a school, let alone a district.

The idea that state control is the only solution for "failing" urban schools is built on a nasty bedrock of racism. But on top of that: State control of schools clearly doesn't work.

I know credulous reporters love to eat up pre-digested talking points about soaring graduation rates and skyrocketing test scores to justify these state interventions. But when you look at these metrics properly, it turns out the grad rates are simply part of overall trends (more here), and the small bumps in test scores are best understood as artifacts from changing the tests, not as real improvements in teaching and learning.

Camden deserves better. It needs experienced, competent leadership that can properly manage the district's finances. It needs adequate and equitable funding. It needs a system of school governance that allows all local stakeholders to have a say in how the system is operated -- just like almost every other district in the state.

State control has failed in Camden. It's time to admit it and move on to something better.




ADDING: I had complained in my last post that the media did not pick up on the Auditor's report on the Renaissance schools. But, to their credit, both NJ Spotlight and the Courier Post have since reported on the Auditor's findings.

Sunday, January 27, 2019

Broken Promises: Camden's "Renaissance" Charter Schools

UPDATE: Below, I write that no one in the media, so far as I know, has reported on this story. I did the usual Google search and came up with nothing. But over on Facebook, a reader reports:

"Matt Friedman did bring the reports up in his daily Politico NJ briefing on Friday (that is how I knew to look for them), but the link was to premium subscriber content, which is behind a paywall that is too rich for my blood."

I stand corrected, and apologize to Matt Friedman and Politico.


Work in education policy long enough and you'll come to understand that nearly all of the promises of "reformers" aren't worth a bucket of warm spit. For the latest example, let's go to Camden, New Jersey:

Way back in 2012, the New Jersey Legislature enacted, and then Governor Chris Christie signed, the Urban Hope Act. It was going to be the game changer: it would finally show, once and for all, that "successful" charter school operators had the secret sauce needed to radically transform schools in disadvantaged communities like Camden.

NJ Spotlight explains the Act:
What it is: The Urban Hope Act was enacted and signed into law in 2012, opening the way for hybrid charters known as “renaissance schools” to open in three cities: Camden, Trenton, and Newark. Only Camden decided to move on this option and now has three networks approved and opening their first schools this fall. 
What it does: The law allows for charter companies to apply to the city to open and build new schools under certain conditions. For one, they are funded at greater rates than traditional charters -- roughly 95 percent of per-pupil costs in the city -- and given incentives to build new facilities. But they also must adhere to the district’s enrollment patterns, drawing from the neighborhood catchments. Unlike traditional charters, each applicant must also be approved by the local board of education. [all emphasis here and below are mine]
That sentence I bolded is the key provision of the act, and a response to charter skeptics (like myself) who have repeatedly noted that New Jersey charter schools do not enroll the same types of students as their hosting public district schools (for starters, see here, here, and here).

The renaissance schools were going to avoid this problem by taking all of the children within their "attendance area." Cream-skimming was going to be impossible; everyone in the neighborhood was going to have access to these schools. The renaissance operators, who are some of the biggest names nationally in charter school management -- KIPP, Uncommon, and Mastery -- would now have to show they could run schools that served all children in the area, not just those who elected to attend.

Again, this was the key provision of the Urban Hope Act; without it, the renaissance schools were simply more charter school expansion, advantaging some children over others.

Well, the NJ Auditor, Stephen Eells, just issued a couple of reports on Camden's schools. And guess what?
EXECUTIVE SUMMARY
CITY OF CAMDEN SCHOOL DISTRICT July 1, 2015 to February 28, 2018 
[...]
  • The current enrollment process has limited the participation of neighborhood students in renaissance schools. Per N.J.S.A. 18A:36C-8, renaissance schools shall automatically enroll all students residing in the neighborhood of a renaissance school. Instead, the district implemented a centralized enrollment system in which families must opt in if they prefer to attend a renaissance school. This process has left the district with fewer than half of neighborhood students being enrolled in their neighborhood renaissance school.
Let's go to the full report on Camden's schools (which has some other stunning conclusions that I'll write about later) and break this down:

The report notes that for the 2016-17 school year, Camden implemented a universal enrollment system. The system is not run by the district, but by an autonomous, nonprofit organization. Families rank their choices and oversubscribed schools choose students through a lottery. It's worth noting that not all charters participate, siblings get preferences, and the charters and renaissance schools choose how many seats they wish to offer. In other words, there are already filters built into the system that advantage some students over other when selecting schools.

Yet the Urban Hope Act is very clear about enrollment: if the school is on land owned by the state's Schools Development Authority, everyone within the school's attendance area must be enrolled:
  1. A renaissance school facility located on land owned by the New Jersey Schools Development Authority or the renaissance school district shall automatically enroll students residing in the attendance area established by the renaissance school district for that property. The renaissance school project located on land owned by the New Jersey Schools Development Authority or the renaissance school district shall allow any student who was enrolled in the renaissance school project in the immediately preceding school year to enroll in the renaissance school in the appropriate grade unless it is not offered; a student enrolled in the immediately preceding school year shall have priority for enrollment in a grade that is at capacity over a student who would otherwise be eligible automatically for initial enrollment in the renaissance school project based on his or her residence in the attendance area established for the renaissance school.
This was the part of the Act its champions were always crowing about: all of the neighborhood kids would get to go to these schools. Of course, these same folks always neglected to mention the next part of the Act:
  1. If there are more students in the attendance area than seats in the renaissance school, the renaissance school shall determine enrollment by a lottery for students residing in the attendance area. In developing and executing its selection process, the nonprofit entity shall not discriminate on the basis of intellectual or athletic ability, measures of achievement or aptitude, status as a handicapped person, proficiency in the English language, or any other basis that would be illegal if used by a school district.
See how this works? The renaissance schools themselves got to determine how many seats they wanted to offer, and if they were over subscribed, they could implement a lottery. The auditor's report explains what happened next:
All buildings utilized by the renaissance schools during the 2016–17 school year and seven of ten utilized by the renaissance schools during the 2017–18 school year were on land owned by either the district or SDA; however, we found neighborhood students were not automatically enrolled in their neighborhood renaissance school, where applicable, in accordance with the Act. Instead, the enrollment process for all renaissance schools was implemented by the district as a choice program, requiring parents and guardians to opt in if they prefer their child attend a renaissance school. Renaissance school neighborhoods overlap with those of traditional district schools, creating ambiguity as to which neighborhood school a student is entitled to attend.
Under the current process, students are guaranteed a seat in their neighborhood district school but only receive preference at their neighborhood renaissance school. Although students are required to submit an application through Camden Enrollment to be eligible to attend a renaissance school, no application is necessary to attend the neighborhood district school. Without a requirement that all district students apply through Camden Enrollment, the district cannot prove that all parents and guardians were adequately informed of their child’s eligibility to attend or if they opted not to accept enrollment in their neighborhood renaissance school. 
The current policy could result in a higher concentration of students with actively involved parents or guardians being enrolled in renaissance schools. Their involvement is generally regarded as a key indicator of a student’s academic success, therefore differences in academic outcomes between district and renaissance students may not be a fair comparison.
I have to say it's refreshing to finally find a New Jersey state official who understands something so many self-styled education "experts" don't: Students who enter charter school lotteries are not equivalent to students who don't. Plenty of research backs this up (see the lit review in this paper for a good summary of this research). Combine this with the high attrition rates in many "successful" charters, and the high suspension rates at many more, and you have a system designed to separate students by critical family characteristics that do not show up in student enrollment data.

The report goes into detail about how this all is playing out in Camden:
The Camden Enrollment process has left the district with fewer than half of neighborhood students being enrolled in their neighborhood renaissance school. We reviewed renaissance enrollment records from the 2016–17 school year and found that only 48 percent of enrolled students resided in their renaissance school’s neighborhood and only 26 percent of all district and renaissance school students, residing in renaissance school neighborhoods, were enrolled in their neighborhood renaissance school. 
In the 2016–17 enrollment lottery, 461 students were accepted to renaissance schools. Of these students, 247 (54 percent) resided in the neighborhood of their renaissance school. In the 2017–18 enrollment lottery, 838 students were accepted to renaissance schools. Of these students, 387 (46 percent) resided in the neighborhood of their renaissance school. Overall, less than half of students accepted to renaissance schools (49 percent) through the enrollment lottery process for the 2016–17 and 2017–18 school years were from the renaissance school’s neighborhood. 
All neighborhood students who submitted applications by the deadline for the 2016–17 lottery were accepted in their neighborhood renaissance school; however, 47 students who applied by the deadline for the 2017–18 lottery had to be placed on their neighborhood renaissance school’s wait list. As of October 2017, there were 195 students on the wait list for their neighborhood renaissance school.
The report goes on to note that because renaissance schools give priority to students who return grade after grade, kindergarten enrollment is critical. But only 48 percent of renaissance kindergarten students were from the schools' neighborhoods. 

It's important to note that the Camden City Public Schools do not have the luxury of setting caps on enrollments, deciding which grades to serve, or not enrolling students who move in after the kindergarten year. Everyone in Camden must get a seat at a CCPS school. But only a lucky subset of students get to attend a renaissance school.

Why does this matter? First of all, the renaissance schools get a whole host of advantages that CCPS schools do not. You'll recall, for example, that the former State Superintendent of CCPS, Paymon Rouhanifard -- hand-picked by Chris Christie -- told Camden residents that the only way to renovate the district's crumbling and dangerous buildings was to turn them over to charter school operators:
(7:52) I'd love to be able to properly address some of the parents' questions here. I want to be helpful. Look, I get that there are anxieties. Guys, for the last three decades, our buildings in this city are falling apart. We haven't had the money to fix them. We don't have the district finances to fix them. Through this partnership, this is our opportunity to do it.  
That, as I pointed out in real time, was nonsense -- the state, under Chris Christie, had made a conscious choice to underfund Camden schools' infrastructure needs. Further, the state turned over land where it had promised to build a public district school to KIPP-NJ, a charter operator which had already tried and failed to run a school in Camden.

Then, in 2016, George Norcross III, the Democratic machine boss of South Jersey, set up a fund to give millions more to the renaissance schools. As a story in the Philadelphia Inquirer told us, these schools were supposedly totally different from other charters:
Hybrids of public and charter schools, Renaissance schools are publicly funded but privately operated. Unlike charter schools, they guarantee seats to every child in the school's neighborhood, and they have contracts with the district mandating services like special education. By law, they must operate in new or renovated buildings.
So the state, which has had direct control over the district since 2013 and near-direct control for years before that, starved CCPS while letting favored charter schools thrive. This created a have/have-not situation that the renaissance schools have apparently only made worse.

Again: we were assured this wasn't going to happen. We were told repeatedly that all the kids in a renaissance school's neighborhood would have a seat. Naturally, the Star-Ledger editorial board, purveyors of all things reformy, believed it: "The campus will grow one grade level at a time, serving every kid in the neighborhood — including those learning English, or with special needs."

They weren't alone: Politico reported renaissance schools would take all the students in the neighborhood. So did the Philadelphia Inquirer. And why wouldn't they? Rouhanifard himself called them "neighborhood schools," and neighborhood schools take all students in the neighborhood, right?

Well, the data is now in, and the whole thing turned out to be a crock. And if any of those media outlets have reported on the auditor's findings, which came out a couple of weeks ago, I haven't seen it.

Let me finish up with two thoughts. First: those of us cursed with a long memory know this entire thing was planned. Way back in 2012, Chris Cerf, then the acting Education Commissioner, got his good friend Eli Broad to pay for a couple of guys to come in and develop a plan that called for the dissolution of local control of Camden's schools. That would allow the state to close CCPS schools and replace them with charters.

Second: the reason these plans were put into place was that they provided an excuse for failing to fully fund urban schools. Chris Christie ran around the state declaring these schools proved that we could -- in contradiction to a raft of empirical evidence -- radically slash school funding.

Renaissance schools -- like charter schools, and test-based teacher evaluations, and high school graduation exams, and "personalized learning," and private school vouchers, and so on -- are excuses. We use them to justify inadequate funding of schools in disadvantaged communities.  But like all reformy schemes, Camden's renaissance schools are, at best, a tepid response to our serious problems of racism, economic inequality, and inadequate school resources.

More on Camden and the auditor's reports in a bit...

Welcome to Camden.


Wednesday, January 9, 2019

Kids In Disadvantaged Schools Don't Need Tests To Tell Them They're Being Cheated

There was a very raw, very discouraging story in The Trentonian this past week that is worth your time, whether you live in Jersey or not. Here's an excerpt:
Over the past year, this newspaper spoke with high school students educated in the Trenton Public Schools (TPS) district. The interviews took place in the presence of an adult and the teens were granted anonymity to speak freely and honestly. Each interview started with vague questions, such as “What is it like to live in Trenton?” While some students also spoke about nice and community-oriented neighbors, each of the conversations began with a discussion about violence. 
[...] 
“The school smells like weed,” a highschooler said. “They smoke in the hallways and stairwells almost everyday.” 
While some students said school guards “try to stop bad behavior” and convince kids to stay out of trouble, others described guards as “too young,” with “not enough care” for what happens. 
“This guy told the security guards what was going to happen to him, but they didn't care enough to do anything about it, so he got jumped,” a teen said. “They don't take their job seriously.” 
And as for teachers: “I feel like it depends on whether they know the student wants to change,” a teen said. 
Students said some teachers will remain persistent in trying to convince a kid to stay out of trouble. But if they realize their advice is not improving behaviors, “they just give up.” 
“I think that’s why a lot of people say teachers don't care either,” a student said.
When asked to estimate the percentage who don’t seem to care about the students in their school, the majority of the teens said approximately 70-75 percent of teachers seem like they don't want to be there.
 
One teen suggested the teachers have cause for not caring: “They have to teach in Trenton and dealing with kids’ attitudes is just overwhelming for them after a little while. It's not getting better, it's getting worse.” 
Teens estimated 60-70 percent of students seem to not take school seriously. One teen described negativity as their greatest challenge living in the capital city. 
“Negativity seems to be everywhere in Trenton; you can't run from it,” she said.
With a toxic environment as described by teens interviewed for this report, it’s no wonder that the TPS district high school graduation rate for the class of 2017 was only 70 percent, according to state department of education data. That low figure is due in large part to Daylight/Twilight’s graduation rate of 34 percent. Both Trenton Central High School campuses graduated more than 80 percent of its 2017 class.
 
The statewide graduation rate in 2017 was 90.5 percent. The graduation rates both statewide and in TPS have gradually increased over recent years.
Again, this is a tough piece of writing. But I thought it was well worth it, because the whole story raises some very uncomfortable questions as states head into their budgeting seasons:

A few days ago, a New Jersey appellate court threw out regulations that made the PARCC Algebra I and English Language Arts (ELA) Grade 10 tests a requirement for high school graduation. One of the most prevalent arguments for the PARCC -- which is based on the Common Core standards -- is that if we didn't have high standards and tough tests to match them, we are "lying" to students.

This argument is aimed particularly at schools like those in Trenton: "failing" schools, as reformers are so eager to call them. Apparently, many of us have deceived ourselves into thinking everything is fine in places like Trenton. Worse, students and parents, according former SecEd Arne Duncan, believed the lie. Only the hard, cold reality of testing could free us all from our delusions.

Now, I've been around a lot of testing skeptics, and I can assure you I've never once met one who was convinced that schooling in disadvantaged neighborhoods was generally acceptable. I've never met a union official who believed schools in impoverished cities didn't need improving. I never met anyone who works in a school or advocates for public education who was fine with the opportunity gap that plagues so many children in this country.

But I'll set that aside and instead make this point: stories like the Trentonian's give us clear evidence that kids who are in these schools themselves know full well what is going on. They are saying, with unmistakable clarity, that their instruction is unacceptably poor. They are telling us many of their peers have given up and have no interest in school.

What are multiple administrations of standardized tests going to tell us that these kids aren't already telling us themselves?

As I said last time: I am all for reasonable accountability testing. Test outcomes have been used by researchers and advocates to show indisputable truths, such as:

1) When it comes to school, money matters. A lot.
2) Children who come from disadvantage need more resources to equalize their educational opportunities.

Some make the argument that we have to have tough tests to motivate schools to improve. Most of these people don't know the first thing about how these tests are constructed or what they actually measure, but even if they did: What, exactly, do they want to do with the information they get from these tests? 

Do they really want to deny students, who went to their classes and played by the rules, their diplomas? What will that accomplish? Will it make the students and teachers "do more with less"? Will it improve the lives of students who don't pass these much more challenging PARCC and upcoming PARCC-like tests?

The idea of requiring students to pass tests with current passing rates of 46 percent strikes me as both cruel and capricious. Cruel because denying graduates a chance to participate in the workforce or join the military or pursue higher education is unnecessarily harsh when these students did exactly what they were told to do. Capricious because we changed the rules on these students quickly and with little forethought, and we didn't even stop to ask if their schools have what they need to help students pass these tests.

Now, if these same people who are demanding more and harder tests were also the same ones demanding full, adequate, and equitable funding for schools, I'd be more amenable to their arguments. Unfortunately, however, these folks -- like Duncan -- don't ever put school funding at the top of their lists of policy preferences.

I've heard a lot of crowing from certain quarters about how great it is that Trenton is getting a new high school in 2019. Certainly, the kids deserve it... but why did it take so long when high school kids in Trenton have been attending a school that is dirty and dangerous for years?

The infamous "Waterfall Staircase" at Trenton Central.

And what about all the K-8 schools in Trenton that are in need of repair and renovation? Where is any urgency to address this?

A couple of years ago I gave a presentation to the Trenton Education Association. Here's one of the slides:



Why are plant costs so much higher in Trenton? Because it costs much more to maintain old buildings that weren't properly cared for over the years. TPS is playing a constant game of catch up. Why?



Because the district has been underfunded millions less than what the state's own law says it needs.

Again: if the folks demanding harder tests were proposing that Trenton, and all other districts, got the funding they needed to at least generate average test outcomes, I'd at least say they were being honest about what it takes to close the opportunity gap. But no -- what we get from them instead is school "choice" and finger-pointing at unions and test-based teacher evaluation and a lot of other stuff that has never been shown to work and/or been brought up to scale... and in some cases actually drains more money away from the schools serving the most disadvantaged students.

This kids in Trenton are telling us the their education is not equivalent to the schooling kids in the Windsors and Princeton receive. We don't need more tests to tell us that -- the kids themselves are saying it.

The only question left is: what are we going to do about it?