COVID and "Learning Loss": Test Scores Should Not Be Our Immediate Concern

I'll get back to Lakewood in a bit, but I want to take a minute and talk about a new report out yesterday from JerseyCAN and others about learning loss and COVID-19.

Regular readers know I've had my issues with JerseyCAN over the years: too many times, their analyses have missed the mark, often because of a lack of understanding about things like what test scores actually represent and how they should be used to direct K-12 education policy. 

I might make similar complaints about this latest report. Using a convenience sample's outcomes on one test to extrapolate outcomes on another for an entire population is inherently problematic. Further, changes in proficiency rates on the same test are often artifacts of the process of setting those rates, and not indications of any changes in actual student learning. There's also a whole problem of equating "proficiency" across grade levels that tends to get pushed aside in these discussions. Plus the easy way we accept "grade level" as some sort of absolute when it's really a social construct...

But in the end, none of that really matters, because what the report is showing is almost certainly correct: student learning has suffered during the pandemic, and the losses are almost certainly greater for students of color and those in economic disadvantage. 

How could this be otherwise? There is a clear digital divide along racial and ethnic lines in this country; when learning moves on-line, the effects will reflect that divide. And, as I've shown repeated, access to in-person schooling has also been racially unequal, a reflection of the inequalities in New Jersey's school funding system. These factors are undoubtedly combining to disadvantage students of color and students in economic disadvantage. 

I suppose there may be some value in presenting these losses as changes in test outcomes, as there are plenty of policymakers in the state who act as if test scores are the only measure by which we can evaluate the quality of schooling. But in all honesty, the report is telling us something we already knew: the pandemic has been bad for students, and worse for some than others.

So if JerseyCAN had stopped there, I'd really have very little problem with their report -- at least, the problems I'd have would be confined to technical details. But they took upon themselves to tell the state what it must do given their findings:

JerseyCAN recommends that stakeholders and policymakers consider the following solutions to help accelerate student learning for our students.

• Urgently prioritize the adoption and statewide implementation of extensive summer programming so that we can stem the COVID slide now and further stop more students from falling behind;
• Adopt and implement personalized, research-based solutions for accelerating student learning like high-dosage tutoring;
• Allow parents to exercise their choice to retain or hold back their child, if desired, to provide additional time to students for learning and the provision of social and emotional supports;
• Incentivize all districts to adopt high-quality instructional materials that are aligned to statewide assessments, which canprovide teachers and parents with ongoing information about student academic growth and that can project proficiency on NJSLA; and
• Administer statewide assessments in Spring 2022 that are comparable to those administered in Spring 2019 to establish a new baseline from which to measure student growth moving forward and to also enable comparisons to pre-pandemic statewide proficiency.

Let's take these one at a time.

• I am all for high-quality summer programming. Summer slide is a real issue, especially for economically disadvantaged children. But if our metric for determining that "loss" is test scores, we're running the risk of creating programs that sit students in front of screens all summer long doing test prep. As an educator, it is my opinion that this is the last thing kids need after this very difficult year. How about a lot of physical activity and socialization, music, art, free reading, exploratory learning -- what those of us in the education research field refer to technically as "fun"! In fact, we could do that, and add access to counseling and health services, family engagement programs, early childhood education... golly, if only there was a model of schooling that did this...
• Tutoring can be good -- but you have to remember a few things. Tutors need training. Good tutoring is not cheap. Tutoring is a supplement for high-quality curriculum and instruction, not a replacement. Again, if all we're doing is getting lower-paid workers to sit in classrooms with kids while they drill on test-prep instruction, we're missing an opportunity to help kids address the trauma they've experienced this past year.
• Expanded grade retention is probably one of the most overhyped policies that has been pushed over the past couple of decades. There is good evidence that the games Florida has played with retaining kids was behind its unearned status as an educational "miracle." While I am certainly sympathetic to the idea of giving parents more of a say in the decision to retain their child, and while I do believe there are times when it is warranted, pushing retention immediately after a cohort has shown learning losses doesn't make a lot of sense: if everybody suffered, why hold only some kids back? Besides, are our schools really ready for a mass mixing of age groups? 
• Everyone is for high-quality materials. But if the primary goal of those materials is to increase test scores, they will run the risk of emphasizing only those skills needed to pass the test, which can narrow the curriculum and constrain instruction. I am not against using test scores judiciously as a data point in helping to assess a child's learning, but state tests are simply not a practical, nor particularly useful, way to inform student instruction -- especially when teachers can't see the exam.
• There are always items put in different administrations of standardized tests to help equate scores. And, to the extent possible, proficiency rates should be equivalent between different years. So, sure, we can use test outcomes to help determine where we stand. But what really matters is this: What are we going to do with those results?

This gets to the heart of my concerns about the premise behind JerseyCAN's report, which I see manifesting in the statements of other education policy stakeholders. We are coming through an unprecedented crisis in modern American history. Children are hurting. Families have been traumatized. Schools have suffered losses in their staffs. We haven't faced this kind of trauma since World War II. So what are these folks proposing to help kids?

Testing. Analyzing test scores. Instruction based on test outcomes in two areas of the curriculum. Tutoring in support of that test-centered instruction. Then... more testing.

Look, I like a good run at the data as much as the next guy. I've said consistently testing has its place. But our kids are hurting, and the last thing we should be doing is trying to recreate the test-obsessed pedagogy we were using before the pandemic. Further, we shouldn't be going back to the test-and-punish systems we used to hold schools and educators "accountable," which was doing nothing to address the root causes of unequal educational opportunity, like inequitable school funding.

We should instead be focused on getting kids to love school -- especially if they weren't loving it before the virus hit. We should get kids and families access to the health and emotional supports they need so students can arrive at the schoolhouse door ready to learn. We should be investing in our school infrastructure to make our buildings safe and healthy. We should be expanding things like arts education, a proven strategy for boosting student engagement and learning.

A less stunted, more comprehensive view of schooling is required at this time. Testing can be part of it, but students, not test scores, should be the focus.

Now, back to Lakewood...

Lakewood, NJ: Where Public Schools Are Left To Wither, Part III

 Here are the other parts of this series:

- Part I

- Part II

What happens when a majority of citizens rejects its own public schools? We need look no further than Lakewood, NJ -- a town that I contend is an object lesson in the consequences of school privatization. Again, I believe at least three things will happen when public schools are left to wither, as they have been in Lakewood:

1. When a large part of a community abandons its public schools, chaos ensues and accountability dissolves.
2. Segregation is the inevitable consequence of school privatization.
3. Educators will be disrespected in a community that does not support its public schools. 

Regarding point #2: in the last post I showed definitively that Lakewood's public and private school populations are highly segregated. In this post, let's spend some time discussing point #1.

I'll note first that, as I've written before, there are plenty of examples of corruption and malfeasance in districts where the majority of children attend the public schools. And the problems with private schools enrolling special needs students in New Jersey have been well documented. 

But one of Lakewood's private schools had, according to the verdict of a jury, taken things to a new level (all emphases mine):

2/27/19: NEW BRUNSWICK - The founder of a Lakewood special education school is guilty of money laundering and misconduct by a corporate official, but not guilty of corruption involving public funds, a jury said Wednesday after deliberating for about 20 hours over four days.

The jury's mixed slate of verdicts expose Rabbi Osher Eisemann, founder of the School for Children with Hidden Intelligence, to possible prison time of five to 10 years, according to prosecutors. He was acquitted of three charges, including the most serious count brought against him: First-degree corruption of public resources, which carried a prison term of at least a decade. 

Eisemann left a courtroom Wednesday afternoon surrounded by supporters and family members, who have occupied courtroom benches throughout the trial. He smiled as he thanked his defense team, which declared victory shortly after the verdicts were announced.

"The reason you saw the defense team is enthusiastic overall about the verdict is this case is not about money laundering or misconduct by a corporate official," said Eisemann's lawyer, Lee Vartan. "This was about theft of public funds. The state made that clear in its multiple press releases, at every court appearance, in their opening statement, in their closing statement. And on theft of public funds, he was completely vindicated."

It turns out the defense team had reason to be optimistic: 

4/29/19: NEW BRUNSWICK - The founder of a Lakewood special education school will not go to prison, but will serve 60 days in jail and two years on probation for laundering $200,000 from the school, a judge here ruled Monday afternoon.

Rabbi Osher Eisemann, who founded the School for Children with Hidden Intelligence, must also pay a $250,000 fine, according to the sentence handed down by state Superior Court Judge Benjamin Bucca, sitting in Middlesex County. Eisemann will begin the jail term on July 1. 

[...]

"The defendant is many things," Deputy Attorney General Anthony Robinson said. "He is a builder, a nurturer and an educator. But he is not a victim. ... The defendant has led a great life but that does not exempt you from the law.”

But Bucca found even the minimum prison term, six years, would be a "serious injustice" in part based on Eisemann's work in special education. More than three dozen letters were sent to the judge in support of Eisemann, many of them outlining Eisemann's dedication to students at the school he started to help his own son. 

The prosecution was not happy with this ruling, however, and appealed the original sentence. The appellate judges, according to the Asbury Park Press, "...issued a scathing opinion saying the judge who sentenced him to probation for money laundering and misconduct ignored the law and a jury’s verdict." At last report, Eisemann's sentence is on hold as his attorneys appeal the latest ruling.

It's worth noting that this would not be the only time SCHI found itself in trouble:

4/19/19: LAKEWOOD — A high-cost school for special-needs children, whose founder and director was recently indicted on charges he stole hundreds of thousands of dollars in public money, overcharged the township's impoverished public school district and other schools by at least $340,000 in one year, the Asbury Park Press found.

The School for Children with Hidden Intelligence overbilled the districts by paying uncertified teachers; paying certain employees more than permitted by state law; and buying items deemed unnecessary for the school, according to an audit by the state Department of Education of the 2011-2012 school year. Some of those items were purchased at Costco, Staples and from merchants on eBay and mailed to the home of an employee not named in the audit. A large-ticket item, a power generator, was ostensibly purchased for the school's summer camp, but it could not be located by auditors. 

The school also failed to do background checks on 71 of the 77 employees it hired that year, the audit found. 

Rabbi Osher Eisemann, the school's founder and current director, was indicted earlier this month for allegedly stealing more than $630,000 in public funds and laundering much of the money. He was not blamed for the overbilling in the state's audit.

Now, I will confess that when I started following this story, my first reaction was: "Where was the oversight? Why wasn't the Lakewood Board of Education aware of this?" But the more I've thought about it, the more obvious it became to me that it's unreasonable for a board of education to oversee the proper use of monies when so much of those monies are spent by entities that are not regulated by that board.

Look at these figures again:

The rest of New Jersey has to keep watch over less than one percent of its students who are placed into private schools. Lakewood, however, has to oversee the placements of nearly six percent of its students.

The amount of spending on private schools is, on average, six percent of a New Jersey district's budget. But nearly sixteen percent of Lakewood's budget is spent on private schools. How can a local board of education keep track of this much money if it is given to entities out of their control? Local districts have enough to do keeping track of their own finances, let alone audit the accounts of private schools.

Eisemann's jury found he engaged in a complex financial scheme to evade the law. It took years for prosecutors to develop the case and win a conviction. It's not reasonable at all for any school board to be  expected to catch a private entity in this kind of act. But if that's true, it stands to reason that bad faith actors will see an opportunity and try to game the system. And when there is a lack of accountability, bad things inevitably happen.

Should New Jersey completely eliminate private schools for special education students? That's a question on which reasonable people can disagree. In my own career I've worked with people who run these schools, and they were all decent, committed professionals who had the best interests of their students at heart. There are much easier ways to make a living than running a school for kids with the most profound special needs, so I give anyone who enters the field the benefit of the doubt.

But public policy can't be run on faith. And the more public revenue that is doled out to nongovernmental entities, the more oversight is required. Lakewood gives out a lot of money to private interests, but it clearly hasn't figured out how to hold all of those entities fully accountable. How could it? Boards of education were never designed to be overseers of private schools... but, in this case, it's precisely what the situation requires.

But now imagine if we expanded the subsidies so all students, not just those with special needs, could attend private schools. Who would provide the necessary accountability? Boards of education clearly aren't up to the job. Who instead would protect the public's interest? Who would be in place to make sure public monies were being spent responsibly? For that matter, given how much support Lakewood provides to private schools right now...

... who's making sure that those private schools are spending it properly?

Point #3 on my list mentions disrespect for teachers. Let's talk about that next...

Lakewood, NJ: Where Public Schools Are Left To Wither, Part II

Here's Part I of this series.

In this series, I'm taking a look at Lakewood, NJ, with an eye toward what might happen if more communities abandon their public schools and, instead, embrace the idea of school "choice." Lakewood, in a state by some accounts with the highest ranked public schools in the nation, is a town where the vast majority of families choose to send their children to private schools.

The Star-Ledger explained what's driving this phenomenon back in 2017:

Lakewood has more than 6,300 students registered in its public schools and another 30,000 mostly Orthodox Jewish students enrolled in the town's 130 private schools. Under state law, towns must fund buses for students attending private schools more than two miles from their houses.

In the upcoming school year, Lakewood expects to spend $27 million on busing alone, more than it spends on classroom instruction, according to the school budget.

Town leaders say Lakewood is severely underfunded by the state, which does not consider the busing of large numbers of private school students when calculating how much state aid the district receives. A permanent source of school funding for busing and special ed would ease many of the town's problems, said Rabbi Aaron Kotler, one of the leaders of the Orthodox Jewish community.

"It would also make Lakewood a place where you wouldn't have people feeling that your children's future is at risk due to a budget problem," Kotler said.

As we'll see throughout this series, the argument that Lakewood is "severely underfunded by the state" is open to debate. For now, let's do what regular readers know always comes next: take a data dive into Lakewood's schools. 

Remember: the data I'm using above, given a very conservative estimate, suggest private schools students outnumber public school students by at least 4-to-1; the Star-Ledger's report suggests the real ratio is even higher. Let's start by looking at the population in the public schools.

Overwhelmingly, Hispanic students make up the population of the Lakewood Public School District -- far more than the rest of Ocean County, and far more than the rest of the state. It's worth noting that, as Bruce Baker and I point out, one of the best predictors of whether a school district in New Jersey is underfunded according to the state's own law is whether that district enrolls an inordinately high number of Hispanic students.

Lakewood's public schools students are also overwhelmingly in economic disadvantage, as measured by eligibility for free or reduced-price lunches. 

Given this population, it's not surprising that Lakewood Public School District's Limited English Proficiency (LEP) rate is very high: over five times the state average.

To recap: the public schools in Lakewood enroll students who are overwhelmingly Hispanic and disadvantaged, with many who do not speak English as their first language. 

Now, we don't have the same data for the private school students. But looking as the religious affiliations of those schools gives us much of the information we need.

The vast majority of private schools students in Lakewood go to Jewish schools. As I said in the previous post, the religious denomination of the private schools students isn't important: what's important is that the majority of the town's families have opted out of sending their children to the public schools. This is, for many in the school "choice" movement, exactly what they'd like to see: a large number of a community's families choosing not to send their children to local public district schools.

There are at least two consequences to the majority of Lakewood's families rejecting public education. First, the student population of the public schools is very different from the private schools. Second, the majority of the town's families do not have a personal interest in the public schools' success. Let me be clear: I'm not saying they want the public schools to fail; rather, whether those schools fail or succeed does not impact them personally.

This has profound effect on the Lakewood Public School District's budget. The local school board is elected by a citizenry that is not invested in the public schools in the same way as is found in most other towns and cities. Whose interests, then, will the district's budgetary decisions serve?

As I've written before, New Jersey must, by law, provide funds for all sorts of supports to private schools attended by local resident children: technology aid, security, nursing, textbooks, handicapped services, and so on. In general, it's a very small amount of the overall K-12 school budget – except in Lakewood. A huge part of the Lakewood school district's budget is spent supporting the needs of students who do not attend its schools.

The Star-Ledger also points out the district spends a lot of money on transporting private school students. The state data doesn't allow us to separate out that cost from the cost of bussing public school kids, but clearly there is an effect: Lakewood's transportation budget is very large.

There's another expense Lakewood incurs far in excess of other New Jersey school districts: private school tuition for special education students. Back to the Star-Ledger:

Lakewood expects to spend nearly $32 million on tuition this year to send special education students -- including hundreds of members of the growing Orthodox Jewish community -- to private and out-of-district schools. The bill is among the highest in the state and one of the reasons the booming Ocean County town is facing a school funding crisis. 

Why do taxpayers pay private school bills?

Under federal law, all children with disabilities are entitled to a "free appropriate public education." So, if a school district does not have the staff, programs or facilities to accommodate a child's specific disability, parents can request that the district pay for a more appropriate private school.

Many parents turn to lawyers and education consultants for help. Their cases often end up before a state Office of Administrative Law judge, who rules where a student should be placed.

In Lakewood, an estimated 361 current and new special education students will be placed in private schools at the district's expense during the upcoming school year, according to the school district's budget. Another 17 students will be sent to other school districts.

Lakewood sends more than a quarter of its special education students to private or out-of-district schools, according to the data.

If you look at the percentage of all students on-roll who attend private schools, the figure is, once again, far in excess of the rest of the state.

Lakewood's private school placement rate is nearly ten times that of the rest of Ocean County. Again, this has a powerful effect on the public school district budget.

Lakewood spends more than twice what the rest of New Jersey's districts spend on tuition to other schools.

So, given the data, what can we surmise about Lakewood?

• The majority of the population has opted out of sending their children to the public schools.
• That majority is ethnically separate from the minority that sends their children to the public schools.
• The budget of the public school allocates inordinately large sums in support of the private school population.

Let's be clear before we continue: these are facts. We can debate the cause, but the data are unambiguous on what is happening right now in Lakewood, NJ's schools.

The question we must address next, however, has greater consequences for the larger conversation on education policy: What happens when a community abandons its public schools? Again, I contend there are at least three consequences:

1. When a large part of a community abandons its public schools, chaos ensues and accountability dissolves.
2. Segregation is the inevitable consequence of school privatization.
3. Educators will be disrespected in a community that does not support its public schools. 

The data clearly support point #2. In the next post, we'll further explore point #1.

ADDING: I'm using 2017-18 data as that is the last year for which we have audited, actual figures for the Lakewood Public School District, as opposed to projected or revised data. Lakewood is not included in the latest NJDOE fiscal dataset (User Friendly Budgets) for 2020-21. 

Lakewood, NJ: Where Public Schools Are Left To Wither, Part I

If you follow K12 policy, you've probably noticed that the school privatization crowd has recently become reenergized. The folks from right-wing think tanks and the professional school "choice" promoters have been pushing the idea of school vouchers hard, certain that the pandemic has created widespread dissatisfaction with our current system of public school districts.

Many times I've seen these people make the argument that private schools are fully open, and the only reason public schools are not is the intransigence of teachers unions and fecklessness of school boards. Of course, it's hardly that simple: consider that many "no excuses" charter schools -- free of both unions and BOEs -- remain closed. We also really don't know how many private schools are open, or to what extent, or with what consequences.

Nonetheless, the privatizers see an opening. And considering the number of wingnuts out there who think COVID isn't a serious threat that requires extraordinary actions to secure schools, they may be right. School privatization was, to my mind, always a secondary part of the overall conservative agenda, but that may have changed, if only in the short term.

Given the renewed focus on vouchers, I think we're justified in taking some time to really think about what it would look like if we embraced the idea of privatized schools in a "choice" system. What would it look like? If they still existed, what would the public schools do? Which students would go where? How would it affect the community? How would it affect the taxpayers? 

In the upcoming series of posts, I'm going to argue that we don't need to speculate; we already have at least one example, right here in New Jersey, of what will happen if a community largely abandons its public schools. 

Welcome to Lakewood*:

An administrative law judge has determined the public school district cannot fulfill its constitutional mandate to provide students a "thorough and efficient education," but stopped short of determining that the state funding formula is unconstitutional. 

In a non-binding decision Judge Susan Scarola recommended that the New Jersey Commissioner of Education conduct a needs assessment of the school district's ability to meet its obligations and make "appropriate recommendations to the district."

The decision scarcely binds the Acting Education Commissioner Dr. Angelica Allen-McMillan to do anything; the recommendation may be adopted, modified or rejected by Allen-McMillan. But the harsh assessment — that the school district cannot meet its constitutional mandate — puts fresh focus on the troubled school district and pressure on the state's top education official to intervene.

The 6,000-student district faces chronic budget gaps, owing in part to a state school-funding formula that fails to account for the fact the district must also pay transportation costs for 37,000 Lakewood students attending private schools.

Yes, that's right: only a small fraction of Lakewood's students attend the public schools. And, yes, that is very unusual -- especially for a state like New Jersey.

Private school data in the United States is at the mercy of survey methodology; in other words, it's very easy to make an incomplete count of private schools students in a community, because it's hard to get to every school, especially the small ones. The Private School Survey from the National Center for Education Statistics is, however, the best data source we have on private schools. When combined with state data, it shows a remarkable story about private schooling in Lakewood. Across the state, a little under 10 percent of students attend private schools. But four in five Lakewood students are in a private school, way more than the state and way, way more than any other town in Ocean County.

A few years ago, the Star-Ledger did an extensive series of reports on Lakewood, dubbing it "N.J.'s most controversial town." The first report gives some background:

Lakewood is booming. Thanks to an influx of Orthodox Jews, it has been New Jersey's fastest-growing town over the last 20 years. It has one of the highest birth rates in the world. Housing is going up at an unprecedented pace.

Why are there so many Orthodox Jews in Lakewood?

Lakewood had long been a popular resort for Jewish vacationers when a refugee rabbi fleeing Nazis in Lithuania arrived in the 1940s to start Beth Medrash Govoha, a higher education institution where men could study the Talmud, one of the religion's sacred text.

The yeshiva, known as BMG, grew over the years into one of the largest schools of its kind in the nation, with more than 6,500 students. A thriving Orthodox Jewish community grew around the yeshiva, which included many Jews from Brooklyn's Orthodox community looking for less costly housing and a more suburban setting to raise their families.

Who else lives in Lakewood?

The population exploded from 45,000 in 1990 to more than 100,000 now, making it New Jersey's fifth-largest city.

The town is 84 percent white, 17 percent Hispanic and 4 percent black, according to the U.S. Census. There is a growing Mexican immigrant community and a large community of seniors in sprawling retirement communities, including Leisure Village. The African-American community has been shrinking in recent years.

This last paragraph is perhaps the key to understanding Lakewood's schools: while there is a large and growing Orthodox Jewish community in Lakewood, it is not the entirety of the town's population. The Hispanic and Black students in Lakewood are not being served by the town's large network of yeshivas; instead, they are enrolled in the public school district.

Now, it's clear that very few communities in the United States are exactly like Lakewood. But that doesn't mean there aren't lessons about school privatization to be learned. I have three:

1. When a large part of a community abandons its public schools, chaos ensues and accountability dissolves.
2. Segregation is the inevitable consequence of school privatization.
3. Educators will be disrespected in a community that does not support its public schools. 

It's going to take several posts to tell this story, because it's big and complex. Fortunately, there has been some excellent reporting on Lakewood's schools, particularly from the Asbury Park Press; I'm going to lean heavily on them as I work through this series.

But in spite of the story's many twists and turns, I think it's an important one to tell it. Lakewood is a cautionary tale, and the ethnicity of the characters in it really doesn't matter much. What's important is that we understand what happens when public schools are left to whither in a community that has largely abandoned them.

I'll dig into the data next.

* All emphases in this post are mine.