Beyond ‘failing schools’: The difficulty of fairly comparing public schools on uneven playing field

File this story under “wonky but important.”

In an era where standardized test scores often determine a school’s reputation and, ultimately, its survival, comparing and judging schools based on results has become a commonplace ritual.

Parents often use the scores as a North Star for deciding where to enroll their kids. Districts often use them when considering which schools to shutter or to turn over to new management.

Within this context, a toxicity has brewed between advocates of Philadelphia’s traditional district schools and proponents of charters — built mainly on the premise that the two sectors aren’t playing by the same rules.

Tension between the two have festered especially in recent years. Steady student migration from traditional public schools to charters caused dozens of schools to close in 2012-13. Funding debates have become especially fraught since Gov. Tom Corbett’s 2011 budget, which cut the state aid that helped districts defray the added costs of charter schools.

But those debates often gloss over some of the important differences between and within the sectors that have a very real effect on a school’s perceived quality.

A prime example of this is the concept known as backfilling.

Hypothetically speaking, say a charter school is authorized to serve up to 500 students, but, for whatever reason, 50 students leave through the course of a school year. A charter that “backfills” will enroll the next 50 kids on its wait list as space becomes available.

Other schools will replace those empty spots at the beginning of the next school year, including filling seats in the upper grades.

Charters that don’t do this will watch their total enrollment in a grade dwindle year by year — retaining only the students tenacious enough to persist.

In contrast, district-run neighborhood schools and renaissance charters must enroll all students living within a prescribed catchment zone, no matter what time of year or grade, when they show up asking for a seat.

At first glance this difference may seem a subtle nuance, but Philadelphia educators say the policy difference tremendously affects school culture and performance.

“Most of the time, the students that we start off with in ninth grade with us in September, they’re usually fine. That’s usually not the problem,” said South Philadelphia High School principal Otis Hackney. “The most disruptive part is really getting new students throughout the year.”

Hackney says his school enrolls new students almost daily. Some return from the district’s alternative placement schools; some have had chronic truancy issues; many, he says, have been “coached” out of charter schools.

“And that’s a real experience. We do keep track of where students come from,” he said. “They don’t always come with the most stellar grades or the behavioral disposition that you would want, but we work hard to make sure they understand quickly what they need to do here.”

Hackney says his team does well with limited resources, but the constant influx of new, often more-challenged students, makes progress more difficult.

“Even if you have a school with a strong culture, new people coming in, they have to learn what the expectations are,” said Hackney. “You have to constantly remind them: This is how we do things here in South Philadelphia High School. This is how we handle crisis. This is how we handle conflict.”

Rigor and attrition

Charter schools, independent by design, cannot be painted with a broad brush. It’s no different when it comes to backfilling.

Some charters actively work to keep attrition rates low and backfill whenever possible — seeing it as a chance to whittle down long waiting lists. Others proudly defend the fact that they don’t take in new students, pointing to the rigor of their coursework and the monumental advances of the students who remain.

Kelly Davenport, head of school at Freire Charter, falls into the latter category.

Freire tests all students in reading and math prior to enrollment in order to plan for each student’s needs.

“If they come in the middle of the year,” she said, “it’s like, how do we make sure that we’re addressing their needs, their skill deficits, where the holes and gaps are from the beginning?”

David Hardy, CEO of Boys’ Latin, subscribes to the same theory. He oversees a rigorous admissions process that begins well before the school year.

Boys’ Latin asks prospective ninth-graders to submit letters of intent in November, nearly a year before they would enroll. Staff then interview students and parents to ensure that they understand the school’s rigor — classes run until 5 p.m., students must learn Latin, wear a uniform, and adhere to a strict code of conduct.

Those who commit attend a month-long freshman academy in July before the school-year-proper begins.

By September, he said, the kids are all on the same page.

“You introduce new people into that, and it can kind of mess up the environment,” said Hardy.

Backfilling would be especially difficult at these schools because both have high rates of student attrition.

In general, in a school that backfills, the more students exiting midyear, the more challenging it is for teachers and the students who remain to adjust to an influx of new kids.

The infographic below illustrates how Freire’s high attrition rate reduced the Class of 2014 by more than half.

Similarly, the 238 students in the Class of 2013 who began their freshman year at Freire dwindled to 113 by senior year – a 52 percent attrition rate.

This picture from last year’s graduation ceremony shows that male students are more likely to leave.

 

Growing up is about earning more choices. You made the right ones, that’s why you’re here today @FreireCharter pic.twitter.com/Dh2E7bagVY

— Darrell Clarke (@Darrell_Clarke) June 16, 2014

 

“The attrition has hit the boys harder than the girls over time,” said Davenport. “We don’t like that trend.”

Freire is actively working to retain more of its boys through hands-on programs such as robotics and media production, she said.

Boys’ Latin showed a similar attrition pattern in grades nine through 12.

*Boys Latin last year served grades six, and nine through 12. It has an agreed-to enrollment cap of 800.

So why do kids leave?

“Normally it’s because a student doesn’t want to take the volume of the work that we do,” said Hardy. And when that happens, “there’s a mechanism to kind of go after them.”

At Boys’ Latin, this process includes tutoring, probation, and Saturday school.

“If the student makes the decision that they really don’t want to do the work, then what’s the point?” Hardy said. “That normally is a family decision. They say, ‘Why keep him in a school that he really doesn’t want to be in?’ That’s the beautiful thing about school choice.”

Hardy says his school doesn’t actively push kids away.

“You can stay here and fail,” he said. “But that doesn’t make sense, does it?”

Davenport, of Freire, says questions of backfilling and attrition reveal a “central tension” of public education.

“Is the city calling Freire charter school to educate all kids, every kid, and provide a basic, standard education that will fit all of their needs -– sort of a one-size fits all, with a basic graduation diploma?” she said. “Or is the city asking us to produce kids that have the grit and tenacity and ability to persevere through rigor and really challenging academic work?”

Davenport says, ideally, she aims to run a school that gives all students that ability. But she acknowledged that, so far, ditching the neighborhood school enrollment model has been a part of Freire’s success.

“We believe it is our job to produce kids who will take part in the leadership of this city,” she said, even if they “can’t afford private school” or “get into the magnet schools.”

Even playing field?

Advocates for the city’s most vulnerable students have long criticized these enrollment practices.

Education Law Center staff attorney David Lapp says they tend to push the students with the greatest needs to neighborhood schools like South Philly High. He says this makes the job of improving neighborhood schools more difficult, and it makes the high-stakes process of comparing schools impossible to do fairly.

“Refusing to backfill, refusing to replace students that leave your school, is essentially cheating the system,” Lapp said. “If we’re making those comparisons, and we’re going to look at their ‘performance,’ then we need to make sure we’re all playing by the same rules.”

Lapp believes a court case could be made that refusing to backfill undermines Pennsylvania’s charter law.

Hardy has little patience for those urging Boys’ Latin to be more like neighborhood schools — pointing to the fact that his school sends more African-American boys to college than any public school in the city, including magnets.

“The thing about charters is, you have to figure out a way to make what you’re trying to do work. And that normally means not doing it the way the district does it,” he said. “The question should be: Why don’t they act like us?”

The inherent flaw in Hardy’s logic, Lapp said, is that Boys’ Latin’s methods can’t be adopted systemwide.

“There’s no way that we can have all the schools in Philadelphia say, ‘We don’t enroll anybody else in the middle of the school year,'” said Lapp. “We’d have thousands of students with nowhere to go to school at all.”

The district’s magnet schools do backfill, but comparing them to neighborhood schools or independent charters is difficult. Magnets have the freedom to select their students up front, and they very rarely lose those students.

Pressure from other charters

More recently, other charter school leaders have criticized those who don’t backfill.

KIPP’s network of city charters has evolved over the years to be more sensitive to the systemic effects of closing their doors to new students. Now, backfilling is the law of the land in KIPP’s four Philadelphia schools.

“As we thought about it, it was really about: if we think we are doing a good job, and our kids our achieving — we’re adding value — then why aren’t we trying to serve as many kids as we can?” said KIPP Elementary Academy principal Ben Speicher.

In a recent interview at his Nicetown school, Speicher acknowledged there was plenty of internal hesitation before backfilling.

Some feared that it would be harder to have the same high expectations with kids trickling in throughout the year.

“I think we just had to take a risk and say, ‘We think we’re good enough. Our culture is strong enough. I think we can hold on to what makes us special,'” said Speicher. “If your attrition is low, and you’re only having to replace a handful of kids, it’s not hard. And it should be something that I think that everybody ought to be doing.”

Mastery Charter and Scholar Academies both accept new students at every grade level throughout the year.

“I think it’s important that all charter schools move towards the same admissions policies that all public schools have.” said Mastery CEO Scott Gordon. “If there’s a parent who wants a child in your school, and there’s an opening, you make sure that parent has an opportunity to enroll that child.”

“Charter schools should serve students in the same way that traditional public schools do,” said Scholar Academies founder and CEO Lars Beck. “We want to have success in the same way that public schools serve students.”

View from District HQ

The leadership of the Philadelphia School District agrees.

“We don’t want there to be an influx of children all year from one sector to the other, going either way,” said Superintendent William Hite. “We have enough of that already just in normal attrition. So we don’t need more of that that is intentional just because a child is misbehaving or not learning fast enough.”

Recently, the Philadelphia School Reform Commission approved five of 39 applications for new charter schools. KIPP and Mastery were awarded new charters, as was Freire’s bid to open a new high school centered on computer science and entrepreneurship. Scholar Academies did not apply. Hardy’s bid to open Girls’ Latin was rejected.

The district’s new authorizing guidelines specifically call for charters to backfill, and each of the new agreements, including TECH Freire, mandate the practice.

Hite calls for every charter in the city to agree to participate in a new accountability progress report that will track student retention and compare schools serving similar demographics.

“It does provide a more level playing field if we can address those issues,” said Hite.

At Freire, going through the recent charter application process pushed school leaders to change their thinking a bit.

The school has become more willing to accept students midyear. A case in point: Freire took in about a dozen fifth-graders who were among the hundreds of students displaced by the sudden closures of Walter Palmer and Wakisha charter schools this year.

At Boys’ Latin, Hardy said the district has specifically asked him to begin backfilling for the 10th grade, but he hasn’t yet made a decision.

“I don’t think I have a pass into heaven, but I’ll tell you, I want to know how you get into heaven when you spent your days really limiting opportunities for children,” said Hardy. “And that’s what I see a lot of people in this city doing.”

The chart below compares the demographics of the three high schools mentioned in this story, and includes scores on the state school performance profile – an index mainly composed of performance and growth on state tests.

2013-14 Total enrollment Male English language Learner Economically disadvantaged Special ed SPP South Philly High (9-12) 1056 60% 18% 97% 26% 43.6 Boys Latin (6, 9-12) 664 100% 0 66% 9% 46.3 Freire (5-12) 947 41% .4 88% 13% 71.2