Report cards for charter schools going public over loud opposition

A moment charter schools in Philadelphia have been dreading arrived Wednesday when district officials made their first public presentation of a school rating system many charters think is a mess.

It’s called the School Performance Index, and the district has used a version of it to decide which low-performing traditional schools should be restructured under Superintendent Arlene Ackerman’s Imagine 2014 program.

A rocky road

The district has been working on a similar measure comparing charter schools and public schools since August, which it says will be used to determine charter renewals and expansion requests.

But this process has been a rocky road.

Many charter schools say they would welcome an honest evaluation of their work – but the district’s measure is not it.

“I don’t think very many charter operators want to avoid accountability; that’s not the problem,” said David Hardy, CEO of Boys Latin of Philadelphia Charter School. “I think where the problem is is that nobody trusts the school district’s numbers.”

All 11 charter providers interviewed for this story said they felt the SPI was deeply flawed. Many have been giving feedback to this effect for months.

Nevertheless, the school district went ahead with a public presentation of the SPI process for charters, which was preceded several days before by the private release to all charters of the final SPI scores for each school.

A tool for all

Besides its use in reviewing whether to renew a charter, the tool could be a useful took for parents trying to decide which school is right for their child, district officials say.

David Weiner, the district’s associate superintendent of academics, sees the SPI as a fair measure of the performance of all types of schools

“Our biggest goal is to [take] out the propaganda that charter schools, and Catholic schools and district schools use,” Weiner said.

Numbers problems

But charter leaders say the numbers behind the district’s ratings are riddled with errors. One example: When initial SPI ratings were released privately to each charter in the fall, some went to the wrong schools.

District officials counter that the scope of the errors has been small.

“We’ve found a couple of errors here and there but nothing major,” Weiner said.

The charters dispute that statement.

At Independence Charter School – which scores very well in the district rating and is a city-wide favorite among parents – the district somehow missed that 450 students in the school took state tests last year. It only gave the school credit in the SPI for 12 test takers, 3 of whom it incorrectly attributed to other schools.

What happened to those 438 lost test scores?

“I’m wondering are my kids helping someone else to score better?” said Jurate Krokys, CEO of Independence. “The whole process is filled with problems.”

And since the district initially developed the measure in a way that excluded charters (Scott Gordon, executive director of Mastery Charter Schools called it a “black box” when it first began), many charters have lost faith in the district’s motivation.

“Everybody in the charter school community pushed back immediately because they were not bringing us into the conversation, even though the school district will tell you that we were, we really were not,” said Krokys of Independence Charter School.

A question of attitude

Since Ackerman took over in 2008, Krokys and other charter operators say, the district’s attitude towards charters has seemed almost hostile.

“They see us as competition and it’s not so much the individuals [in the charter school office] but the people way at the top,” said Krokys.

This perception wasn’t helped by a meeting on Oct. 19  with some charter operators, to which Deputy Superintendent Leroy Nunery was invited. The meeting was not scheduled as an SPI discussion, Nunery said, though many present wanted to bring that topic up.

“I didn’t want to have the conversation [at that moment,]” Nunery said.

From many charter operators’ perspective, the session that followed added fuel to the notion that the SPI process was not inclusive.

Hardy, who attended the meeting, said, “Lee has been trying to bully us.”

A letter from Philadelphia Charters for Excellence and the Philadelphia Coalition of Public Charter Schools referred to the meeting as “hastily convened” and held it as an example of a poor history of contact regarding the SPI.

Nunery thinks the charters are way off base on these points.

“I don’t think that what they said is true. I do not think I bullied anybody,” he said. “We weren’t dismissive of their concerns – that’s what I deny, we hadn’t even gotten to the numbers yet [at that meeting]. What we talked about was: Is there a way to create for the public or the parents a form of accountability?”

Charter operators saw that October meeting as their first chance to raise concerns about the index, which they thought at that time were to be released shortly to the public.  Given how little grasp they had at that time of how the rating system was structured, charter operators were in an uproar.

Weiner said the district’s intention has always been to work with the charters on the ratings. That’s why, he said, the district agreed to hold one-on-one meetings with charters – 68 of them – prior to Saturday’s release of the final SPI scores.

“We want to be sure that everyone in the charter school community is fully aware of what it means and what it measures,” he said.

But many charters say those meetings came only after they made a stink. And some charters have still felt compelled to hire outside consultants to evaluate the district’s numbers and how they got them.

Layers of confusion

According to one of those consultants, the SPI numbers literally don’t add up. Dr. Alex Schuh, a specialist in educational accountability, said he and some charter operators met with the district about the SPI. He described the session as unsatisfactory.

“The most important thing that they haven’t been able to share with any charter school that I know of, or me, is the formula that they use to calculate the school performance index,” he said.

Weiner insisted that the information has been on the district’s Web site for the last two years. But charter operators say the site is missing key information.

A question of value

On Wednesday, Mastery’s Scott Gordon said he was satisfied with the way the district got its data together for his schools, but he had a major problem with how the SPI values certain measures. It places too much weight on test score improvement, he says, which is unfair to schools that already perform well on tests. They’ll be hard-pressed to maintain as a high a rate of improvement as a low-performing school just getting its act together.

Other charter leaders object that the index doesn’t seem to take into account ways in which a charter may intentionally differ from traditional schools, in pursuit of academic innovation.

For example, Green Woods Charter School, which is a science-centered school at the Schuylhill Nature Center in Roxborough boasts excellent test results for science in fourth and eighth grades. But that success nets it no points on the SPI.

Charter advocates note that measures such as attendance, safety and parent participation, where charters often outperform traditional schools, are not part of the SPI.

Krokys believes one basic problem is that the district’s five-person charter school office is overwhelmed and can’t stay on top of all the details.

Jean Wallace, CEO of Green Woods, sees a deeper structural problem.

“When you get to decide what is important,” she said. “You can pretty much make the outcomes be whatever you want.”

This article is part of a year-long collaboration between NewsWorks and the Philadelphia Public School Notebook to report on Philadelphia school improvement efforts.