‘High-quality’ pre-K: The phrase everyone loves, but few know how to gauge

Nearly everyone agrees that high-quality pre-K can make a big difference for kids. But how should states measure quality?

That question is at the heart of a new research brief from Research for Action, the Philadelphia-based education research group. The answer will be critical to cities like Philadelphia as they try to expand pre-K coverage that actually delivers lasting benefits.

The problem is there’s no straightforward answer.

Shooting for the STARS

The RFA brief looks specifically at Quality Ratings and Improvement Systems. It’s a clunky term, but it refers to the rubrics that states use to rate early childhood programs. Pennsylvania’s is called Keystone STARS (for “standards, training, assistance, resources and support”), and it’s one of the oldest in the country.

There has been scads of QRIS research. Most of it, however, looks at whether a given QRIS has been implemented faithfully. The research is comparatively thin on whether ratings systems actually do a good job measuring quality.

In other words, a QRIS might rate one center a two and another center a four, but it’s not clear there’s a big difference between level-two centers and level-four ones in terms of student achievement.

“They have been studied. It’s just a matter of figuring out how to best study the relationship between QRIS scores and student outcomes,” says Katie Englander, who co-authored the RFA brief. “And so it’s a mixed bag so far.”

One study on the California QRIS suggested the system did a good job measuring criteria that have been linked with positive child outcomes. Another examined Colorado’s QRIS and found that students at highly rated centers in the state fared no better than those from their low-rated counterparts.

A study from the University of Pennsylvania’s Child Research Center gave mixed reviews to Keystone STARS, which works on a four-star rating scale. It found that children who attended a three- or four-star center did measurably better on assessments than children who went to one- or two-star facility. But it also found little difference between one- and two-star centers, and little to distinguish three-star centers from four-star ones.

That’s in part why Philadelphia’s new pre-K expansion is geared toward funding centers with either a three- or four-star rating. The city also has a provisional program for centers who are attempting to earn three-star status.

By doing so, city officials have essentially anchored their expansion efforts to this notion that Keystone STARS can indeed discern quality. And they’ve done so at a time when STARS itself is evolving.

Time for a makeover

Earlier this year, the state announced it would rehaul Keystone STARS in order to make it more meaningful and easier to navigate. Right now, just 48 percent of childcare facilities statewide have a STARS rating, with many opting to avoid the system altogether. A preliminary report that will guide the rehaul is due out later this month. The state plans to pilot a new STARS system early next year, according to a timeline released by the Office of Child Development and Early Learning.

The Research for Action brief identifies a number of areas where STARS could improve. Many of them revolve around how to best measure employee qualifications and support teacher improvement.

A center’s STARS rating relies heavily, for instance, on the degree levels of its employees. A facility could drop a level if an employee with a bachelor’s degree takes another job — or climb to new heights if a well-educated teacher jumps on board.

The problem is the early-childhood field suffers from high teacher turnover. RFA suggests the state give centers more leeway so that small staffing changes don’t lead to ratings swings for centers. Massachusetts has a more flexible ratings system in this one regard.

RFA also suggests Pennsylvania move away from using degree attainment as the primary measure for judging teacher quality.

“The research shows that, while teacher education level is really important and matters for child outcomes, there are other dimensions of teacher quality that are also important,” says Della Jenkins, a co-author.  “So one of the things that other states have done is look at ways to measure teacher quality beyond a bachelor’s degree or a credential in early childhood education.”

That includes taking into account professional development, workshops attended, and other indicators that are admittedly more difficult to measure. Delaware is among the states pioneering this approach.

Then there’s the trick of measuring all that happens in a pre-K classroom without being overly prescriptive or intolerably vague. At present, Keystone STARS uses a tool called ECERS-R (the “Early Childhood Environment Rating Scale, Revised”) to evaluate almost everything about a pre-K facility — from the academic lessons to the physical space. The RFA brief finds promise in another rating system called CLASS (“Classroom Assessment Scoring System”) that examines teacher-student interactions with more detail.

All this may seem wonky, and it is. At the end of the day, RFA researchers admit, the best quality rating system in the world won’t matter if pre-K centers don’t have the funding they need to pay teachers a competitive wage or if decent centers aren’t accessible to students of every socioeconomic stripe.

Still, there’s little doubt — at least in the research community — that if pre-K is to meet its potential, states will need reliable ways of measuring quality. Otherwise, cities like Philadelphia will have thousands of new “high-quality” seats on paper, and only on paper.