New technology could help cities inspect and maintain streets more efficiently

Street inspections are time-consuming and, ultimately, subjective. What if a computer could do that work instead of a human inspector? 

Christoph Mertz spends his days looking at cracks in the street.

“Once you’re involved in something like this, you see every crack in the road, every pothole, you say, ‘ohhh, this is interesting,’” he said as he wove around sizeable potholes on the narrow streets behind Carnegie Mellon University, in Pittsburgh. A small camera mounted on his windshield much like a GPS device shot video of the pavement unspooling in front of him as he drove. He said he relished finding really deteriorated streets because “it’s a really good example for my data.”   

Mertz is a researcher at Carnegie Mellon University. He works with CMU’s Traffic 21, a research arm of the university focusing on practical solutions to transportation problems, and he’s built a computer program that uses photos to detect cracks in street pavement. He’s been testing it out using photos extracted from the video he shoots in his own car, but he’s also working with the city to test the technology. He’s placed cameras inside three city vehicles. 

For Pittsburgh, which currently uses manual, visual inspections to assess city streets, the technology could one day be a more reliable and cheaper alternative tool for maintaining roads. 

Human vs. computer inspections

Pittsburgh maintains 1,100 miles of streets, divided into 17,000 segments. Three inspectors are responsible for visiting each of those segments to evaluate them. The inspectors use their eyes to assess the streets, to note any cracks, potholes, or rutting, the severity of the deterioration, and its proliferation in the segment. Though the inspectors are trained, Deputy Director of Public Works, Lee Haller, said inspections can be “fairly subjective.”

Also, the inspectors have other responsibilities; it takes them four to five years to review all of the segments.

The city uses a pavement management system and models to predict a street’s lifespan but Haller said because of the subjectivity and long inspection periods, there’s a good chance much of the base data is inaccurate. “After winter the condition of a given street can change rapidly because what was just cracking after a number of freeze-thaw cycles turns into major potholes,” he said. 

Pittsburgh’s street inspection system is standard for cities.

Mertz’s program, he claims, automates the process, speeds it up, and eliminates human bias. It uses photos from the cameras installed in city vehicles — vehicles that would be making rounds in the city anyway, not requiring dedicated city employees’ time. The program breaks the pavement in each picture into a grid and then the photos are combed digitally, using an algorithm, looking for texture. A newly paved road looks flat to the machine, but a road with cracks in it will have texture. Then it calculates how much of the given surface is covered by cracks and scores the road’s condition.       

The program can ‘inspect’ streets as quickly as the photos from the city cars come in. Mertz said running the algorithms on large data sets can take a while, but for him that meant several days, a relative wink-of-an-eye compared to the city’s years-long inspection cycle. 

For Haller, having access to more accurate, up-to-date street data would make planning easier and more effective. He’d know exactly which street segments need work to get all streets in the city up to a certain quality threshold, for example. He’d be able to ask for money and promise specific results, something he can’t always do now. 

“I think when you have the conversation with Council and say these are the extra streets specifically by name that we would be able to do in this upcoming season if we had extra money, it’s more tangible,” he said.   

A better, more predictably maintained road network means a more viable city.

“That is not only the public getting to and from work, which is very important, but it’s also bringing goods and services…freight and the like,” said American Society of Civil Engineers fellow, Brian Leshko. “It really does have a big impact on the economy of a city.” 

Working out the kinks

Back at the lab, Mertz is still working out kinks in the program. He pulls up a map of Pittsburgh filled with green, yellow and red dots, each representing an analyzed image of a street segment. Green means the road is good, yellow means it has some damage, and red means it’s bad. As he clicks on the dots images of the pavement pop up.  

He clicks on a red dot, which means the road is bad, but the photo shows smooth pavement, broken only by the shadows of overhanging tree branches. It turns out the algorithm still can’t tell cracks and branch shadows apart. Mertz says the machine learning algorithms are still…learning.

There are some other issues, too. For now, files have to be transferred and handled manually, which Haller says doesn’t work for the city. Mertz also hasn’t trained his program to see potholes, patches, rutting, or to analyze non-pavement road surfaces (Pittsburgh has many brick and stone streets). He said he hopes to get to all that.

When he does, the technology could be a game changer for cities like Pittsburgh with aging infrastructure and limited resources.