Penn students help robot grasp the art of reading

Students at a University of Pennsylvania lab have been working on a robot they call Graspy. This spring, they taught Graspy how to read.

The humans sharing an office with the robot at the General Robotics, Automation, Sensing and Perception (GRASP) lab at Penn said the training exercises got a little annoying.

The robot would repeatedly read phrases taped to a wall as students tinkered with the text-recognition program.

“My name is Graspy,” the mechanical voice said, over and over and over. “Graspy can read and talk. Graspy is awesome.” 

Technology allowing a computer to recognize text directly in front of it and read it aloud isn’t new. What is novel, said professor Kostas Daniilidis, is the software the students developed to enable Graspy to move around and find words with its attached camera.

“It’s really like the ‘I Spy’ books, where you say, ‘Find for me the banana’,” Daniilidis said. “The same thing is here for the text: Find me everywhere in a room where is text.”

The program enables the robot to look for strokes of uniform width and evenly spaced lines of text.

Graspy’s reading abilities are a work in progress. It has trouble with two-letter words and text that doesn’t stand out from the background it is printed on. 

Robotics master’s student Menglong Zhu, who wrote the program, said the program could eventually allow robots to use words on signs at train stations or in buildings to figure out where they are.

“The robot can use the words to do some analysis so it can understand a scene,” Zhu said.

Zhu said the software could also be developed into applications for the blind.