Playing video games when you can’t see the screen

A computer scientist is pushing large video game makers to design games that blind people can enjoy.

Listen 6:32
Brian Smith plays a racing game blindfolded. He made a user interface for blind players to drive in video games, just like sighted players. He tested it with blind volunteers, as well as blindfolded sighted volunteers.

Brian Smith plays a racing game blindfolded. He made a user interface for blind players to drive in video games, just like sighted players. He tested it with blind volunteers, as well as blindfolded sighted volunteers. (Alan Yu/ WHYY)

Think about what it’s like to play a racing game, like Need for Speed.

You hurtle through busy streets, sometimes at 100 miles per hour. You see a shortcut through a fence, so you drive right through it. You dodge police cars, quickly reacting to what’s on the screen, all without slamming into walls.

Now imagine doing all that without seeing the screen. Software engineer Karen Stevens says blind gamers explained to her how they do it:

“Different pavements have different types of vibrations so you can tell what kind of ground you’re on. There’s feedback for when you hit other cars, or cars hit you, so you feel the controller vibrate,” Stevens explains. “They also have whooshing sounds when you go past obstacles, so they … can keep track of where they are based on that.”

Stevens says there were other aural cues, too:  “You could hear the other cars turning, so they knew they had to turn too. Between all that, they kind of memorized the tracks,” she said.

But this is not the way the game was designed.

“But it’s totally accidental. We did not plan it that way, but it does work for them.”

It's hard to play a racing game like Need For Speed without seeing the screen. For example, what do you when you have two police cars chasing you and you have to figure out where they are, and how to avoid them? But some blind gamers have figured out ways around it. Courtesy of Electronic Arts Inc
It’s hard to play a racing game like Need For Speed without seeing the screen, but some blind gamers have figured out ways around it. (Courtesy of Electronic Arts Inc.)

Stevens is an accessibility lead at EA Sports, which makes games like Madden NFL, NBA Live, and the soccer game FIFA. Her job is to push developers to make their games more playable for more people.

“Inclusion is important for everyone,” said Stevens, who is nearly deaf. “I’m disabled myself and I really dislike being excluded from things.”

Stevens wanted to show her colleagues that blind gamers could play – and were playing – mainstream games like the ones her company makes. So, she invited a blind player to come to the office with her. He challenged her co-workers to the fighting game Mortal Kombat

“It’s really hard to say a blind gamer doesn’t exist when he beats you at a game,” Stevens said.

Stevens wants to make sure there are features for blind players by design, not just by accident. For example, she made vibration cues for the visual prompts in the game Madden NFL, so you don’t have to see the screen to play.

When you have to kick the ball in the football game Madden NFL, you have to aim your kick. That's hard to do if you can't see the screen, so programmers at EA added several vibration cues to the game controller. Courtesy of Electronic Arts Inc
When you kick the ball in the football game Madden NFL, you have to aim your kick. That’s hard to do if you can’t see the screen, so programmers at EA added several vibration cues to the game controller. (Courtesy of Electronic Arts Inc.)

Stevens works in the huge, commercial game world, but there is also a smaller market of accessible games — like those developed specifically for blind people.

Software engineer Austin Seraphin tried them when he was younger. He has been blind since birth. He says instead of a video game, a racing game designed for the blind is basically an audio game. A voice tells you to steer left and right.

He says it’s like having a backseat driver all the time. And, you can imagine that’s less fun than the freedom of navigating a game on your own.

Seraphin says many blind people enjoyed video games before losing their sight — and wish they could recapture some of the the memories of driving, freedom and exploring from before the time they were blind.

That sense of freedom you can get in a video game matters to Brian Smith, who’s from New Mexico.

“Games have meant a lot to me growing up in middle of the desert. I really like the way that video games could take me to places that I just couldn’t go before growing up and give me experiences that I couldn’t otherwise have,” Smith said.

As a kid, one of his favorite games was Super Mario Brothers. He really liked finding secrets and shortcuts in that game.

“And then, everyday at school, you’d go and tell your friends … ‘hey, did you realize there was a coin block or a vine here?’ And word of mouth would spread,” Smith said.

Smith is now a computer scientist doing his Ph.D. at Columbia University. He decided to make a user interface so that blind gamers could have more freedom to explore worlds and drive cars, without a voice telling them what to do.

He wanted the world of accessible games to be better. He is sighted, so he talked to people who are blind to understand their experience.

Brian Smith ties a strip of light absorbing blackout cloth around his eyes as a blindfold. He made a user interface so blind players can play racing games, and tested it with blind volunteers, and blindfolded sighted volunteers.
Brian Smith ties a strip of light absorbing blackout cloth around his eyes as a blindfold. He made a user interface so blind players can play racing games, and tested it with blind volunteers and blindfolded sighted volunteers. (Alan Yu/WHYY)

In a racing game, you have to know where your car is, and you have to know when and how to turn. So he made an interface where the engine sound is a proxy to help blind gamers place their car in space.

“Imagine that you are sitting directly behind the car that you’re controlling, so that you hear the sound of the car’s engine right in front of your face,” Smith explained. “And then as you steer the car left and right, and this is with a standard video game controller, you’re actually controlling that engine sound directly.”

You don’t need fancy headphones for this – standard headphones offer stereo sound, offering individual channels of sound in each speaker. So your car turns left, the engine sound in your left ear gets louder. The sound slides as the direction of your car moves.

Brian Smith puts on a pair of headphones to play his racing game. He made a user interface so blind players can drive in video games just like sighted players.
Brian Smith puts on a pair of headphones to play his racing game. He made a user interface so blind players can drive in video games just like sighted players. (Alan Yu/WHYY)

Smith asked blind volunteers to test the game.

“It was a really cool thing the first time we saw one of our participants who are blind get a lap time that was better than one of our sighted participants,” Smith said.

That winner was Edis Adilovic, a visiting graduate student from Denmark who lost his sight when he was 2 years old.

These days, Adilovic mostly sticks to audio games made for the blind, because he says none of the mainstream games he has played are fully accessible.

For example, he tried playing the game Grand Theft Auto 2 (GTA 2.) In the game, you can assassinate gangsters, find hidden packages, and yes, steal and drive cars.

“I didn’t know basically what I was doing; I usually had to have some sighted assistance,” Adilovic said. “One of the difficulties … in GTA for instance was actually to drive the cars, so I see a big potential in implementing this system that Brian developed.”

WHYY is your source for fact-based, in-depth journalism and information. As a nonprofit organization, we rely on financial support from readers like you. Please give today.

Want a digest of WHYY’s programs, events & stories? Sign up for our weekly newsletter.

Together we can reach 100% of WHYY’s fiscal year goal