Playing video games when you can’t see the screen

Think about what it’s like to play a racing game, like Need for Speed.

You hurtle through busy streets, sometimes at 100 miles per hour. You see a shortcut through a fence, so you drive right through it. You dodge police cars, quickly reacting to what’s on the screen, all without slamming into walls.

Now imagine doing all that without seeing the screen. Software engineer Karen Stevens says blind gamers explained to her how they do it:

“Different pavements have different types of vibrations so you can tell what kind of ground you’re on. There’s feedback for when you hit other cars, or cars hit you, so you feel the controller vibrate,” Stevens explains. “They also have whooshing sounds when you go past obstacles, so they … can keep track of where they are based on that.”

Stevens says there were other aural cues, too:  “You could hear the other cars turning, so they knew they had to turn too. Between all that, they kind of memorized the tracks,” she said.

But this is not the way the game was designed.

“But it’s totally accidental. We did not plan it that way, but it does work for them.”

Stevens is an accessibility lead at EA Sports, which makes games like Madden NFL, NBA Live, and the soccer game FIFA. Her job is to push developers to make their games more playable for more people.

“Inclusion is important for everyone,” said Stevens, who is nearly deaf. “I’m disabled myself and I really dislike being excluded from things.”

Stevens wanted to show her colleagues that blind gamers could play – and were playing – mainstream games like the ones her company makes. So, she invited a blind player to come to the office with her. He challenged her co-workers to the fighting game Mortal Kombat

“It’s really hard to say a blind gamer doesn’t exist when he beats you at a game,” Stevens said.

Stevens wants to make sure there are features for blind players by design, not just by accident. For example, she made vibration cues for the visual prompts in the game Madden NFL, so you don’t have to see the screen to play.

Stevens works in the huge, commercial game world, but there is also a smaller market of accessible games — like those developed specifically for blind people.

Software engineer Austin Seraphin tried them when he was younger. He has been blind since birth. He says instead of a video game, a racing game designed for the blind is basically an audio game. A voice tells you to steer left and right.

He says it’s like having a backseat driver all the time. And, you can imagine that’s less fun than the freedom of navigating a game on your own.

Seraphin says many blind people enjoyed video games before losing their sight — and wish they could recapture some of the the memories of driving, freedom and exploring from before the time they were blind.

That sense of freedom you can get in a video game matters to Brian Smith, who’s from New Mexico.

“Games have meant a lot to me growing up in middle of the desert. I really like the way that video games could take me to places that I just couldn’t go before growing up and give me experiences that I couldn’t otherwise have,” Smith said.

As a kid, one of his favorite games was Super Mario Brothers. He really liked finding secrets and shortcuts in that game.

“And then, everyday at school, you’d go and tell your friends … ‘hey, did you realize there was a coin block or a vine here?’ And word of mouth would spread,” Smith said.

Smith is now a computer scientist doing his Ph.D. at Columbia University. He decided to make a user interface so that blind gamers could have more freedom to explore worlds and drive cars, without a voice telling them what to do.

He wanted the world of accessible games to be better. He is sighted, so he talked to people who are blind to understand their experience.

In a racing game, you have to know where your car is, and you have to know when and how to turn. So he made an interface where the engine sound is a proxy to help blind gamers place their car in space.

“Imagine that you are sitting directly behind the car that you’re controlling, so that you hear the sound of the car’s engine right in front of your face,” Smith explained. “And then as you steer the car left and right, and this is with a standard video game controller, you’re actually controlling that engine sound directly.”

You don’t need fancy headphones for this – standard headphones offer stereo sound, offering individual channels of sound in each speaker. So your car turns left, the engine sound in your left ear gets louder. The sound slides as the direction of your car moves.

Smith asked blind volunteers to test the game.

“It was a really cool thing the first time we saw one of our participants who are blind get a lap time that was better than one of our sighted participants,” Smith said.

That winner was Edis Adilovic, a visiting graduate student from Denmark who lost his sight when he was 2 years old.

These days, Adilovic mostly sticks to audio games made for the blind, because he says none of the mainstream games he has played are fully accessible.

For example, he tried playing the game Grand Theft Auto 2 (GTA 2.) In the game, you can assassinate gangsters, find hidden packages, and yes, steal and drive cars.

“I didn’t know basically what I was doing; I usually had to have some sighted assistance,” Adilovic said. “One of the difficulties … in GTA for instance was actually to drive the cars, so I see a big potential in implementing this system that Brian developed.”