How AI and machine learning led to ‘mind blowing’ progress in understanding animal communication

This story is from The Pulse, a weekly health and science podcast.

Find it on Apple Podcasts, Spotify, or wherever you get your podcasts.

Behavioral ecologist Jaclyn Aubin has studied beluga whales in Quebec’s  St. Lawrence River Estuary for years. Belugas are plump white whales that are bigger than dolphins and smaller than killer whales. They communicate using whistles, calls and clicks. 

“Belugas are a very vocal species,” Aubin said. “They’re often called the sea canaries because of how loquacious and vocal they are.”

She studies their calls, and has collected thousands and thousands of hours of recordings. In the past, she would go through these calls in her lab.  Her eyes would get tired from staring at a screen after two hours, so she had to take breaks. 

Then she got in touch with a computer scientist, who helped her develop a machine learning model so a computer could go through the recordings and flag parts that could be interesting. Aubin said that “really ramped up the process and allowed me to process so many more hours of recordings, really like a mind blowing amount of progress.” 

She has now been able to identify some specific calls, like the call that a beluga mother will use to get a curious calf to return to her side. 

For decades, people have dreamed of being able to understand what animals are saying. Artificial intelligence and machine learning have led to dramatic progress in recent years for researchers studying belugas and sperm whales. Some scientists are excited for future progress, whereas others caution there may be obstacles to understanding animals that humans will never overcome. 

Whale biologist Shane Gero has also benefited from machine learning in his research. Sperm whales are the largest toothed whales, can dive deeper than nuclear submarines, and can live for as long as 70 to 90 years. 

He has studied a group of sperm whale families in the Caribbean for more than 20 years, and has gotten to know them so well, his research team has named most of them. 

“We don’t do this flippantly — it makes an important point about who they are. 

Using names is sort of a shortcut, a reminder that actually each individual is different,” he said. “These are large, big brain, long lived animals like us.” 

Gero is part of a research group called Project CETI. He said scientists have known for a long time that sperm whales communicate with each other in sophisticated ways, but machine learning helped study their calls in a new way.

For instance, they now understand how sperm whales vary the rhythm, tempo, and duration of their clicks, in the same way that humans combine words in different orders with different tones to change the meaning of what they say to each other. 

That led to the discovery that sperm whale clicks work like an alphabet, where elements can be combined differently to mean different things; compared to hieroglyphics, where each symbol represents a certain meaning, said computer scientist Pratyusha Sharma, who also works on Project CETI with Gero.  

She said this kind of system, which humans, primates, and some birds use — is rare in nature. 

“There needs to be … the right kind of communicative, environmental, and social pressures to even give rise to the system.” 

This kind of research excites animal behaviorist Con Slobodchikoff. He said this reminds him of another recent experiment, where scientists played back the sound of humpback whale calls to a whale, and the whale responded. He said he can see a time in the future when we would be able to do human-whale translation. 

“And as we get better at it, and as our dictionary of animal signals gets better, we might be able to have much more sophisticated kinds of playbacks,” he said. “I think that we’re getting in the realm of possibility.” 

Slobodchikoff has dedicated much of his research to studying prairie dog calls. He is now the chief scientist and founder of Zoolingua, a company working on a human-dog translation system. He said he cannot discuss details for now, but guesses that it is around five years away.

However, neuroecologist Yossi Yovel said machine learning and the breakthroughs it has led to won’t be able to break all the language barriers between humans and other animals.

He studies communication in Egyptian fruit bats at Tel Aviv University in Israel.

As an example, he said human scientists can study animal behaviors in contexts we can see and understand: fighting, eating, sleeping, mating and so on. But what if the animals are behaving in ways that us humans cannot even imagine, because they are so different from what we can think of? 

For instance, sperm whales and belugas navigate the world using clicks. Most humans cannot picture what it would look like to navigate the world entirely using sound rather than sight.

“These are barriers that will not be crossed,” Yovel said. “I think that it’s still very, very interesting to see how far we get. I don’t think we’ve reached the dead end.”

“I do think machine learning will allow us to take things further. But I think these obstacles are fundamental.” 

But, whale biologist Shane Gero said the point of this research is not to talk to whales. 

“Everyone is, at least from the public facing side, very excited about this idea of talking to the whales. But that really does not drive the work that I do or that CETI does.”

He said the point is to understand what is important in a whale’s world, so that humans can change our behavior accordingly. He refers to a quote from writer Cormac McCarthy, who was a friend of famous humpback whale researcher Roger Payne. 

“He has this great line in one of his books, which is: what if God comes back and asks you if you figured it all out? And then before you can answer, said, ‘hey, did you ever consider asking the whales?’ And then looked around and said, ‘where have all the whales gone?’”