Your smartwatch might be able to tell you the weather, but can it answer Mae West’s old query: “Do you have a gun in your pocket, or are you just happy to see me?” If a new emotion-detecting wearable device developed by Massachusetts Institute of Technology students is an indication, then the wearables of the future will actually be able to tell you if people are happy (or sad) through the sound of their voice.
Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Institute of Medical Engineering and Science (IMES) say their new wearable artificial intelligence system can analyze speech patterns and infer emotional states. The system might be useful to help people with anxiety, Asperger’s syndrome, or anyone else who has difficulty understanding the emotional subtext of speech.
“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious,” Tuka Alhanai, a CSAIL graduate student who worked on the project. “Our work is a step in this direction, suggesting that we may not be that far away from a world where people can have an AI social coach right in their pocket.”
The system uses two AI algorithms and a Samsung Simband, a concept wearable. It measures vital signs such as skin temperature, heart rate, and blood pressure. Simultaneously it measures the speaker’s pitch, tone, and vocabulary. From this data, it estimates the speaker’s emotional state every five seconds. For example, if someone speaks slowly, with long pauses and monotonous vocal tones, it assumes that the speaker is telling a sad story. If the speech is quicker and more varied in tone, it assumes that the speaker is happy.
“As far as we know, this is the first experiment that collects both physical data and speech data in a passive but robust way, even while subjects are having natural, unstructured interactions,” said Mohammad Ghassemi, a CSAIL doctoral student. “Our results show that it’s possible to classify the emotional tone of conversations in real time.”
The researchers have been encouraged by the results so far. The device can classify the mood of a speaker 18 percent more accurately than chance, and 7.5 percent more accurately than other existing methods.
The team that developed the device isn’t yet ready to launch it into the real world. Before they do, they want to improve its reliability and accuracy. They may even used common commercial wearables such as the Apple Watch in order to collect more data.
“Our next step is to improve the algorithm’s emotional granularity so that it is more accurate at calling out boring, tense, and excited moments, rather than just labeling interactions as ‘positive’ or ‘negative,'” said Alhanai. “Developing technology that can take the pulse of human emotions has the potential to dramatically improve how we communicate with each other.”