audEERING and Hanson Robotics develop robots with the highest social competence for everyday use

The German market leader for AI-based audio analysis audEERING is developing humanoid robots with empathy capability together with Hanson Robotics, one of the world’s leading robot experts. Through audEERING’s voice analysis, the robots recognize emotions. This enables social interaction between humans and robots at the highest level for the first time. Emotion recognition is expected to enable and improve the use of robots in everyday life, especially in the health and care sector. 

Hanson Robotics is known worldwide for its lifelike robots. Appearance and social skills are modeled on humans. How far the development of human-like machines has come is demonstrated by the most famous robot and the first robot with official citizenship, Sophia. She has already spoken to the then-German Chancellor Angela Merkel in 2018 and the United Nations, demonstrating how humans and machines can interact.  

The cooperation between Hanson Robotics and audEERING also seeks to further develop Sophia Hanson’s social skills. In the future, Sophia will recognize the emotions of the conversation and be able to respond empathically as a result. In nursing and other fields affected by the shortage of skilled workers, such robots equipped with social AI can help in the future. Through supporting AI diagnostics, automated speech analysis improves preventive care and therapy for patients and replaces costly blood tests, among other things. For example, audEERING was recently able to show in a 1-year study with MS patients that disease-related indicators of multiple sclerosis, fatigue and depression can be detected through short, regularly performed speech tests. In doing so, audEERING’s AI technology, which was developed according to ethical principles, analyzes the content of what is being said based solely on acoustics and phonetics which allows best possible data protection.  

Empathy through emotion recognition

To enable robots to interact with humans, Hanson Robotics already analyzes human facial expressions (nonverbal), eye contact, and speech (verbal), and works with an emotional facial expression of the human-like robots. With audEERING’s patented voice and speech analysis technology devAIce, the robots now also recognize the emotions of the speaker and can put themselves in the other person’s emotional state. For example, if a robot analyzes anger in the human counterpart via the voice, it could react calmly and soothingly, and if it detects joy, it could react particularly euphorically. The technology from the German voice AI expert thus closes the remaining gap in paralinguistic information (voice pitch, tone of voice) and allows social human-machine interaction of a kind that was previously impossible.

audEERING’s privacy-compliant voice and speech analysis, which is already being used internationally by GN/Jabra and other leading companies, not only enhances robots’ perceptive abilities. In combination with existing AI models such as text-based speech recognition (ASR) or visual analysis, robots can also perceive irony or contradictions in human expression, for example.  

Dr. David Hanson, founder and CEO of Hanson Robotics says:

“Emotions are the basis for all interpersonal interactions. With emotion recognition, we are a big step closer to our goal of a natural interaction between humans and robots. For us, it was important that the data analysis is carried out according to the highest ethical standards and that the AI contributes to the robots working for the well-being of society. audEERING’s technology convinced us in this regard.” 

Dagmar Schuller, AI expert and managing director of audEERING

Dagmar is also positive about the collaboration:

“Empathy is the most important success factor of our time. Especially in the area of increasing automation and optimization of processes, empathy, and better understanding is more important than ever for our well-being. With our technology in connection with social robotics, we especially want to improve this empathy ability in human-centered AI and create added value for all humans.” 

The most natural form of interpersonal communication is analog conversation. In it, interlocutors exchange information – verbally and nonverbally (facial expressions, body language) and paralinguistically (voice, prosody). Until now, it has not been possible to replicate all these levels in a conversation with a robot, so no full-fledged social interaction between humans and machines could take place. The patented technology of the German AI pioneer audEERING will change that with emotion recognition.

Here you can download the general press release on the new collaboration between Hanson Robotics and audEERING as a PDF (German version).