Affective Computing in Gaming Industry

From Machine Learning to Game User Interface

by Soroosh Mashal, Software Engineer and Gaming Expert at audEERING

Some weeks ago, we wrote a short general article about emotions in video games. Make sure to check it out here if you have missed it. This week, we want to get a bit more specific and see how it actually started and how it works.

In the beginning of this decade, scientific community proposed integration of affective computing with games. Focusing on serious games and specific emotions, they have already stablished the scientific proof for such applications.

World of Warcraft, Innkeeper Interaction (third item is added)

It is always a matter of time before industry catches up with the scientific community, and start integrating proposed technologies into their applications and products. audEERING as a leading company in affective computing is actively working on development and test of several use cases for different genres of video games.

So, let’s see what is going on under the hood. We take the voice input through microphone and break it into small equal pieces. Then, we look at the wave in each one of them. For example, if you are excited, you have higher arousal, and when you are sad and depressed your voice have really low arousal. We take different features into account and start to categorize the emotions based on it. But how?

Here comes the machine learning part. We take a lot of voices and literally listen to them and tell the computer how we felt. Then, the computer will understand these patterns the same way that we do. The rest is just adopting it to cultures, situations, and cases to increase accuracy.

Let’s look at two famous games to see how seamless these new features can be integrated into the games. The first example is from World of Warcraft by Blizzard Entertainment. Normally, you click on the merchant and you have only first two options. The merchant would already give you a discount if you have a high reputation with that faction.


World of Warcraft, full game interface

However, we are going to be even more natural. We just have added the third option here. Right after clicking on it, you would have 10 seconds to talk to the merchant, depending on how friendly your voice was, you will get a discount from the merchant.




Third item is added to give the use the possibility to give a speech


Let’s take another example into account. Imagine you are playing a strategy game like Total War: Warhammer and you are leading thousands of troops into battle. As seen on the screen below, your general now has a third icon on the bottom left corner which allows him to give a motivational speech to his soldiers before the fight. Just click on it and based on how heroic your voice was in the next 30 seconds, you will get a bonus attack for all of your troops.




Totalwar: Warhammer game interface


These examples clearly illustrate how easily any genre of video game can be enhanced and more humane simply by integrating such technologies. Science and technology have always increased the quality of our lives, and now is the time for our lovely video games to be more interactive. An interaction as deep as the emotions we express.