Two weeks ago, we wrote a short general article about emotions in video games. Make sure to check it out here, if you have missed it. This week, we want to get a bit more specific and see how it actually started and how it works.

In the beginning of this decade, scientific community proposed integration of affective computing with games. Focusing on serious games and specific emotions, they have already stablished the scientific proof for such applications.

The relationship between industry and the scientific community

It is always a matter of time before industry catches up with the scientific community, and start integrating proposed technologies into their applications and products. audEERING as a leading company in affective computing is actively working on development and test of several use cases for different genres of video games.

So, let’s see what is going on under the hood. We take the voice input through microphone and break it into small equal pieces. Then, we look at the wave in each one of them. For example, if you are excited, you have higher arousal, and when you are sad and depressed your voice have really low arousal. We take different features into account and start to categorize the emotions based on it. But how?

Machine learning simply explained

Here comes the machine learning part. We take a lot of voices and literally listen to them and tell the computer how we felt. Then, the computer will understand these patterns the same way that we do. The rest is just adopting it to cultures, situations, and cases to increase accuracy.

Integrating emotion recognition into the game

Let’s look at two famous games to see how seamless these new features can be integrated into the games. The first example is from World of Warcraft by Blizzard Entertainment. Normally, you click on the merchant and you have only first two options. The merchant would already give you a discount if you have a high reputation with that faction.

However, we are going to be even more natural. We just have added the third option here. Right after clicking on it, you would have 10 seconds to talk to the merchant, depending on how friendly your voice was, you will get a discount from the merchant.

Let’s take another example into account. Imagine you are playing a strategy game like Total War: Warhammer and you are leading thousands of troops into battle. As seen on the screen below, your general now has a third icon on the bottom left corner which allows him to give a motivational speech to his soldiers before the fight. Just click on it and based on how heroic your voice was in the next 30 seconds, you will get a bonus attack for all of your troops