Transparent AI Part 2: Modeling Emotions

©2021 audEERING GmbH       Soroosh Mashal         03.02.2021

In our first episode of this series, we covered the definitions of feeling, emotion, mood, and their role in our lives. In this episode, we lay the foundation for the modeling of emotions in Affective Computing.

There are a lot of theories and models out there trying to explain feelings and emotions. We would need a whole series to talk about each one of them. They try to explain emotions from different perspectives, namely: neurobiological, cognitive, psychological and so on, but let us take an affective approach here to be more effective.

Discrete vs. Continuous Modeling

The first step in modeling is deciding on whether we should model it discretely or continuously. Discrete modeling is easy to understand. For example binary modeling is the easiest: “happy” vs. “not happy”, “sad vs. “not sad”. But we all know that it not the reality, right? You can be joyful, happy, excited. It hints at the existence of a spectrum here. So, let us try using continuous modeling.

Dimensions of Emotions

Now, we need to decide on the dimension or dimensions. A dimension will be something that can differentiate the emotions better than anything else. What would be the best candidate for the first dimension? It is relatively intuitive: How emotion is perceived. This is usually referred to as pleasantness or valence. It means whether that is a good or bad emotion.

So, our X dimension is going to be valence. For example, sadness is negative, and happiness is positive. Depressed is more negative than sad, and excited is more positive than happy.

Now, let us find a way to differentiate between sad and angry, or happy and excited. We need a second dimension. How do they differ when you consider their expression? So, when I am excited, I am more aroused and I am loud, but when I am happy, I am less aroused. It is the same with anger and sadness. When I am angry, I am yelling (there is also cold anger which we address separately) and having negative emotions, and when I am sad, I am weeping almost silently and having negative emotions. So, it seems that we have found a good candidate for our second dimension: arousal. The more aroused I am, the more intense the emotion is. Now we have a two-dimensional space.

Dominance as Third Dimension

This two-dimensional space is already great for distinguishing a lot of emotions, but let us go one level further. Here is an example: When you are afraid and you panic, you are experiencing a negative emotion, and you are also yelling, which means you also have high arousal. Similarly, when you are angry, you also experience a negative emotion with high arousal. However, they are not similar at all. That is why our two dimensions are not enough. So, what is the difference between panic and anger. Well, it is the control over the situation. It is your dominance. So, the third dimension will be control or dominance.

The Three-dimensional Emotion Space

Finally, we have a 3D space that we can use to model emotions. Now that you know this representation, try the following experiment. Close your eyes and listen to someone’s voice, you can see that using these dimensions, you can identify almost any emotion and find a space for it. Now that we have our emotions represented mathematically, let us try to teach computers to see the world like this.

In our next episode, we will dive deeper into the technical side and see how Machine Learning works. Meanwhile, feel free to check our products for the industry using this science under the hood.

Read more about audEERING