devAIce®
The Voice Translator

The VAD module detects the presence of voice in an audio stream. The detection is strongly robust to noise and independent of the volume level of the voice. This means that also faint voices can be detected in the presence of louder background noises.

Detecting voice in large amounts of audio material leads to a resource-saving and efficient analysis process. If VAD runs before the voice analysis itself, large amounts of non-voice data can be filtered and excluded from the analysis.

The Expression module analyses the expression of emotions in the voice. The module was developed to analyze expression in all languages. Currently, the module combines two independent expression models with different output representations:

• A dimensional arousal-valence-dominance expression model
• A four-class categorical expression model: happy, angry, sad, and neutral

In devAIce® we offer three Expression modules: 

Large
devAIce® provides the Expression Large module for scenarios where maximum prediction performance is required. It delivers the highest accuracy and is ideal for applications with sufficient computational resources and memory capacity.

Small
The Expression module in devAIce® offers a balanced approach between performance and resource consumption. It is well suited for applications running in environments with moderate computational and memory availability.

Tiny
For deployments with strict hardware limitations or low-memory devices, the lightweight Expression Tiny module is the optimal choice. It enables efficient on-device processing while minimizing resource usage.

The Acoustic Scene Module distinguishes between 3 classes:

• Indoor (small, medium, large)
• Outdoor (traffic, no traffic)
• Transport (bus, car, railroad)

The AED module runs acoustic event detection for multiple acoustic event categories on an audio stream.

Currently, speech and music are supported acoustic event categories. The model allows events of different categories to overlap temporally.

The Speaker Attributes module estimates the personal attributes of speakers from voice and speech. These attributes include:

• A perceived gender (sex), divided into the categories:
  • female adult
  • male adult
  • child
• A perceived age, in years

devAIce® provides two gender modules: Large and Small.

The age-gender models are trained on self-reported gender.

You can achieve more robust speaker verification by reliably verifying a single speaker, pushing the boundaries of what was previously possible. 

• Enrollment mode: a speaker model for a reference speaker is created or updated based on one or more reference recordings.
• Verification mode: a previously created speaker model is used to estimate how likely the same speaker is present in a given recording.
• Embedding mode: vocal features that can be used for tasks such as speaker diarization.

The Prosody module computes the following prosodic features:

• F0 (in Hz)
• Loudness
• Speaking rate (in syllables per second)
• Intonation

The openSMILE Features module performs feature extraction on speech. Currently, it includes the following two feature sets based on openSMILE:

• ComParE-2016

This feature set consists of a total of 6373 audio features that are constructed by extracting energy, voicing, and spectral low-level descriptors and computing statistical functionals on them such as percentiles, moments, peaks, and temporal features.

While originally designed for the task of speaker emotion recognition, it has been shown to also work well for a wide range of other audio classification tasks.

• GeMAPS+

The GeMAPS+ feature set is a proprietary extension of the GeMAPS feature set described in Eyben et al.2. The set consists of a total of 276 audio features and has been designed as a minimalistic, general-purpose set for common analysis and classification tasks on voice.

Leveraging the powerful whisper.cpp library, this module opens a world of possibilities by allowing developers to transcribe spoken language into written text seamlessly. The additional integration of this module means that resources no longer need to be used to integrate external ASR engines separately and combine them with audio analysis, as devAIce® offers an all-in-one solution.

The Audio Quality module estimates the Signal to Noise Ratio (SNR) and the reverberation time (RT60) of a given audio input. Input signals of any duration are accepted. Every audio input yields one value for each parameter.