Skip to main content
Skip table of contents

What is SG Com?

The SG Com SDK provides access to Speech Graphics' real-time audio-driven facial animation technology, enabling developers to build applications with avatars driven by voice.

SG Com's processing Engine converts an incoming audio stream into synchronous, high-fidelity facial animation which may be streamed to other endpoints. Output animation includes lip sync, full-face emotional expressions, blinks, eye darts, and even head movement, all driven by the speaker's voice through proprietary Speech Graphics algorithms. The processing delay between input and output is 50 milliseconds.

SG Com also includes a Player that handles buffering, decoding and synchronized playback of the animation produced by the Engine.

The SDK includes:

C# bindings and sample Unity integrations are also available on request.

What’s new in SG Com 5?

CPU cost savings

For bone rigs, the CPU cost of SG_COM_UpdateAnimation has been reduced by two thirds (per bone). See SG Com Compute Resource Usage.

Licensing

There are now two licensing options:

  1. key-based licensing

    1. Key string is input into license_data argument of SG_COM_Initialize

  2. cloud-based licensing

    1. Path to .lic file is input into license_data argument of SG_COM_Initialize

    2. Available on Windows and Linux only

    3. Requires internet connection with two open ports

Expression Changed status type

SG Com emits a new status type for the SG_COM_EngineStatusCallback: SG_COM_STATUS_EXPRESSION_CHANGED. This is triggered whenever SG Com starts a new facial expression. The payload includes the full path of the expression. This can be used to synchronize other animations with the expressions.

Auto Mode Controls

We now use the enum SG_COM_AutoMode for automatically detected states such as emotions or other vocal events.

SG Com 4 relied on specific naming of the character's behavior modes to trigger them automatically, such as "positive", "negative", "acknowledge". Now auto modes can be mapped to any behavior mode at runtime, so there is no longer a naming requirement.

For example:

SG_COM_SetAutoMode(engine_handle, SG_COM_POSITIVE_MODE, "happy");

maps the positive auto mode to the character’s behavior mode called “happy”, whereas

SG_COM_SetAutoMode(engine_handle, SG_COM_NEGATIVE_MODE, "angry");

maps the negative auto mode to the character’s behavior mode called “angry”. Note that an auto mode must be mapped to something in order for it to have an effect on behavior.

To clear the current auto-mode mappings, use

SG_COM_UnsetAutoModes();

The default behavior mode, which is used when no auto mode is detected, can also be changed:

SG_COM_SetDefaultMode(engine_handle, "friendly");

Note that in order to activate automatic mode detection, we now use

SG_COM_ActivateAutoModes(engine_handle);

Whereas to turn off automatic mode detection and instead use a fixed mode, e.g. "happy", use

SG_COM_SetMode(engine_handle, "happy");

Effort Detection

There is a new auto mode, SG_COM_EFFORT_MODE, which is triggered when a grunt or effort sound is detected in the voice. This may happen when one is undergoing physical exertion, and typically involves a tightening of the vocal chords and guttural non-speech sounds. This auto mode can be used to trigger expressions that reflect effort, such as an extreme contraction of the facial muscles and gritting the teeth.

API Changes

There are various API changes for the new nonverbal features, as well as some straight name changes such "moods" now being called "modes", and "controls" now being called "modifiers".

SG Com 4

SG Com 5

Description

SG_COM_STATUS_MOOD_CHANGED

SG_COM_STATUS_MODE_CHANGED

Name change

SG_COM_EngineControl

SG_COM_Modifier

Name change

SG_COM_SetEngineControl

SG_COM_SetModifier

Name change

SG_COM_GetEngineControl

SG_COM_GetModifier

Name change

SG_COM_CTRL_SCALE

SG_COM_MOD_SCALE

Name change

SG_COM_CTRL_SPEED

SG_COM_MOD_SPEED

Name change

SG_COM_CTRL_EXPRESSION_FREQ

SG_COM_MOD_EXPRESSION_FREQ

Name change

SG_COM_SetMood

SG_COM_SetMode

Name change

SG_COM_GetMood

SG_COM_GetMode

Name change

SG_COM_GetMoodList

SG_COM_GetModeList

Name change

SG_COM_EngineRole

SG_COM_Role

Name change

SG_COM_SetEngineControl

SG_COM_SetModifier

Name change

SG_COM_GetEngineControl

SG_COM_GetModifier

Name change

SG_COM_STATUS_EXPRESSION_CHANGED

New member of SG_COM_Status enum. The payload includes the path of the expression in the expression library.

SG_COM_AutoMode

New enum of auto modes

SG_COM_POSITIVE_MODE

Member of SG_COM_AutoMode enum

SG_COM_NEGATIVE_MODE

Member of SG_COM_AutoMode enum

SG_COM_EFFORT_MODE

Member of SG_COM_AutoMode enum

SG_COM_ACKNOWLEDGE_MODE

Member of SG_COM_AutoMode enum

SG_COM_SetAutoMode

Maps an auto mode to a specific behavior mode of the character. If not specified, the auto mode will not trigger any change in behavior

SG_COM_UnsetAutoModes

Clear the auto mode mappings

SG_COM_ActivateAutoModes

Use this to activate auto mode detection. Replaces use of keyword 'auto' in SG_COM_SetMode

SG_COM_SetDefaultMode

Use this to change the default mode, which is the mode that will be active if no other mode is automatically or explicitly set.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.