What is SG Com?
The SG Com SDK provides access to Speech Graphics' real-time audio-driven facial animation technology, enabling developers to build applications with avatars driven by voice.
SG Com's processing Engine converts an incoming audio stream into synchronous, high-fidelity facial animation which may be streamed to other endpoints. Output animation includes lip sync, full-face emotional expressions, blinks, eye darts, and even head movement, all driven by the speaker's voice through proprietary Speech Graphics algorithms. The processing delay between input and output is 50 milliseconds.
SG Com also includes a Player that handles buffering, decoding and synchronized playback of the animation produced by the Engine.
The SDK includes:
SG Com API, a C API
SG Com Unreal Engine Plugin, for easy integration into the Unreal Engine
C# bindings and sample Unity integrations are also available on request.
What’s new in SG Com 5?
CPU cost savings
For bone rigs, the CPU cost of SG_COM_UpdateAnimation has been reduced by two thirds (per bone). See SG Com Compute Resource Usage.
Licensing
There are now two licensing options:
key-based licensing
Key string is input into
license_data
argument of SG_COM_Initialize
cloud-based licensing
Path to .lic file is input into
license_data
argument of SG_COM_InitializeAvailable on Windows and Linux only
Requires internet connection with two open ports
Expression Changed status type
SG Com emits a new status type for the SG_COM_EngineStatusCallback: SG_COM_STATUS_EXPRESSION_CHANGED. This is triggered whenever SG Com starts a new facial expression. The payload includes the full path of the expression. This can be used to synchronize other animations with the expressions.
Auto Mode Controls
We now use the enum SG_COM_AutoMode for automatically detected states such as emotions or other vocal events.
SG Com 4 relied on specific naming of the character's behavior modes to trigger them automatically, such as "positive", "negative", "acknowledge". Now auto modes can be mapped to any behavior mode at runtime, so there is no longer a naming requirement.
For example:
SG_COM_SetAutoMode(engine_handle, SG_COM_POSITIVE_MODE, "happy");
maps the positive auto mode to the character’s behavior mode called “happy”, whereas
SG_COM_SetAutoMode(engine_handle, SG_COM_NEGATIVE_MODE, "angry");
maps the negative auto mode to the character’s behavior mode called “angry”. Note that an auto mode must be mapped to something in order for it to have an effect on behavior.
To clear the current auto-mode mappings, use
SG_COM_UnsetAutoModes();
The default behavior mode, which is used when no auto mode is detected, can also be changed:
SG_COM_SetDefaultMode(engine_handle, "friendly");
Note that in order to activate automatic mode detection, we now use
SG_COM_ActivateAutoModes(engine_handle);
Whereas to turn off automatic mode detection and instead use a fixed mode, e.g. "happy", use
SG_COM_SetMode(engine_handle, "happy");
Effort Detection
There is a new auto mode, SG_COM_EFFORT_MODE
, which is triggered when a grunt or effort sound is detected in the voice. This may happen when one is undergoing physical exertion, and typically involves a tightening of the vocal chords and guttural non-speech sounds. This auto mode can be used to trigger expressions that reflect effort, such as an extreme contraction of the facial muscles and gritting the teeth.
API Changes
There are various API changes for the new nonverbal features, as well as some straight name changes such "moods" now being called "modes", and "controls" now being called "modifiers".
SG Com 4 | SG Com 5 | Description |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
|
| Name change |
| New member of | |
| New enum of auto modes | |
| Member of | |
| Member of | |
| Member of | |
| Member of | |
| Maps an auto mode to a specific behavior mode of the character. If not specified, the auto mode will not trigger any change in behavior | |
| Clear the auto mode mappings | |
| Use this to activate auto mode detection. Replaces use of keyword 'auto' in | |
| Use this to change the default mode, which is the mode that will be active if no other mode is automatically or explicitly set. |