SGX Interactive Processing
The aim of SGX is to automatically produce facial movement that accurately matches the speaker’s vocal performance, with minimal need for user intervention. However, there may be some cases where you want more control over the outcome.
For this reason, SGX 4 provides a fully automated workflow, but also offers an interactive workflow for fine-tuning.
The stages of this workflow are discussed below.
Visualization
Using SGX Director, events can be loaded into an event editor that supports interactive playback of audio and animation. The animation output occurs via live-link from SGX Director to an SGX Studio plugin on the target 3D platform. Below is an example of playback in the event editor with live animation visualized in the SGX Studio Unreal plugin.
This functionality provides the immediate visual feedback necessary for making edits on the timeline.
Editing
After visualizing output, if desired, you can fine-tune the animation through direct editing of the event’s metadata or transcript (see Editing Events).
You can also make edits to the processing environment, including edits to the Character Control File or other settings.
For any edits to affect the output animation, the event must be reprocessed.
Reprocessing
Processing can performed repeatedly. Each time you reprocess, any new edits will have an impact on the resulting animation. Under the hood, reprocessing an event happens as follows:
All sequences in the event are deleted – excluding:
the input (audio and transcript);
any metadata sequences that were edited by the user
The event is then processed as normal, except that edited metadata sequences are used as a substitute for default analyses.
There are ordering effects to be aware of during reprocessing. Sequences in an event are produced from top to bottom. Therefore, changes to higher (earlier) sequences will cause changes in lower (later) sequences. For example, changes to the behavior mode sequence will lead to changes in the expressions sequence. However, if the lower sequence has also been edited, it will not be regenerated, regardless of upstream edits. The only way to regenerate an edited sequence is to delete it.
Antecedents for each metadata sequence are specified in the Metadata descriptions.
The workflow in action
Below is an example of the visualization/editing/reprocessing workflow in action. The animation is previewed, an expression is changed, the event is reprocessed, and the animation is viewed again to inspect the result. This workflow can be iterated until the user is satisfied with the result.