In SGX, characters are animated through their muscles. We define a "muscle" as an independently controllable movement – similar to the concept of "action unit" from the Facial Action Coding System.
Because they are biologically based, muscles provide a natural way to parameterize facial movement. They also have the benefit of being largely universal and analogous across characters, thus allowing a generalized behavior model.
Speech Graphics characters start with a default set of muscles but this is open ended. Users may add new muscles and delete muscles. (However there is a core set of protected muscles that are expected to exist by SGX algorithms and cannot be deleted.) One can even add muscles to control non-human anatomy, such as antennae, gills, eyeballs on stalks, or other non-human appendages. Such muscles can be driven by the nonverbal animation system along with other muscles.
Each muscle has a time-varying activation, represented by a parameter with range [0, 1], where 0 is fully relaxed and 1 is fully contracted. For bidirectional muscles, which have two opposite effects, the range is [-1, 1]. The muscle activations are converted into pose weights (see Muscle Poses). Muscle activations are driven by internal SGX “muscle-dynamic models” that simulate speech production and nonverbal movement. Each muscle is independently controlled and has its own dynamic properties.
Muscle properties
Muscles have two kinds of settable properties:
-
Muscle poses – the actual deformations caused by each muscle, either individually or in combination
-
Muscle dynamics – parameters of the muscle’s dynamic profile, determining factors such as speed of contraction and relaxation.
Muscles are the building blocks of Speech Graphics animation. Therefore good muscle setup is the primary key to good character animation; and poor muscle setup can be an obstacle to good animation.
See Editing Muscles for a user guide to the muscles editor in SGX Studio Maya.
Is this FACS?
Speech Graphics is not unique in analyzing the face into constituent motor functions. One of the first analyses of facial motion into basic motor functions was proposed in 1970 by Carl-Herman Hjortsjö and expanded upon by psychologists Paul Ekman and Wallace Friesen, whose Facial Action Coding System (FACS) provides a comprehensive analysis of facial motion. They claimed that while there are 268 muscles in the human face, the face performs only 46 possible basic actions, which they called action units. Each action unit is the motion of one independently controllable muscle group, which gives rise to a characteristic displacement pattern on the facial surface. In principle, any facial expression, whether involved in speech, affect, or some other behavior, may be described as one or more action units activated to a certain degree. While originally designed for description of facial expressions, FACS is widely used today as a basis for facial rigging, with rig controls modeled after action units.
The Speech Graphics muscle set can be related (though not exactly matched) to FACS action units. There are a number of important differences between the two systems. Speech Graphics muscles:
-
are more tailored to an understanding of the dynamics of speech production;
-
include not just facial muscles but also muscles of the internal vocal tract, including the tongue, as well as muscles of the body such as head movements and chest breath;
-
are an open-ended set which can be extended by animators, including muscles for non-human anatomy such as antennae;
-
can be bidirectional;
-
have built-in dynamic properties.
Muscles are not visemes
Muscles are used to animate both speech. However, note that muscles are not visemes.
A viseme is a group of several speech sounds which are considered to look similar. For example, the sounds /t, d, n, l/ may form a single viseme, likewise /p, b, m/ – because the members of these groups are highly confusable visually. In traditional facial animation, lip sync is achieved using a target pose for each viseme, with some form of interpolation between successive visemes.
Speech Graphics does not use visemes. While the muscle-dynamic system does produce speech poses corresponding to consonants and vowels – it does so through the cooperation of independent muscles working together to achieve articulatory objectives. For example, achieving /t/ involves contributions of tongue muscles, jaw muscles, and lip muscles operating independently, rather than a single monolitic pose affecting all of that anatomy.
Muscles are not facial expressions
Muscles are also used to animate nonverbal behavior. However, note that muscles are not the same as facial expressions, which are usually more complex than single muscles. For example, a smile might be composed of mouth corner retraction plus various cheek and eye movement – actions that are actually independently controllable on a real person’s face. If we create a single monolithic muscle for each complex expression, the animation will look less natural, because it will move from pose to pose in a way that is not grounded in what is physically possible. See Behavior Modes on how nonverbal facial expressions are constructed from combinations of muscles.