Behavior Modes

An initial set of behavior modes will be created by the Speech Graphics team as part of the character setup service. However you can edit these in SGX Studio.

Behavior modes make it easy to determine an entire behavior pattern with a single label. At any point in time during animation, your character is in some behavior mode or other. Behavior modes are a powerful creative tool for designing behaviors of the character at setup time.

Behavior modes may correspond to emotional or physical states, like “happy”, “angry”, “tired” or “struggling”. The set of a character’s behavior modes is entirely user-defined and open-ended: you can create as many as you wish (see SGX Studio product guides on how to set up behavior modes). Your initial character control file will include a set of default behavior modes, which correspond to the automatically detected modes of the system: Positive, Negative, Effort, and Neutral (See SGX Auto Modes.)

Here is an example of an animation with changing behavior modes. The current behavior mode is printed above the character as she speaks (as a “mood”).

Behavior mode definitions

Essentially, a behavior mode definition contains a variety of nonverbal poses – which we call “expressions” – that are exemplary of that mode. For example, if you have a behavior mode “Happy”, then you would populate that mode with a variety of happy-looking expressions. When the character is in a behavior mode, it draws on expressions from that mode and deploys them automatically with timing and dynamics determined by Speech Graphics algorithms. By this process the face is continuously changing, autonomous and alive.

When constructing a behavior mode it is important to populate it with a number of different expressions. One may be tempted to use a single prototypical expression to represent the entire emotion or state. However, this can result in repetetive and predictable behavior, which is highly unnatural. Using a sufficent number of alternate expressions ensures that the character appears organic rather than mechanical. Keep in mind that real people never make the exact same facial expression twice.

For example, below is a set of expressions created for the behavior mode “Effort”.

Expression definitions

Each individual expression is defined as a pose of the character’s muscles. For example, the following expression consists of the muscle activations outlined in the table below.

Thus whereas each muscle is formed by setting the rig’s animation targets, each expression is formed by setting the muscles. This hierarchical relationship is shown in the diagram below:

Expressions should involve the entire face, as this keeps the face alive and prevents any “dead” areas. Furthermore, expressions are not restricted to the face. They can include head motion and other parts of the body as well. Using the full musculature of the character in defining expressions keeps the character feeling alive.

During animation, expressions are automatically selected, timed and blended by the muscle-dynamic system. The expressions are also scaled according to the intensity of the speech. For example, the expression from above is shown here at different scales.

However, when defining a new expression, it is essential to pose it at the maximum intensity conceivable for that expression. If expressions are posed too weakly, then they will result in not very expressive animation. Note that in actual animation you will probably never encounter an expression at 100%. But if the maximum pose is not truly maximum, then by extension all lower scales of the expression will appear weaker than the system intends, and there will be degrees of intensity the character cannot reach when desired.

When defining an expression, pose it at its maximum intensity.

Expressions are a good way to achieve idiosyncratic behavior in the character. Assymetries and other quirks can be accomplished easily using expressions.

Importing expressions

Expressions consist of poses of the Speech Graphics muscles. However, at times you may have poses of a character in terms of the rig parameters themselves – for example as a result of motion capture or hand animation – which you want to use as expressions. To do this, you may import the poses into the muscle space: SGX Studio provides a tool which finds the best fit of the muscle activations to a current pose of the animation targets on the rig. Note, this does require that the controls you use to create the pose are included among the animation targets.

Default behavior mode

Every character has one default behavior mode that is active if no other mode has been specified or automatically triggered. Normally, the default mode should be one in which the emotion is neutral. A neutral mode can still have expressions, but should not express any intense mood or physical state.

Bi-tonal behavior modes

Behavior modes may be either bi-tonal or uni-tonal. Uni-tonal modes contain a single pool of expressions. Bi-tonal modes contain two pools: high-tone expressions and low-tone expressions. The distinction is based on speech intonation patterns. During processing, high-tone expressions are triggered during high or rising pitch, and low-tone expressions are triggered during low or falling pitch. The expressions in the two groups should have contrasting characteristics. The following characteristics are recommended:

By causing expressions to change quality as the voice rises and falls, bi-tonal modes make the facial animation more responsive to vocal changes and more closely aligned with speech patterns. This gives a more a convincing, coherent and expressive performance overall.

Behavior mode best practices

Here is a summary of the best practices with respect to authoring behavior modes:

• Do make behavior modes bi-tonal, especially modes that are expected to be used during speech.

• Pose each expression with the maximum intensity the expression could conceivably have on a real person’s face.

• Use a good number of expressions per mode to support variation and avoid repetitive behavior. A good rule of thumb is 7 or more expressions per tone group.

• Move various parts of the musculature in the expressions to keep the character alive. No part of the character should be continuously still. Include head muscles in expressions to generate head motion. Also use expressions to drive non-human anatomy like anntennae.

• Add assymetries and other idiosyncratic quirks to expressions.

• When using assymetry, avoid randomly switching sides. For example, if the character raises the left eyebrow sometimes and the right eyebrow other times, this may cause odd-looking alternations. Keep assymetries natural and one-sided.

• When creating behavior modes that will be used during speech, avoid extreme displacements of the lip muscles, which may interfere with speech; avoid jaw opening altogether.