Abstract
In this final blog, I'll walk you through the process of making character animation while highlighting two essential elements: body mechanics and lip sync. The keyframe animation technique is used in my approach to combine several reference elements from three different videos to create a cohesive and compelling sequence of animation. The character animation shows the value of reference materials, the accuracy necessary for keyframe production, and the blending of numerous aspects to bring a character to life in a compelling way.
Acknowledgements
I would like to express my sincere gratitude to the following people and organisations for their significant contributions and steadfast support during the process of making my animation project: I am incredibly appreciative of my supervisor Mr. Marco Savastio’s constant advice, mentoring, and knowledge throughout the animation project. Their advice and constructive criticism had a critical role in determining the project's course and elevating its quality. My Professor Mr. Richard Cunnigham for his endless support and advises equipping me with the skills and techniques required for animation. I’m grateful for Kingston University for providing me with necessary, tools, and facilities that enabled me to undertake and complete this project. My genuine gratitude goes out to my classmates and friends who contributed creative thoughts, had helpful discussions, imparted their knowledge and encouraged and inspired me throughout the process.
Introduction and Background
Introduction
This project has been a fascinating exploration into character animation that incorporates the three key elements which are body mechanics, facial animation, and lip sync. When I came across a compelling line from the film "Crash Pad" (2017)(9.1), it gave me the idea for this project. I was initially intrigued and curious by the dialogue's limitless potential for different artistic expression which could be depicted through character animation. The initial phase of this project involved the conceptualization of two distinct scenarios for contextualizing the dialogue. In the first scenario, a character would be seated at a bar stool while another, who appeared to be inebriated, approached to deliver the dialogue. The main goal was to highlight the subtleties of bodily mechanics, particularly those of a drunk individual. The intriguing task of reproducing the swaying movements, shaky balance, and exaggerated gestures that characterise such a character arose from this. The project's second concept introduced a whole new dynamic. In this scenario, I imagined a barman conversing with a customer who was seated at a bar while expertly completing bartending tasks. This idea was selected in order to display a wider range of body motions, and the bartender's elaborate trick performance added an exciting dimension to the story. This specific situation offered an intriguing chance to investigate a greater variety of body mechanics. The barman made quick, precise movements that required close monitoring and animation as he expertly prepared drinks. Here, the difficulty wasn't just to accurately depict bartending techniques; it was also to make sure that they were seamlessly included into the dialogue being spoken between the individuals. I had a foundation in character animation techniques thanks to my prior coursework. As I began this project, this underlying knowledge proved to be invaluable because I had already studied the subtleties of converting emotions and language into accurate facial expressions and lip movements. All the available tools, including AnimBot, Studio Library, and AnimSchool Picker, worked with Maya without any issues, speeding my process and enabling more effective and accurate character animation.
Background
I did a lot of preliminary research to get a thorough understanding of these important components as I got ready to work on my character animation project, which includes body mechanics, facial animation, and lip sync animation. To provide a solid foundation for building an engaging and realistic animation sequence, my goal was to go deeply into each aspect of character animation. For example, while researching and learning about body mechanics I came across this article which had some great insights such as the first thing you have to break down is how the character stands. You must arrange body parts to describe inner life. How a character stands reveals emotional state, energy level and attentiveness. Everything that you do from there is a sequence of lead and follow that shows how that particular character moves. Nothing moves without a force either driving or pulling it. I maintained a close eye on market developments and the newest strides in character animation throughout this background investigation such as mocap, unreal meta human which is a next big thing in the current animation industry. Unreal MetaHuman is a new tool from Epic that allows you to quickly create a digital character for Unreal Engine. This character is fully ready to be animated and has all the necessary prep to work with motion tracking systems (8.2) I kept up with industry forums, tutorials, articles on the subject, and case studies of successful character animation projects in order to stay updated of best practises and new methodologies. In conclusion, my previous studies on character animation, which included body mechanics, facial movement, and lip sync animation, was an essential first step. It gave me the information and tools I needed to approach this job with assurance, enabling me to make an engaging and lifelike animation sequence that effectively conveys emotions and storytelling.
Main Body
Aims and Objectives
My final project's main goal is to dig deeper into the intriguing world of character animation, with a focus on understanding the challenging nuances of body mechanics, facial animation, and lip sync animation. This project is a result of my unrelenting passion for the animation classics produced by studio behemoths like Pixar, Dreamworks, and Sony Pictures Animation, all of which have had a lasting impression on the field with their distinctive and visually arresting works. I've always been a huge fan of these animation studios and have always been completely enthralled by their talent for bringing characters to life through distinctive and highly styled animations. Particularly Pixar is well-known for its talent to give characters a variety of emotions and characteristics that connect with viewers of all ages. Sony Pictures Animation frequently pushes the limits of innovation with its unique visual styles, while Dreamworks excels at creating characters with humour and wit. I will be able to give characters a sense of physicality and realism by using body mechanics animation, which is the cornerstone of genuine movement and action. I want to become an expert in the intricacies of weight, balance, and motion so that characters can move fluidly and convincingly inside their respective settings. I want to go deeply into the field of facial animation, as shown in these studios' expressive characters. Understanding the complex interactions between facial muscles and emotions will help me create animations that have a powerful emotional impact on viewers. Another skill I want to learn is lip sync animation, which is the technique of matching character speech to realistic mouth movements. Characters created by Pixar, Dreamworks, and Sony Pictures Animation are exceptional at having lip movements that match their words, which improves the overall immersion and storytelling. A crucial phase in my quest would be figuring out how to achieve this level of synchronisation.
Project Plan
Methodology
Four different references were taken and recorded for this project. First reference which was a main sound of the project was taken from the movie Crash Pad (2017). The second reference was taken from the movie Uncharted (2022) in which Tom Holland is acting as a bartender and performing tricks. Third reference was recorded by me lip syncing and acting out the first few seconds of the animation. Fourth reference is the recording of the person sitting on a barstool. I created in which one would be the focus of body mechanics whereas the other one would focus on lip sync part of the animation. The poses from lip sync will be added to studio library and then referenced to the main file containing body mechanics.
Work Plan
The main steps in my work plan were to shoot and gather references, choose rigs and the setting, and then begin the animation process for the body mechanics and lip sync. Shooting and gathering references was the first step. I started by recording and gathering references in the initial phase. This was a key step to do to guarantee that my animation is realistic and portrays real motions.
Gathering References:
My first step involved gathering reference materials such, the sound/ dialogue which I wanted to animate my narrative animation. For that I already had dialogue by Domhnall Gleeson from crashpad (2017) which could be explored for different artistic expression as a mode of character animation. I wanted to change the context from the movie to something different from character talking to himself to a talk between two characters and gave the environment setting of a bar/pub. As for video reference for the dialogue depicting people seated on bar stools, bartenders performing bottle tricks and other relevant actions or scenarios I researched about many videos online. When it came to creating animation references for body mechanics, I found myself heavily relying on the performance of Tom Holland in the movie "Uncharted." In this endeavour, I sought to meticulously observe and analyze his actions as he portrayed a character engaged in bartending tricks. For the person sitting on the bar stool I shot the reference of my friend trying to act as a drunk person. These references proved to be immensely helpful in capturing the nuances of body mechanics.
Analysing References:
Once I collected the references, I dedicated time to meticulously studying them. My aim was to gain an understanding of the intricacies in the actions and movements that we planned to animate. This thorough analysis played a role, in ensuring that I accurately captured the subtleties of motion and body language. Second step involved choosing Rigs and Setting The next step involved selecting the character rigs and environment for the animation.
Character Rigs:
I took care in choosing character rigs that were well suited for our project. These rigs needed to be able to depict the required body movements and facial expressions while also aligning with the style and aesthetics of the animation. I decided to use the Malcolm rig from Animschool because of its sophisticated controllers and compatibility with the character I was animating as a barman. The rig included a comprehensive set of controls that made it possible to precisely control the character's expressions and movements. This degree of control was essential, especially when animating a barman who engages in complex movements and interactions with shakers, bottles and glasses. I was able to provide the degree of detail and realism I sought in the animation thanks to the sophisticated controllers. The second rig of a character sitting on a barstool was Dude rig provided by our professor which gave an attractive silhouette when looking from the back.
Environment:
Selecting the setting was crucial, in creating a scene. Whether it was a bar or any other location I had to ensure that the environment complemented the narrative and contributed to the storytelling. I searched or it online and I came across bar environment by anim_matt on gumroad which felt like a perfect fit for my animation. Third step was animating body mechanic file After setting up the environment, rigs, and references, I started the animation process by focusing on the body mechanics.
Blocking Pose for Character 1:
The first character, who was seated on a bar stool, needed to have his positions blocked out. This involved establishing the character's essential poses and motions and making sure that they matched the reference I had shot of my friend. I tried to portray the character as defeated, which was based on the character's sadness and lack of confidence. This artistic choice played a crucial role in accurately expressing the character's emotional state. I posed the character's body language and posture to achieve this. The character was deliberately posed to depict the emotional burden he was carrying with slouched shoulders, a drooping head, and a bowed back. The character's sense of failure and hopelessness could not have been expressed graphically without these physical characteristics.
Animating Character 2(Barman):
I decided to use Tom Holland's performance from "Uncharted" as a model for the blocking of the second character in my animation project, concentrating on his portrayal of a barman performing tricks. This choice was motivated by the requirement to combine complex body mechanics and a range of props, a difficulty I had never experienced in my animation work. The use of several props, including bottles, glasses, and shakers, increased the difficulty of this animation work. It took careful preparation and execution to include these props into the character's motions in a seamless manner. It was a challenging task because I wanted to make sure that the character's body mechanics and interactions with these props looked realistic and plausible. Step four is doing lip sync animation The lip sync portion of the animation was now added, and it was carried out in a separate file. The idea behind this was to animate the lip sync of the character and import the whole animation through studio library from the temporary file to the main file.
Lip sync file setup:
This file involved importing only the character rig of barman as I wanted to `animate lip sync of only one character. The environment was not needed to do the animation.
Lip Sync Animation:
With the file ready, at first I made all the different controllers sets in animbot for whole face, mouth, cheeks, nose, tongue etc so that it gets easier for me to animate selecting just the sets. To research and learn more in depth about the lip sync animation i referred and looked through 2-3 animation videos which my supervisor suggested me. After that, I concentrated on animating the characters' lip sync. I posed my character in the T position. This choice was made in order to simplify the animation process and free up my attention to focus solely on the character's face emotions and lip movements. By keeping the actor in this position, I was able to avoid the difficulties of animating body movements like arm gestures and torso changes. I was able to focus entirely on the complex details of lip sync because to this approach's simplification. To ensure that his facial expressions and lip movements matched the spoken words precisely, this entailed painstakingly synchronising them with the dialogue or audio file that was being recorded.
I did the research and utilised the concepts of the Facial Action Coding System (FACS) in order to produce a successful lip sync animation. By making this calculated decision, I was able to get a thorough understanding of the complex facial muscle motions required for lip sync and face expressions, laying the groundwork for the rest of the animation process.
I developed a thorough framework for examining and interpreting face movements by studying FACS. The Facial Action Coding System (FACS) refers to a set of facial muscle movements that correspond to a displayed emotion.
I ran into a hurdle when I started the animation process for the lip sync project. The speech contained specific words and phonetic sounds that I have never before tried to animate. This unfamiliar territory presented a problem, so I asked my supervisor for advice. Upon discussing my predicament with my supervisor, he offered a valuable solution. He suggested that I start the lip sync process using a conventional method, similar to puppeteering, by concentrating only on the character's mouth's basic opening and shutting motions. The basic pass for lip syncing would be this first step.
My main goal was to ensure that the mouth moved in time with the spoken words, synchronising the character's voice with the dialogue. Despite the difficulties posed by unknown phonemes and words, this streamlined approach allowed me to advance on the animation.
I moved from the first pass of my lip sync animation project, which concentrated on fundamental mouth movements, to the more challenging process of moulding the character's mouth to match the precise phrases in the dialogue. The key objective was to keep the animation process organised and clear while avoiding chaos and clutter.
I used a structured process in order to complete this step successfully. Instead of making new animation keys from scratch, I expanded on the framework created by the first iteration. This method allowed me to operate within a framework that I was already accustomed to, resulting in an animation workflow that was smoother and more cogent.
In the third stage of my lip sync animation project, I moved my attention to the challenge of enhancing the established lip sync for certain dialogue words. This stage aims to improve the character's dialogue in terms of expressiveness and clarity, giving the animation more depth.
In order to achieve this, I carefully selected dialogue points where exaggeration would be advantageous. These situations usually required the character to express strong feelings and emphasise key words. I went back and carefully adjusted the keyframes I had already established for each of these moments. To provide a greater sense of expressiveness, it was necessary to alter the character's lip movements and forms in addition to their natural articulation. I was very mindful to time these exaggerations so that they matched the character's vocal delivery and the desired emotional effect. Throughout this stage, I tried to strike the right note that best suited the character's personality and the dialogue's tone by striking a balance between exaggeration and subtlety. It was essential to make sure that the exaggerations gave the animation depth and realism rather than coming off as forced or artificial.
I finished animating the lips and then moved on to animating the character's other face features, which included the cheeks, nose, and brows. Before incorporating the character into the main body mechanics file, this stage was essential for giving the character a wider range of expressions and feelings. For example, moving the side parts of nose up and down as per the mouth shapes. The context in which my character was meant to display emotions was decidedly nonchalant and carefree. This particular characterization influenced my animation decisions, leading me to limit the range of expressions portrayed by the character. Given the character's casual and laid-back demeanour, I chose to use more muted and mild facial expressions. The idea was to portray feelings in a way that fit the character's carefree demeanour. I avoided using extremely dramatic or exaggerated facial gestures as a result. I purposefully chose not to animate the character's face rotations and eyes at all in this particular animation file. This decision was made for several reasons, including the fact that the file was effectively a draught without a setting or other characters. As a result, difficulties with context and viewer perception would have arisen from animating the character's facial expressions. Without a clear setting or interaction with other characters, it would have been challenging to animate the face and eyes. Here I faced an issue for the animbot sets. I experienced a problem with the setups I had made for AnimBot during the animation process. I had configured a complete set for the entire face in the hopes that it would successfully assign keys for all necessary controllers. Nevertheless, I found that, despite the set's controllers appearing to be selected all at once, not all of their keys were being set as I had anticipated. This problem was visible in numerous areas of the animation when some controllers were missing keyframes.
Since the lip sync animation was largely concerned with mouth motions and face expressions, this disparity wasn't a major issue. However, I understood that moving the animation from one file to another could prove to be somewhat difficult. I required the animation to be placed on the exact same controllers in both the source and destination files in order to ensure consistency between files. I took a methodical method as suggested by my supervisor to deal with and fix the problem. I started by finding a blank frame in the timeline of the animation, which served as a neutral reference point. I then chose the whole AnimBot set, which included all of the facial controllers, at this frame.
I chose the full set and started to work methodically. I carefully went over each controller in the set one at a time. My goal was to make sure each controller's keyframe had been set appropriately on that particular frame. This careful examination made sure that no controller was unintentionally left out of the animation keying procedure. The methodical process allowed me to verify that each controller in the AnimBot set had the necessary keyframe on the specified frame, eliminating the problem of certain controllers' missing keys. A more reliable and robust animation workflow was intended by this extensive verification process, which sought to remove any errors and discrepancies and ensure that the animation data was appropriately collected on all pertinent controllers.
Step 5 was adding in-betweens to the body mechanic’s file.
As I began the process of inserting in-between frames into the body mechanics file, I quickly realized that I needed to make significant adjustments to the initial poses. This realization was crucial to ensure that the animation did not appear overly snappy or abrupt, which would have compromised its overall quality and fluidity. Despite the fact that the initial poses were crucial anchors for the animation, the transitions between them were frequently abrupt and unnatural because they were frequently too severe or inflexible. This was notably apparent while attempting to achieve smoother, more lifelike body motions.
I ran into a number of timing and spacing issues when working on the animation in various portions of the project. These problems became more apparent at several crucial times, one of which featured an obvious issue with hand placement.
I noticed an apparent inconsistency that required fixing when the character's hand appeared to be clipping through the table at a specific point in the animation. The hand's movement in respect to the table was not coordinated, which led to an unnatural and distracting overlap. To guarantee that the hand interacted with the table more convincingly, this issue required careful keyframe correction. Another problem I stumbled into was a noticeable snap when switching the character's arm from an earlier frame's IK (Inverse Kinematics) setup to the next frame's FK (Forward Kinematics) setup. The animation appeared jerky due to the sudden change in the arm's motion, which broke the animation's natural flow and smoothness. In order to obtain a more seamless look, this problem was fixed by going back to the frames and positioning the FK arm extremely close to the IK arm.
Furthermore, I had not yet constrained the props to the character's hand at this point. Since my main goal was to finish the full animation before moving on to restricting the props, I had decided to postpone this step. Because of this choice, I was able to complete the character's movements thoroughly before introducing the complexity of prop limitations.
When I got to this point in my project, I realised I had made a big mistake: I had unintentionally shifted the rig's master controller. The fact that all the other controllers were moved from their intended placements in relation to the character's bodily parts as a result of this miscalculation had serious consequences.
I came up with a method to reinstate the proper rigging in order to repair this significant mistake. My strategy was to refer back to the rig after any accidental movement and then input the master controller's value. The goal was to exclusively animate the body parts in the frames before the master controller was initially relocated, while transferring the complete animation, including the character's movements, from the old rig to the new one. Nevertheless, despite my best efforts, this strategy did not provide the expected outcomes. The value I copied and pasted from the previous rig didn't exactly align the master controller of the new rig to the same place for reasons I couldn't immediately understand. Because of this misalignment, the animation was unable to switch between the old and new rigs without significantly disrupting the character's movements.
In my second try, I thought of removing earlier frames from the animation, notably those up to frame 150. My justification for this method was the notion that these opening frames served as a context for the discourse that followed. But it soon became clear that just removing these frames wouldn't produce the required results. The reason for this was that by eliminating them, the accompanying sound became disjointed and lacked sufficient context. Without the earlier frames, the animation lacked the essential buildup to support the dialogue that began in frame 200 and appeared fragmented.
After much careful deliberation and reflection, I arrived at a decision regarding the body mechanics file. I realized that I was not entirely satisfied with the outcomes I had achieved in the earlier version of the file. Several issues had surfaced, and it became evident that rectifying these problems required a comprehensive approach. One of the another fundamental issues I identified was related to the elbow joints. In my previous file, I had unintentionally rotated both of the character's elbows in various directions. whereas in the context of FK, it is generally recommended to move the elbow joint in just one direction to achieve smoother and more predictable results without twisting of meshes. I made an important decision as a result of this understanding and my general displeasure with the old file's animation quality. I made the important decision to redo the whole blocking out of positions in the body mechanics file. This project involved a sizable 700 frames.
The main driving force behind this decision was the conviction that starting over would enable me to correct all the errors and inconsistencies that had gathered in the prior file. By reblocking the animation, I was able to guarantee a more logical and technically solid foundation for the following phases of animation.
I made a number of adjustments as I worked to improve the overall quality and narrative flow of my animation project. The frame sequence adjustment was one of the key changes I made. I chose to remove the animation's first 99 frames in particular.
This choice was made with two justifications in mind. First, I noticed that neither the storyline nor the character development was much impacted by the first 99 frames. In essence, there wasn't much action or interesting material in this part of the animation. As a result, I decided that it was necessary to delete these frames in order to make the video shorter and more interesting to watch.
Second, I wanted the initial character to serve as the animation's central focus by being introduced while perched on a barstool. In order to do this, I used camera cuts to switch from the opening frames to a new angle, emphasising the first character sitting on a barstool.
The handling of the items in the barman's hands as he performed his sophisticated tricks became a crucial emphasis as I advanced with my animation project. Given the complexity of the animation, at first I tried to keyframe the placement and orientation of every prop in every frame.
I had a total of six props and about 570 frames to work with, so it quickly became clear that this strategy wasn't realistic. The method was time-consuming and cumbersome due to the enormous number of keyframes needed for each prop on each frame.
As a result, I considered the possibility of constraining the props to the character's hands, which looked like a more practical solution. I did, however, face a big obstacle in the process. There were a few scenes in the animation where the character had to pick up a prop, set it down, and then pick it back up. When I applied the constrains, it had an impact on every frame of the sequence, even the ones where the character was supposed to let go of the prop.
In an effort to fix the problem, I changed the constraint's mode in the hopes that it would give me greater control over how and when the props were restrained and released. Unfortunately, I was unsuccessful in using blend mode modifications to discover a workable solution.
I asked my supervisor for advice when I ran into a problem managing the animation's props appropriately. My supervisor gave me two solutions to solve the problem and speed up the animation process.
The first technique required using Animbot's "xform" feature. The second solution involved applying Maya constraints and then baking the prop controller for particular frames when constraints were required. Importantly, my supervisor stressed the value of applying limits to prevent disrupting the entire animation. In order to do this, I carefully handled the keyframe editor, making sure that only the frames that needed limitations were kept and that the others were removed.
I had previously ignored this guidance when making my initial attempts at constraining the props, but it turned out to be a crucial step. By employing these suggested techniques, I was able to manage the props with more accuracy and control, which led to a smoother and more realistic animation in which the props acted as planned without interfering with the entire animation process.
Although at one part I faced an issue where the bartender is turning the bottle over his head. The bottle was not following the wrist directions and was clipping with the head. So, I constrained it part by part with only those frames selected in which wrist is moving in one direction and another part in which wrist is moving another direction. That way I could ensure that the prop was not clipping it to the head of the bartender.
I had to make multiple corrections after constraining the props as per my supervisor’s suggestions and every time I went back and deleted those particular baked frames, readjusted the positions and apply the constrain and bake it again. The last part where I faced the problem was with the spoon as I had to animate the spoon turning after constraining it to the mug. I tried doing it in an anim layer but after doing that the spoon didn’t remain constrained to the mug, so I hand keyed the part were the spoon is turning in the mug and then baked the later part of the frames.
Self Critique
As I reflect upon the whole project, I realised I couldn’t do lot of things which I wanted to do initially such as create fx of liquid pouring out of the bottle and into the glasses, but I didn’t get enough time to work in that direction because of lack of time management. I spent most of my time blocking out the poses which I could’ve finished early on. On top of that I did lot of mistakes in my old file such as moving the master controller and rotating elbow in all directions which could’ve posed lot of issues later. My biggest issue was time management. I couldn’t give lot of time for doing in-betweens and incorporate principles of animation such as squash and stretch, anticipation in barman character, exaggeration etc. I wanted to include more complex body mechanics but I couldn’t find more references on that. Moreover, I had to make lot of modifications till the end for the poses which affected my constrains as well as timing and spacing which again stems from the same issue of proper time management. I also have still lot of practice to do in regard of applying timing and spacing in the blocking stage.
Conclusion
In summary, this animation project has been an adventure filled with artistic inquiry, technical difficulties, and priceless learning opportunities. I had to recreate the blocking postures and constraining several times throughout the project, but it also taught me a lot about the subject. My supervisor's advice and insights were quite helpful in helping me polish the animation and solve challenging technical problems. Their recommendations and comments acted as a beacon of direction for the entire endeavour. I can't wait to put the knowledge I've gained from this project to use in my future endeavours. My goals are to continue honing my craft, research cutting-edge animation methods, and push the envelope of animation innovation. In essence, this animation project marks the start of an exciting path towards becoming a more skilled and creative animator, as well as the fruition of hard labour. It has given me priceless insights and lessons that will certainly influence my future career in the animation industry.
Ethics
According to Kingston University’s rules and regulation regarding plagiarism, the project aligns with it. Every reference is duly credited and discussed and agreed upon by the supervisor. There is no known or intentional copying of any sorts. The inspiration behind this animation came from a movie named Crash Pad (2017) directed by Kevin Tent in which the actor (Domhnall Glesson) is speaking to himself in a mirror whereas I changed the context of the dialogue to a conversion between a customer and bartender in a bar. The bartending tricks reference is taken from the movie called Uncharted (2022) by Ruben Fleischer in which actor Tom Holland is performing the tricks.
Comments