Shot 1 - Walk Cycle against live action backdrop

The last piece of animation that I did for Synaesthesia was a walk cycle which happens at the very beginning of the film.  This is the shot in which our character, Alex is introduced, as well as her condition.

The scene starts with a low angle shot of the sky, which pans down to the house of our protagonist, Alex. Alex is standing outside her house, looking over her shoulder; she eventually looks in front of her and proceeds to walk in to the house.  This scene will involve the CG Alex rig, but will take place against a live action backdrop.

To do this, we recorded to pieces of footage, one with Arpit walking through the front door and one that was just a POV shot going through the front door.  The footage with Arpit would be used as reference for the poses in the walk cycle and also the pace at which the character would need to work.  The POV shot footage would actually serve as the live action scenery for the final piece.  Both of these pieces of footage were tracked by Valentino using a piece of software called PFTrack.  These were then exported to Maya files for me to animate Alex against.

I started with the Maya file that had the tracked footage of Arpit, so that I could animate her walking correctly.  The walk needed to start with her looking over her shoulder at the camera and making the connection with the audience.  She was required to hold this pose for around 60 frames, before turning around and starting her walk.

As you will see, I have animated both the looking over the shoulder movement and the subsequent walk cycle against Arpit's movement; both Alex and Arpit move at exactly the same time.  I could not entirely work out where the inbetween frames should fall, so I just placed them evenly between each keyframe and breakdown.

Once I got the walk cycle movement worked out, I added a slight rotating movement to the looking around pose to create a sense of life in Alex, as well as some arm movement and facial expression.  I initially had her turning quite far around to look at the camera, but upon the wish of the director, I slightly reduced the amount of rotation, just to make it more subtle.

The next stage was to put this animation in the tracked Maya file of the second piece of footage, which would serve as the backdrop for the final render.  I imported the walk cycle into the scene file and then I keyframed the translation of the master control to create the illusion that she moves through this live action space.

I was actually pretty impressed that when I look at this footage through the camera, that she appears to be walking through the front garden and eventually through the front door; what will really be needed to increase the illusion that she is actually part of the scene would be carefully placed lighting and rendering.

As of now, the animation for the Synaesthesia project is finished.  As you will see from my personal schedule, I have completed all 10 shots that I was given to produce character animation for, in just under 5 weeks.  This means that we now have one week left for rendering and compositing; the majority of which is complete.  Synaesthesia is well on the way for being completed before the deadline of 28th May (was extended from 25th) and I can gladly say that I am going to have my name on at least one complete final film, which will be screened at the degree show.

A recap on the order of the scenes and shots as they are now

Since I have been doing the animating for the Synaesthesia project, we added some new camera shots and camera movements into the scenes for greater effect.  As a result some of the shots that come up later in the film now have a different numbering.

For example, the shot in which Alex approaches the piano was originally called Shot 8 and was referred to as Shot 8 in some of my previous blog posts; however, this shot is now to referred by the team as Shot 11, primarily because we have added some more camera shots to the scenes that come before, thus making this shot the eleventh shot chronologically, and no longer the eighth.

I have therefore decided to post up this document which Valentino wrote up and that clarifies what happens in each shot, the shot number and which scene each shot belongs to.

This list will also give a clear indication how I will be referring to each shot in the video that I am making for my Portfolio that will show the blockouts, rough tests and final animation for each shot I worked on.

Piano Animation

Today, I have just finished the piano scene, which consists of a POV shot and a side shot of Alex playing the piano.  This follows on from shot 8, in which I animated Alex approaching the piano and sitting down, and will involve her playing notes on the piano, which causes colours to fill the room.

This piece of animation has been one of my most involved that I have worked on for this project.  Since beginning the degree, I have worked with animating against sound and music before, but I have not actually had to animate a musical instrument being played accurately.  This required me to really make use of both my wide knowledge of animation and somewhat hazy knowledge of musical theory that I acquired from playing guitar for 8 years.  There was a huge amount of trial and error involved before I had this sequence animated in a way that was convincing and appealing.

To work out how hands should move on a piano, Valentino got the composer for the film to record this reference.

The thing that is immediately noticeable in this reference footage are the rotations of the hand.  The left and is very much a straight arc as the hand move between two notes.  The right hand has a slightly more complex movement; the wrist moves in a clockwise direction as each note is played with this hand.

In order to animate the piano, I decided to animate the piano keys first and then animate the hands, as once I had the keyframes for each note laid down, I would then know how the fingers and wrist had to move at each keyframe.  I initially made a test animation directly against this piece of footage; I did not get around to the animating the fingers but I did animate the keys accurately.

I worked out how to animate the keys fairly quickly.  I simply used linear curves and I set a keyframe for each note, when the key went down and then two frames either side of the keyframe in which the key was in its upward position.  It was also fairly easy to visualise when each note played and which one it followed, as the only channel the I was animating to was the Rotate X and the motion was simply linear.

As you see here, I have selected two piano notes and I can see how when one note is played, the other follows straight after.  This would make cleaning up this animation much easier and would allow me to animate against the tune accurately.

However, I needed to redo this animation as the piece of music played in the reference was not the same as the piece of music that was actually in the animatic and that I needed to animate against for the final cut; it was actually at a faster tempo and a higher key.  The left hand plays more or less the same set of notes in the final animatic, but the notes that the right hand should play in the animatic are actually different to what I initially animated and what the composer plays in the reference video.

In order to reanimate the left hand music accurately and in time with the actual notes in Maya, I got the help of Valentino's girlfriend, who is also called Alex and who plays the piano.  She worked out the notes for both the left hand and right hand and wrote out each of the notes on the actual sheet music, as pictured below.

Essentially, these four lines can be split into two parts; the top line is showing the notes that should be played by the right hand and the second line is showing the notes that should be played by the left hand at the same time as the right hand notes.  Equally, the third and fourth line represent the next set of bars in this piece of music.  The third line shows what notes the right hand should play and the fourth line show what notes should be played at the same time.

So for example, for the first two bars of D through to B flat, the keys F and C should be animated to play at the same time as each of the notes.  For the second lot of bars, the notes of C and G must be animated.  The second line of notes must play at exactly the same time and tempo as the first line of notes.

In order to do this, I animated the keys for F and C eight times, followed by the keys of C and G and then B flat and F so that they were roughly in time with the progression of D through to B flat and then D through to A.

In order to make sure that these notes were animated in sync, I decided to tweak the animation using the Dope Sheet.  Firstly, I gave all of the keys that needed to be animated meaningful names (in other words, the names of the notes that they represent), so I could see where each left hand note was falling in relation to the right hand note.

For example, when I clicked on the keys for D and F, the first two notes of the piece and studied the frames on the dope sheet, it was apparent that they were out of sync with one another.

Each downward movement of a key is represented by three frames, two in which the key is in the upward position and one in between them in which the key is pressed down.  All three frames for the notes of F and D, had to be aligned.  The same is true for the next two sets of notes in the bar, C and E and so on.

Here is how the notes should look on the dope sheet.  Completely in line with one another.

I went through each set of notes in each bar until the left hand keys and right hand keys were in sync with the soundtrack, as illustrated in this video below.

The next layer of animation for this shot was the movement of the hands.  The keys had been synced up with the notes in this shot, now I needed to animate the hands so that they were synced with the movement of the keys.

As you see in the video reference, the left hand is more of a straight arc movement, whereas the right hand has a slower arc movement in the wrist, alongside a more linear movement in the fingers.  The movement of the fingers was done in much the same way as the keys; as each of the fingers has its own individual channel for driving the bending, I could keyframe the bend of the fingers in time with the key movement.  Each key frame of a finger going down needed to fall with the key frame of a key going down.

Due to practical reasons, such as the distance at which the fingers could actually spread, I had to have a slight jump movement in between the D which is played with the thumb and the F, G and A which are played with the next three fingers.  I could sort this out simply by increasing the rotation on the X axis and then smoothening it out to create a natural arc.  

I had to create this animation manually (i.e. without cycling the curves), as I needed to keep the finger movement in time and with the music and key movement.  

As for the left hand, this was very much a single rotation back and forth of the wrist, which is accompanied either by an increase in the thumb tap or an increase in the Pinky channel.  This too had to be aligned with the keys that were being played.

I had set a key frame of the wrist which falls in between each key of the bar which did slow the wrist movement and make it stiff, so i simply deleted the rotation channels for these keyframes (but kept the channels for the fingers as these needed to maintain an up and down movement).  I also flattened everything out to create a smooth movement in between each keyframe.

After playing two bars of F and C, she switches to playing two bars of C and G, and then ultimately playing two bars of B flat and F.  In order for her to make this transition, I simply key framed the elbow control to rotate and move the wrist to the correct set of notes.  I also keyframed and extended the finger spread for a small amount of frames to make it look as though her fingers are stretching to move to each set of notes, as they do in the reference video.

To tidy up the hand movement, I went through each note in to the graph editor to check the rotate X channel of the relevant keys was aligned with the relevant finger channel.  For example, this image displays the note of F being played by the ring finger.

In the graph editor, I can see that the ring finger comes down slightly before the F key comes down.  Therefore, the timing is slightly off.  I resolved this by moving the keyframe in the Pinky channel (which is in white on the graph editor) forward so that it was perfectly aligned with the keyframe of the F key coming down (which is red on the graph editor).

In this image, the Pinky and the Rotate X channels are now perfectly aligned.  I can tell that the pinky will come down and tap on the F key when it goes down and that both the finger and key will be in time with the note of F in the soundtrack.

I went through each note on both hands a I did this check until all of the finger movements were aligned with the respective notes.  Here is the final outcome of this piece of animation.

The camera shot changes at around frame 163 from a POV to a mid to close up shot that slowly zooms in.  This shows Alex playing the piano; I simply added a smiling face with some head movement to show her concentrating on the piano playing and the colours that erupt around her.  I also added some slow shoulder and elbow movement to maintain that she is playing this piano.   

This entire piece of animation took me three days straight; there was a huge amount of trial and error involved, along with constantly having to redo animation from scratch but I eventually got it fixed.  This is one of the shots that I am most pleased with in this film as it had taken the longest to do, but I also learnt a new way of animating.

Shot 11 - The Canvas Painting

In this next shot which I just finished animating, Alex is painting a picture of a parrot and the room fills up with colours and patterns.  For this shot, I also went through the camera movement with Valentino and we came up with a long shot which pans around Alex and eventually zooms into a mid shot of Alex, which in turn zooms in to an extreme close-up of Alex's eye.

This particular shot, I was very anxious about animating as I knew that I had to get it perfect, because her movement needed to be free and natural when she was painting. When painting, she needed to have very free, loose and organic movement.  In order to work out what kind of brush strokes she could do and also to work out the speed, angle and movement of the camera.

As you will see in this video, I directed Valentino to mimic a wide range of brush strokes from very precise, small brush strokes to big and free brush strokes.  I then used the movement that Valentino makes in this piece to inform the movement and the range of brush strokes that Alex makes when doing her canvas painting.

For the planning of this animation, there was not very much that I could do as far as working out each pose, but I decided to be somewhat spontaneous with the animating of her painting.  I decided to use several of the brush movements that Valentino illustrates in this reference video; but I largely chose to be free and improvisational with the movement that Alex displays, much like an abstract expressionist painter would be with their work.

Once I got the right arm movement in, I needed to animated the fully body movement around this right arm movement.  At first, I made it very exaggerated and included a huge amount of hip sway, but this made it look ridiculous and as though she was dancing, so I toned it down a bit more and here is the final result.

I managed to finish this piece of animation in the space of a few nights.  I actually thought it would take much longer and that I would struggle with it, but I decided to approach this as creatively as possible, so I improvised much of the animation work up to the point that she looks up to the camera.  I think this  approach definitely helped to express a freedom of movement in the character, while she is painting.

Another last minute job for the Technical folder

For the piano shot, which I am currently animating, the Alex rig needed full freedom of movement in her fingers so she could stretch to her fingers to reach certain notes.  Initially, the fingers could bend, but they could not spread out.  Equally, the thumb could bend in but could not tap.  To resolve this I quickly created two drivers in each of the wrist controls, Thumb Tap and Finger Spread.  These attributes were set to drive the rotation of the joints in the fingers.

Dying dog scene

I have just finished animating the scene in which Alex approaches her dog and strokes the dog, before the dog dies.  The animation of the dog here is very minimal as the dog is unwell and slowly dying.  One of the requirements of this scene was for the dog to fade away as she dies; I created this effect simply by keyframing the transparency of the lambert that contained the dog's texture.

For the POV walk, we were originally keyframing the rotation of the camera in order to create a bounce and the feeling of a handheld camera shot, but it looked terrible, so we settled with a smooth zooming in shot instead.  The only way we could really produce a convincing handheld shot with CGI is if we had technology much like The Third Floor use for their pre-visualisation.

There is now a close-up shot of Alex, which conveys the transition to a sad, worried expression, when she realises something is wrong.  It was my idea to include this camera shot as I felt that it would allow the audience to make a connection with her when she realises, and eventually we realise that the dog is dying.

This shot then ends with a walk cycle in which Alex walks away from the dying dog and into the piano room of shot 8.  I had to make sure that her right foot is forward and her left foot is back for the final pose of this shot, to match with the first pose of shot 8.  Here is a playblast of the finished Dying dog scene.

Last minute rigging

For the dying dog scene in Synaesthesia, we were going to use a dog that we downloaded from the internet.  However, while we were testing this dog rig, we found that it was a very poor rig; it was badly weighted, much of the history of the mesh was still present and it had no blend shapes.  We could not put in any blend shapes because of these problems.  As it would take more work just to tidy up this problematic rig, we decided to use a dog that Perri Wheeler modelled and to rig this ourselves.

As we have very little time left and this dog has very little movement, given that she is dying, I made the rig as simple as possible, with only a few drivers put in.  I rigged this dog but I received help from Arpit with the weighting and the parenting of the master controller to the joints, controls and IKs.  This should be the last rig that I make for my Portfolio and indeed this year.