With full-process 3D editing, over one thousand VFX shots, and Dolby Atmos® sound, Director Cheang Pou-soi’s 3D fantasy action feature filmThe Monkey King 2 was the most anticipated film release in China for 2016.

Released in Mandarin in February, the full-length feature film, a sequel toThe Monkey King, presented an unprecedented visual and sound experience achieved through the collaboration of industry artists from around the globe.

Under the supervision of Director of Post-Production Angie Lam the film’s post-production was assigned to professional creative teams in China, Korea, the U.S.A., Mexico and Australia. Working effectively across borders presented Ms. Lam with significant, thought-provoking challenges across the entire post-production system.

Working seamlessly with VFX, grading and multi-department coordination

Principal photography commenced in November 2014, primarily using RED Dragon and Epic cameras.

On-set editing was finished on three workstations each installed with Media Composer 7.0. Firstly, the original R3D material was converted into DNxHD 36 in RedCine software. Next, the autosync and stereoscopic grouped within Media Composer, and then the rough cut was taken as a reference for the director.

For in-house editorial, Lam had seven workstations running Media Composer, and chose DNxHD 36 with the understanding that cutting the film would be a lengthy process and the “journey of combat” would generate a vast number of mutual files. Despite its small file size, the advantage of DNxHD was that the resolution also qualified in 3D mode.

“This project was special in that it was trans-regional and we had to move 6 TB of material on a portable hard drive between Beijing, Hong Kong and Mexico,” Lam said.

Thanks to Media Composer, which made synchronization possible, all of the materials were stored in an Avid MediaFiles directory, allowing chronological classification of footage by generating a subfolder such as 20150211, 20150212. Each day Lam’s assistant would send the newly generated subfolder in the main workstation to other assistants in different regions. As a result of the database structure, the workstation would automatically link to the material by simply opening the .AVB file. The entire process was very intuitive and convenient.

Lam explains, “The material management, based on the Media Composer database, diminished many other types of software I had ever used. Whether to transfer the material to another workstation, change the drive letter, or even a system crash, there was no need to worry about the offline issue.”

Lam attached great importance to keeping the material library “clean” and requested her assistants clean the bins regularly. Media Composer provided media management features like searching, archiving, filtering, and deleting, which fully met Lam’s expectation of a “clean” library and helped the team manage the file size and keep it under 6 TB.

Working with ten sets of 3D cameras supplied by China-based CPG (Cameron Pace Group), The Monkey King 2 delivered a vast amount of footage to the post-production department. Lam quickly discovered that a series of 3D toolsets in Media Composer were capable of facilitating a very smooth transition from 2D mode to 3D mode, including logging, editing and management of a complete 3D end-to-end workflow at rough cut stage.

To begin, Lam’s assistant grouped all the left-eye and right-eye material respectively, which were rough cut in 2D mode. This was then replayed in 3D mode for fine-tuning because 3D images stimulate strong reactions that may require modification.

According to Lam’s experience, because of the difference in the focal point, especially for the action movies, the rhythm between 2D and 3D was different. For example, a 3-second-long piece of footage in 2D needed to be converted into 3 seconds plus 10 frames for 3D footage to avoid viewer discomfort. With this knowledge, Lam insisted on recording both the left-eye and right-eye material and editing in 3D mode. In this way, despite longer preparation than 2D mode, it was easier for the editor to spot and solve any issues, such as the asymmetry between left-eye and right-eye, as early as possible, which significantly alleviated the pressure on the post-production department.

Media Composer made editing VFX footage very efficient

The Monkey King 2 produced 1,500 VFXs, which made up 70% of the entire footage. Six VFX companies participated in the production, and more than half of the visual effects were completed in Korea.

Before sending VFX footage to other companies, Lam’s team did a temp-comp Media Composer timeline in order to give the director an intuitive feeling. The production process involved a lot of high-resolution images and material and Media Composer’s multi-resolution mixing capacity meant all the material on the timeline could coexist with ease.

VFX footage was initially delivered as 1K H.264 .mov files, which could be placed into the Media Composer timeline after transcoding. Since these files are quite small, a lot of downloading or uploading time was saved, allowing fast feedback and making it convenient for VFX companies to make their modifications.

Once 80% of the work was completed, the format was switched to 2K ProRes, which helped the director check consistency. When 90% was done, EXR high dynamic files were used to check the effects quality for bad frames or frame numbering errors. Once confirmed, the EXR files were converted back to 2K ProRes format.

As one of the native formats in Media Composer, the performance of the ProRes editing files was excellent and the operational experience was as good as DNxHD.

In order to best manage visual effects files, Lam preferred to name the file in the order of “scene” and “take” and each effect received an ID, facilitating improved communication. The final outcome was marked with version tags, such as v1, v2. One particular effect took out the prize as most modified before being finalized in version 24. Although that effect was modified the most times, repeatedly, the fight scene of the White-Boned Demon, whilst tagged as version 18, was, in fact, modified internally more than 50 times!

With so many companies involved in effects production, for color modification, Lam assigned color space and LUT’s to unify color management. Media Composer not only supported DCI P3 color space, but also supported importing external LUT’s to unify the whole tone of the timeline or materials, which made the effects department see the same color on the timeline during screening.

The audio production of The Monkey King 2 was carefully divided amongst three places. Soundfirm in Beijing was partly responsible for effects creation, ambient sound and foley (except footsteps). The independent studio of the film’s main sound designer, Yin Jie, was responsible for dialogue, premix, foley (except footsteps and fight sounds) and Soundfirm in Australia was responsible for part of the effects of the production, foley for footsteps, monster sound, and the final mix.

The three companies cooperated seamlessly with each other primarily through their use of Pro Tools. Working from one dedicated network, session files were uploaded in pre-production and they could then achieve synchronization via subsequent updated sessions with all material stored in an audio files folder. Files generated by effects forming or speed variation had to be transferred separately, but the transfer speed was very fast. After relink, everyone’s work was clear. Once they had the reference track, they could each learn about the other’s progress and requirements.

During the middle of the production, Yin Jie’s work was mainly shifted to one audio company located in Melbourne, where the local practice was to upload all the engineering files to the central server to allow file-sharing by the local area network shared storage. In this way, even with a different workstation there was access to the latest real-time progress of the project. To review the situation from the previous day was a matter of clicking on the “Reference Track and Import Session Data,” which would show how much work other colleagues had done.

Yin Jie used Pro Tools 11 and was greatly impressed by the Offline Bounce feature, particularly for collaboration. For example, when the action scenes needed to unify sound tones, one could use group clip or share all the stems with workmates. The Monkey King 2 audio team built a Library folder that contained all the sound tones designed over many months, so the team could conveniently retrieve sound tones.

Clip Gain was another frequently used function. Upon starting Dialogue Pre-Mix, Yin Jie hoped that every fader stayed at the zero position, so he used Clip Gain to adjust all the level modifications during the editing process when needed. With the exception of several scenes that used production dialogue, such as some emotional scenes of Tang Monk, which used noise reduction RX, most scenes used ADR. For lipsync, the team used Pro Tools Elastic Audio feature extensively, making telescopic adjustment based on actors’ mouth shape—little by little—and assigning a dubbing director to provide on-the-spot guidance for coordination with emotional scenes. In addition, during the ADR edit, the team still took the sound bite as reference. Although the script had been adjusted many times later, and much dialogue had been changed, fortunately, all the tracks had backup.

The Pro Tools | S3 control surface was used for premixing of the Cantonese version with Yin Jie liking the VCA Spill and Flip function very much. In sound design and premix, Yin Jie used C42 with his own workflow. Taking the mark point as a core, allowed flexible recall of preset orbit display and the facilitation of track management.

Yin Jie’s team mixed natively in Atmos® and then output 5.1 and 7.1 versions through down mix, so they did some 3D sound preparation during the editing process. This included placing some sound, which needed to be done as objects, onto the object track, separately, and adjusting the acoustic images of those objects during the final mix. Thanks to the excellent compatibility between Pro Tools and Dolby Atmos Panner AAX plugin, the Dolby rendering and RMU worked beautifully together, which gave the audio mixer easy access to Dolby Atmos plugin via Pro Tools to manage objects and bed tracks. The plugin interface directly linked with RMU via the Internet and all of the sound and image editing was done within the plugin. When the team wanted to modify the final mixed Dolby Atmos, they simply opened the Project files in Pro Tools and adjusted the Atmos plugin.

“The complexity of Dolby Atmos was self-evident but it also expanded the creation space from 2D to 3D,” enthused Yin.

Hollywood-based composer Christopher Young noted there are some differences in the definition and function of movie background music (BGM) between China and Hollywood.

For example, compared with Hollywood's style, China pays more attention on the melody of the BGM while leaving flexible room for the match between music and image movement effects, leaving a lot of room for creativity.

As in the past, Young continues to work on The Monkey King 2 with Sibelius software. Young composes first with pen and paper, and then Music Editor Ben Schor records the music into Sibelius. With the Sibelius interface being highly intuitive, from composing and editing to playing, the operation was quite quick and simple.

Young explains that he likes to pay close attention to whether the tones of instruments match the characters and the emotions of the scenes. For example, when considering how to display the sadness and resentment of the Monkey King the melancholy and beautiful sound of the Erhu stood out. Chinese musical instruments had already been widely used during the prequel production of The Monkey King, and that style was carried into the sequel. However, Young did try other ways in the second episode such as trio and ensemble. The movie also has a variety of interesting musical languages and young recommends one scene when the White-Boned Demon, appears where Young performed something similar to Mongolia Khoomei-style singing.

During BGM production the parts to be played by instruments were marked by a unique symbol in Sibelius, which was embedded into Pro Tools, giving the team a clear picture of the entire progress. Young would display a piece of BGM in a symphony structure by using several folk instruments such as five Chinese lutes and six zithers. Recorded engineering files and dozens of sub-tracks were then sent by Schor to the sound editors and mixing department for the next phase of work. Working with Pro Tools, provided a common platform, which overcame many challenges for Schor and facilitated smooth communication amongst the team.

The seamless audio production workflow brought by Pro Tools gave every musician in each stage of production opportunities to further focus on their own creation.

Seamless collaboration of visual and audio production

Usually, audio editing of Project files begins after the initial clip draft has been completed in a movie clip however video production sees continuous revisions beyond that stage. Lam points out that if the platform failed to efficiently coordinate the audio and video production, progress would be slowed down dramatically.

Yin Jie was also impressed by the seamless audio and video collaboration, particularly when he did the premix and final mix, when it allowed him to control the frames on Media Composer using Avid Video Satellite. High-definition frames of Media Composer could be tracked and synchronously played in Pro Tools. If the storyboard changed, the mixer could refer to Media Composer and modify the audio immediately. During editing, the audio team also used the Avid Mojo DX video card to play the high-definition frames, and the clarity was perfect without any deadlock.

The Monkey King 2 was updated almost every day during post-editing. To audio adjust and synchronize in time Yin Jie would send an updated Cue-track to Lam, which would be returned in AAF format after editing. By simply opening the AAF file and comparing it to the previous Cue-track, the related modification would be displayed in obvious visual forms immediately through waveform and editing points. He could even bring the audio plugin on the Media Composer timeline into the file and when the audio team opened the AAF file they would know, “Oh, the director needs to add a reverberation here,” thus reducing communication costs for the post-production editing staff. To date, only .AAF files generated by Media Composer to Pro Tools can achieve this.

In the future, with Avid Everywhere, the MediaCentral platform and Cloud Collaboration, cross-regional and multi-department cooperation will become even more automated, enabling Chinese movie producers to work more closely and seamlessly with international talent.