Video games with Hollywood style visuals are fast becoming the norm, and like blockbuster films, can now command top acting talent for voiceover and performance capture work. One of the biggest such endeavours is Star Citizen from Cloud Imperium Games (CIG) and Foundry 42, created by legendary game developer and film producer, Chris Roberts.

This massive multiplayer online title, which also aims to encompass first-person shooter, space trading and combat simulator elements, is a mammoth creative and production effort on a galactic scale. Key elements include thrilling space combat in a massive play area, first person battles, planetary exploration and the ability to fly and travel in multi-crew ships—with the participation of 1 million-plus “citizens.”

Recognized by Guinness World Records as the top crowd funded game, and incidentally the top crowd funded project, in the world, it has gathered more than $100 million to allow it to command the best actors, and the best technology, to further the development of its sci-fi simulation.

The latest development, Star Citizen: Squadron 42, is a single-player campaign that takes place within the Star Citizen universe. With ground-breaking cinematic techniques and best-in-the-industry game design, major stars such as Gary Oldman, Mark Hamill, Gillian Anderson and Andy Serkis have provided performances and voices for CG characters in the mission driven game. 

A fully equipped development team, based at the Foundry 42 studio in Manchester, is responsible for creating this new experience for the Star Citizen gamer, which has a production pipeline that shares much with a Hollywood movie. Top of the line editing of cinematic visuals, story elements and pro-level audio are enabled by a collaborative workflow that embraced Avid Everywhere and is powered by the Avid MediaCentral platform.

Key elements of the Avid Artist Suite, such as Avid Media Composer and Avid Pro Tools, combined with Avid Storage Suite components such as the powerful Avid NEXIS, have been deployed across a network which spans four production studios, in California, Texas, Germany and the UK. Together they bring the next iteration of Roberts’ first person universe to life.

Squad Leaders

One of the mainstays of this workflow is film editor Michael Freedman, who is responsible for the editing and turnover of every actor that was performance captured for Squadron 42. This includes not only all the cinematic cut scenes in the game that play like a movie and don't have player options, but also all the gameplay scenes and vignettes which do allow player interaction.

“It’s an incredibly exciting time to be working in games, because gaming is now the new frontier of the entertainment industry,” says Freedman. “Advances in technology are forever speeding up, we are entering an era where only our creative ideas are going to limit us in what is achievable. The story lines can be just as compelling as any film or TV show and with video game graphics now so photorealistic, soon enough we won’t be able to tell the difference.”

Another key team member is Bob Rissolo, the dialogue supervisor at CIG. Rissolo manages the dialogue pipeline, taking it from the hands of the writers to intelligently speaking in-game. “I help the creative vision come to life, through dialogue,” he says. “Sound can be described as a landscape, but sound for games is like trying to design and mix an ever-changing landscape.”

“The biggest challenge for Squadron 42 is the sheer size and scope of the project,” he adds. “With a whole persistent universe to populate, we need to engineer creative new ways to manage a dialogue project of this magnitude.”

Suiting up for Battle

Freedman, used to editing indie films, had also worked on Avengers: Age of Ultron for Andy Serkis' company The Imaginarium Studios as their on-set editor, which is how he came to be introduced to Chris Roberts, and performance capture producer John Schimmel for performance capture on the Star Citizen title.

“I'd worked for The Imaginarium Studios as their on-set Avid editor during The Avengers: Age of Ultron shoot at Shepperton Studios,'' recalls Freedman. “I was capturing takes into Avid Media Composer, syncing dailies and turning over to the production’s VFX editor. When Squadron 42 came to Imaginarium, it was nice to be remembered by the studio and to have them refer me to Cloud Imperium Games and Foundry 42.”

At the beginning of the Squadron 42 shoot, Freedman thought his role was going to be more technical and similar to what he had been doing on The Avengers, but it would prove to be much more involved. “In performance capture, just as in traditional filming, the actors performing in a scene are captured together. The significant difference however is that the director has the ability to use the actor's performances in whichever way he/she likes.”

At Andy Serkis’ studio, each actor's body, face and dialogue performances were captured using different systems. That generated an enormous amount of data which then had to be synced together on Avid during editorial to re-create their full performances. Production audio records the actors dialogue via LAVs, DPAs and a boom, and is delivered to editorial as BWAV files, also sharing the house time code which is used for syncing. “With the sheer magnitude of content both for dialogue and animation, we needed our pipeline to run smoothly and efficiently,” says Bob Rissolo.

“Michael Freedman works in Avid Media Composer to line-up footage and edit each scene, interstitial, and wild line. These edits are passed along to animation and myself where I can then edit and clean the audio within the Avid session with Pro Tools. I then export dialogue that retains the same timecode that our animators need to sync audio with animation in our Maya toolset. Maintaining this sync is crucial to our immense project and pipeline. It is a huge benefit that our editor and I are sharing the same Avid toolset!”

“As the shoot grew more demanding, Chris felt that it would be useful to have me also cut the scenes based on my creative eye and his on-set preferences, which we would then review in sessions later down the line,” continues Freedman. “It was a very exciting prospect, knowing that I was going to have a hand in crafting performances and cutting scenes with actors like Gary Oldman and Mark Hamill, actors that I have admired as a film lover for years.”

“I love listening to my heroes bring our script to life on-set and in the VO booth,” says Rissolo. “Set life is intense, but some of the most rewarding work. The crew becomes your family, and the talent your honored guests.”

Mission—Critical Kit

Freedman reveals the editing process to be similar to that found on movie or a TV show. “My edit suite, along with my assistant, were based in a building just moments from the stage,” he says. “We would start to receive each day's rushes the morning after, along with our continuity notes, animation reports and marked up script. My assistant would then sync rushes and have them in my edits bin by the afternoon. I would watch them through, start to assemble the scenes and have them ready for review sessions with Chris and our lead writer, Dave Haddock.”

Freedman continues, “In our edit sessions, we would watch my edits through and I would take their notes, usually making changes on the fly and signing off scenes that day. As the edits were signed off, my assistant would then access those edits from my Avid and copy them into his Avid for turn over prep. It really was a seamless workflow.”

“Avid Media Composer can separate its media with separate folders in the Avid Media Files folder, which helped us with our day to day ingesting and logging of face camera MXF media,” adds Freedman. “Each actor’s face is shot separately via their own head cam, which is labelled as a color by the Imaginarium Studios, and so in order to keep our media logged neatly at Finder level for ingest into Avid, I found it useful in being able to log each actor’s rushes into separate folders.”

The other challenge with having multiple option scenes arises during turnover. The game contains hundreds of scenes which, unlike in a movie, all get turned over individually. Freedman calculates that so far his team have turned over something in the region of 14,500 files!

“Editorial has three main vendors that we turn over to,” says Freedman. “The Imaginarium Studios, who are dealing with body solves, Cubic Motion who are dealing with facial animation and CIG audio for dialogue clean up. On average, each scene would include three or four segments, each with around a similar number of characters who are all edited separately from one another on different video tracks, and therefore, all need their body and face edits turned over separately. Per scene, we could end up in the region of 65 files to turnover, including QuickTime movies, EDLs and AAFs.”

Avid Media Composer’s versatile ‘Find’ tool proved very useful on a day-to-day basis. Freedman explains, “The ability to add metadata, such as scene descriptions to the bin columns, made it easy to find scenes based on keywords rather than scene codes. Scene names that came from Imaginarium would eventually be renamed to a special cataloguing system created by Bob Rissolo in order to make it easy for the animation and audio teams further down the line.”

“Cataloguing was a huge challenge for this project.” exclaims Rissolo. “The production names for Imaginarium had to be different. It was important to keep the flow of production, so at times, Chris would shoot multiple scenes together as one. Plus scenes already included multiple edit points due to branching, gameplay triggers, loop animations waiting for player interaction, etc. This created a lot of work for Michael as he later had to breakout each of these segments and provide them with their proper in-game names.”

“Each scene holds many segments that are all turned over separately, so with thousands of different segments, it’s imperative that I’m able to find media quickly and without fuss.” says Freedman. “In essence, my editing process doesn't involve editing for shots, but for performances and for the overall rhythm and pacing in every scene,” he adds.

“My edits become the blueprints that the animation teams build onto. Cutting for performance and pacing still means stitching together various takes to craft the best performances as you would in a movie,” he explains. “However, unlike in a movie where an editor would cut to another shot to hide a cut point, I leave the cut points bare, making one continuous performance that can be viewed from any angle—much like a scene from a stage play. The cut points are then blended out later during animation to create a seamless effect.”

Avid Media Composer’s powerful picture-in-picture effect allowed Freedman to easily work within this complex pipeline. “I found the ability to assign different colored borders to each character an effective way to work out who I was viewing at any one time, while making sure that the scene was flowing smoothly,” he says.

“It wouldn't be uncommon to use 14,16, or even 18 video tracks when we had numerous actors in a scene, so Avid's ability to utilize 64-bit processor power also helped the scene play out without any jagged payback,” said Freedman. “The ability to now disable tracks was also very useful when I only wanted to playback one or two actor tracks which had video tracks sandwiched between them. Also with the number of titles I was creating, being able to save and apply titles straight to the timeline was also a handy feature.”

“Without these features in Media Composer I wouldn't have been able to conduct this type of editing so easily,” he adds. “If someone was to ask me what is the most beneficial thing they could do to get a job when starting out in the industry, I would tell them—learn Avid!”

Storage lockers

With Cloud Imperium's animation studios based between Los Angeles, Austin, Frankfurt and Manchester—and director Chris Roberts flying between all of them—Freedman wanted an easy way to share edits between the different studios.

“I set up Avid suites in Los Angeles and Manchester, with my main suite located in London's Soho. With all of this material to ingest, log, sync, edit and export—it is imperative that the NLE we are using has a rock-solid approach to file and media management. Knowing that Avid Media Composer has paved a path on heavyweight productions like this before, I felt confident using it to see this project through,” Freedman said.

“Each Media Composer was connected to a Synology NAS, while my assistant and I shared an Avid NEXIS storage with 32TB of storage. As I would update edits and wanted to get feedback, Avid made this process incredibly easy with Media Composer’s unique ability to share bins separately from the project file. I would upload bins via our FTP, download them through team viewer at the other end, add them to the Avid project's document, and boom—there they were online and ready to view inside Avid! No hassle.”

“I don't know how I would have been able to work so seamlessly using any other software,” adds Freedman. “It seems only logical that bins are separate to the project files for easy collaboration and in the event that a project file gets corrupt, the rest of your work isn't contained inside. The ease in sharing edits and clips between my assistant and myself was also helped through this system, and also by the ability of Avid storage to quickly show what bins are open and locked, or closed and ready to open by another user at any one time.”

A universe of opportunity

In his time as an editor Freedman has seen massive changes across the entertainment industry. “Avid has been aware of this and has tried to help editors in terms of what is required from increasingly more complex workflows” he observes.

”It seems productions are shooting more and more footage these days, which makes daily media management, logging, syncing and editing a more demanding task; consequently editors and assistants are finding they are having to manage their time more creatively. However, with the introduction of several time-saving features by Avid, including AMA editing, background transcoding, background rendering, the ability to edit in an assortment of aspect ratios, the evolution of the DNX codec and the ability to edit with ProRes, the introduction of frame flex and more, means that editors are able to take on more challenging day-to-day requests from their clients.”

“Each year brings new challenges and we continue to find ways to push the envelope,” agrees Rissolo. “But it’s only through good tools, plugins, and pipelines that we can achieve the goals we set. Thank you Avid for being such a vital integral part of our pipeline solution.”

“It's incredible to know that whilst we are editing there are literally millions of fans out there waiting to play this game,” concludes Freedman. “It's a very exciting feeling to know that so many people will see your work. Overall it's been an incredible journey and one that's not over yet!”