Revolutionizing motion tracking planning for CGI/VFX with Artec Ray II and TVA

An Artec Ray II in a simplified setup capturing a large mocap space.

Before we dive into this new workflow, let’s clarify a few things. What do we mean by a motion capture stage? These are essentially rooms packed with cameras that track the movement of markers attached to people or objects (you’ll likely have seen them in “behind the scenes” clips of TV shows or movies).

Industry-leading motion capture technology company OptiTrack markets advanced optical tracking cameras renowned for their high accuracy and low latency, alongside a robust software ecosystem. Its technology is used in biomechanics, medical research, and the creation of virtual production sets that allow directors to merge live action with immersive visual effects using giant LED screens – think of scenes like superhero showdowns in Marvel movies!

Advances in CGI & VFX have changed cinema in recent years, but it’s not easy to set up a motion capture stage. You need to ensure enough cameras can see the markers, even when sets and props are in the way. Currently, it relies quite heavily on a technician’s experience with a specific tracking camera. Failure to properly lay out your capture stage leads to camera occlusion, preventing tracking and requiring a shoot to halt while camera positions are adjusted.

When you consider the cost and lead time penalties incurred by keeping actors waiting while crew try to adjust the set, it’s clear that ensuring consistent, reliable results is crucial. It’s also vital to be able to easily visualize and communicate the impact environmental or stage changes have on tracking.

That’s why Tracklab, the tracking department of Artec Gold-certified reseller Freedspace developed TVA as a way of simplifying volume planning, making it agnostic to a user’s experience, and getting the most out of shoot time – by optimizing tracking coverage.

Merging 3D scanning & motion tracking

Freedspace is uniquely well-positioned to make TVA a success. The company has three departments: Thinglab for 3D printing, Scanlab, which sells and offers services with Artec 3D scanners, and Tracklab, its dedicated motion capture solutions business.

According to Richard Burt, Freedspace’s director of 3D tracking, there are “cross-pollination” opportunities between these technologies. The latest example of this has seen the Artec Ray II long-range LiDAR scanner used to digitize motion capture spaces for optimization in TVA.

Optimizing a motion capture stage in TVA is straightforward, with or without 3D scanning. But importing a 3D scan of your stage takes enhanced tracking to the next level.

Digitizing motion capture spaces with Ray II is simple. The device can be mounted to a tripod and moved to different locations around the room. “While Ray II is powerful enough to capture the entire volume in a single sweep, capturing it from multiple locations ensures maximum data density across the environment,” explains Richard.

However, it’s also important to ensure high accuracy – and the built-in Visual Inertial System (VIS) pre-aligns scans for seamless post-processing. Artec Studio, Artec 3D’s professional scan capture and processing software, then makes quick work of data fusion, tidy-up, and export.

Processing 3D scans of a mocap room in Artec Studio.

“The Ray II does a great job of capturing color information too, not just geometry,” says Richard. “And, there are multiple ways to capture colored textures. This allows us to bring a realistic colored model into TVA, which enhances the project documentation quality and makes it easier for others to quickly understand what is being shared.”

“We could potentially expand these 3D scanning applications further,” he added. “By seamlessly integrating scans of people or specific high-fidelity props captured with Artec Leo or Artec Spider II. Furthermore, scanning large-scale physical assets – such as production vehicles – allows us to fully visualize the tracking volume in advance, completely removing setup guesswork.”

Mocap stage optimization in TVA

Once 3D scan data is processed in Artec Studio, it’s imported into TVA for planning and analysis. Freedspace’s software has all the tools needed to assess tracking for individual marker sizes and different OptiTrack marker types. Inside TVA, individual positions can be turned on/off, so different layouts can be tested in the virtual world ahead of film shoots.

Individual camera customization and aiming tools in TVA for volume optimization.

It’s worth bearing in mind that getting high-quality data depends entirely on camera setup. If cameras are placed incorrectly, you get “dead zones” where motion isn’t captured, and people are blocked from view. As Richard explains, these issues can have a real impact on the way scenes pan out, as positioning ensures smooth tracking and calculating volume boosts predictability.

“For instance, if we’re setting up in a cave environment where physical rock formations block camera sightlines, scanning those assets allows us to know their exact dimensions and positions,” says Richard. “Placing them precisely within the virtual volume takes all the guesswork out for virtual production. Or, when performers carry out dynamic movements, such as jumping from elevated structures, it’s incredibly valuable to understand a set piece’s height to accurately predict where cameras must be positioned.”

Ultimately, using Ray II data and TVA’s volumetric heat mapping, it’s possible to work out whether cameras were positioned well, with green indicating good coverage and red showing the opposite. Of course, the software comes with everything necessary to build “virtual sets” from scratch. But importing a calibration file directly from OptiTrack’s Motive software offers absolute certainty with your current setup. In fact, Motive’s continuous calibration and robust tracking algorithms work seamlessly with TVA to ensure uninterrupted data capture.

Volume Heatmap Tools in TVA provide a precise understanding of the OptiTrack camera coverage across a volume.

Expanding into other tracking areas

Richard sees huge potential for this cross-over between 3D scanning and TVA optimization, not just in traditional CGI & VFX, but the wider manufacturing space. When shooting a video game fight scene, for example, it’s important to know the size and position of props like tables that might interfere with character movement. Leo or Spider II would be ideal for capturing these.

The same can be said for robotics. As Richard explains, industrial environments and factory floors are highly dynamic – and frequently filled with moving pallets, containers, and staging equipment. Simulating these environments with TVA allows users to optimize camera layouts, making sure robots move unobstructed with a low risk of occlusion.

“I anticipate that many customers using Artec 3D scanners could benefit from TVA integration with OptiTrack systems, and vice versa,” concludes Richard. “Our users span video game development, virtual production, and animation, alongside aerospace, advanced manufacturing, robotics, and more. For any business looking to digitize and optimize motion capture volumes, TVA makes it possible.”

Find out more about the TVA (Tracklab Volume App) here.