Behind the scenes: Capturing the Grand Ducal Palace in Luxembourg for the World Expo

When Artec 3D’s support team got the call to digitize Luxembourg for the World Expo, they were delighted. Push 3D scanning to its limits and put your country on the map? What an opportunity! But there were a few hiccups, as they sought out the fastest, most straightforward method.

Initially, they tried digitizing a sugar model of the palace with the high-resolution Artec Spider II. While it featured a very realistic appearance (and Spider II captured its nuances brilliantly) the replica didn’t have the geometric accuracy necessary to create a true one-to-one 3D model.

Artec Spider II capturing a sugar model of the Grand Ducal Palace in Luxembourg.

When their shortcut didn’t work, the Artec team were left with no choice: they’d have to go and scan the Grand Duke’s Palace itself. This came with its own challenges. Artec Ray II is fantastic at capturing large objects and areas. However, its tripod-mounting limits what it can “see” from above. Fortunately, Artec Studio now features drone-compatible photogrammetry.

The next problem was how to get a drone in the air around a busy heritage site. This meant visiting the Palace’s Premier Conseiller, who managed to get permission for the flight. Lastly, there’s the issue of pedestrians. The popular tourist attraction is often swarmed with people. In 3D data capture, this is bad news as it can lead to obstructed details and motion blur.

But thanks to a bit of cautious double-scanning, Ray II’s built-in Visual Inertial System (VIS), and advanced algorithms inside Artec Studio, the team were ultimately able to capture an incredibly large yet detailed 3D model, with a realistic appearance for immersive VR applications.

Straightforward scanning, exceptional results

According to Keynan Tenenboim, one of the 3D scanning specialists behind the project, Ray II data capture wasn’t actually that difficult. Its VIS system allows it to track its location in a 3D space, meaning users only need to position the device for capture from all angles. “My job in the field was basically to move the scanner around and press a button,” said Tenenboim. “There was no other skill involved. I could have given this job to any person on the street!”

The real challenge was scanning from a sufficient distance to capture at scale. Ray II can achieve an accuracy of up to 1.9 mm from a range of ten meters. But if Tenenboim got too close, some parts of the palace’s architecture would obscure others. This meant taking a step back – to a range of around 20 meters, so the whole structure was in view, and making a loop around the complex.

The final 3D model of the Grand Duke’s Palace.

While Tenenboim was scanning with Ray II, his colleague Jerry Klein was piloting a drone over the area. Though he had permission to fly, this would be no easy feat. The palace is surrounded by a complex network of roads, and any navigational error would’ve been hugely expensive. Yet, despite not being an experienced pilot, Klein managed to capture the scene in its entirety.

“Flying a drone is like playing a video game, it’s nothing special. It’s quite intuitive,” explained Klein. “The advantage of capturing pictures is that you get higher-fidelity imagery and less noise. In Artec Studio, you can also create 3D models from videos – I captured both just to be safe. We could use video to do it faster and pictures for parts that were harder to reconstruct.”

Merging different datasets in Artec Studio

Artec Studio is now an all-in-one toolbox for capturing, processing, and combining 3D datasets picked up with structured-light, laser, and LiDAR scanners, as well as photogrammetry. To merge 3D scans and photos/videos, users first need to generate 3D meshes. When it came to fusing their Ray II data, VIS was again key to the project, streamlining pre-alignment.

“Ray II’s VIS system basically aligns all the scans automatically, so when you upload data to Artec Studio, all of the objects are already pre-aligned,” added Tenenboim. “Instead of seeing 100 disparate scans, you would actually see one object on screen. One of the things we really tried to do was keep track of that system and maintain it as much as possible.”

Artec Ray II outside the “Hollow Tooth” heritage site in Luxembourg.

Generating a 3D mesh with AI Photogrammetry is a little different but still yields excellent results. Artec Studio’s algorithms turn photos or video into a “model preview” with an adjustable bounding box. Once told which captured data to use, the software generates lifelike 3D models in real time, which can be merged with those created from 3D scans for high detail at scale.

Inside Artec Studio, it’s also possible to fill holes, remove moving objects, map textures, and decimate models to reduce the number of polygons. All of this allows users to add polish – and in the case of Artec’s 3D scanning experts, it was critical to creating lightweight models for VR use cases.

Embracing next-gen heritage preservation

Combining Ray II, AI Photogrammetry, and handhelds like the wireless Artec Leo, the team were eventually able to digitize other landmarks for the Luxembourg Pavilion. This includes “Hollow Tooth,” a UNESCO heritage site with a cliff on one side, requiring the use of a drone – and data was again merged with Ray II scans, for a model captured from every angle.

Alongside all the other beautifully captured models, Hollow Tooth is now available to view via the Osaka Expo app, in a way that brings the country’s history to new audiences. In fact, visitors to the expo itself can join those tuning in remotely to “tour” popular heritage sites in VR.

A 3D model of Hollow Tooth, also captured with Ray II & AI Photogrammetry.

Klein sees significant potential for this approach, not just in heritage preservation, but other large-scale applications, where different 3D scanners and photogrammetry can bring users incredibly high accuracy and capture scale: the best of both worlds.

“I can see this approach being applied everywhere from the Hollywood film industry to video gaming, anywhere you traditionally don't care about geometry, it’s more about texture,” concluded Klein. “Since you can combine both 3D scans and photogrammetry, you can get a good geometry and high-fidelity texturing. That’s quite useful, you don’t have to choose!”

Artec 3D is honored to have contributed to the creation of the Luxembourg Pavilion in Osaka.