Case Study - The Matrix Resurrections

KFTV Underwater Filming Report

Part One: Case study of The Matrix Resurrections.

Director Lana Wachowski’s sequel to the Matrix film trilogy broke new ground by performing the infamous bullet time shots ‘underwater’ at the world’s first volumetric underwater studio. Here the production team, including Sven Bliedung, CEO of Volucap, explain to KFTV how they did it.

Wachowski was keen to take things to a whole new level with The Matrix: Resurrections and so decided to experiment with ground-breaking volumetric technology, which digitises a three-dimensional space or object in real-time using an array of cameras. This meant turning to the leaders in the field, Volucap, headquartered at Studio Babelsberg in Germany, where the production team ultimately based itself and did most of the shooting.

Breaking new ground

The experienced team of Wachowski, John Toll, DoP, and Sven Bliedung von der Heide, COO of Volucap, got together to discuss their outlandish plans. “In prep, we discussed using an array of 100 cameras shooting at 100 frames per second,” says Daniele Massaccesi, camera operator for Toll.

In the end they resolved to film the actors in motion using 30 synchronised cameras in 3D at 30 frames per second. Something that had never been done before.

“But not only that, we decided to do the whole thing also underwater,” laughs Sven. “Lana wanted to get a specific look and feel of drag and force of the skin that you get when you move underwater.”

They had just two and a half months to get the underwater studio and equipment ready for the first day of shooting. This was a huge challenge, all the more so because they were experimenting for the first time with everything.

“Initially I thought we could utilize existing camera housings off the shelf for underwater. But after three days of testing, we found there weren’t any suitable because we needed to synchronize all the cameras at the same time, have a live feed from other cameras in 4K resolution, remote control them, and manage overheating with proper ventilation because the cameras run at very high resolution,” explains Sven. That’s before even getting into the different optics and specific lenses required for their cameras.

So, they decided to build their own mobile volumetric camera systems and housings (40 of them in total) with specific measurements for their cameras and lenses, and all aligned using trigger signals, with specially built water-proof cables, to allow live views in the video village above water of all cameras in 4K with 10bit and 60 frames per second.

“We needed a system that was a lot more supple and pliable to work around the way we were shooting… we could modify the rigs with 10 or 12 cameras very quickly with engineering and 3D printing pieces overnight,” says Dan Glass, visual effects production supervisor on the film.

To build all this incredible equipment, they set up a 1,000 square meter factory with the help of Studio Babelsberg’s own art department.

Filming of The Matrix Resurrections at the factory using the synchronised cameras. Credit: Volucap

“Lana was keen to utilize all the newest technology and even push it further. She would ask what is possible and what is not possible. If you told her the latter, she’d say ok let’s just do that and make it happen,” laughs Sven.

When it came time to test all the equipment there were no guarantees it would all work and plenty of technical issues to factor in, like the fact that colours change when you go underwater, and that there are dust particles and bubbles to contend with. As well as the problem of maintaining a clear image with multiple cameras focused on one spot. Having divers on hand helped managed these problems.

“Nobody had tested 3D measurements or reconstruction underwater before. Setting up cameras and doing video is one thing, but getting your software to measure the depth correctly is another challenge,” says Sven. “Plus, everything is a lot slower than above water, even simple things like changing the lenses.”

Initially they were keen to use a professional pool to trial things out but could only get access to a public pool belonging to a friend of Sven’s diving instructor. “It was very early in the morning, about 4am, when we set up, but by 9am we were having to film below old people swimming above us,” laughs Sven.

By contrast, their final volumetric studio built in a seven-meter deep pool at Studio Babelsberg looked really impressive. Although, unsurprisingly, there were still little hiccups along the way, like a leak in the housing. “We had to stop production temporarily to take a look and It was actually quite a quick fix, but when we turned around everyone had gone. They had left us to focus and not stress us out. It was a crucial moment for the production and all team members because it showed we weren’t freaking out when something goes wrong,” says Sven.

There was a lot of experimentation though along the way. “Lana would often have an idea of what she wanted, but not how it would look in the end,” adds Sven. “Fortunately, because we could capture the whole set, including everything behind the actors in 3D, Lana could be flexible and crazy in terms of visual effects afterwards. With volumetrics, you can stop at any time and move around, you can do bullet time at any point, so it’s an amazing system.”

Communication is key

As for communication between those above and below water during filming, a special speaker system was built for direct contact, and the actors were fitted with discreet breathing equipment, and could return to the surface any time on ropes.

The team even found an inventive way to bring the camera equipment to the surface. Normally it needs to be very static. “We came up with a quick calibration system bringing air in and then releasing it fast,” explains Sven. “Fortunately, because we controlled everything remotely, we didn’t need to touch or adjust the cameras very often.”

Once the team understood how the bullet time effect could and should look underwater, they then copied this shooting style, look and feel above water in Studio Babelsberg’s stages. “Having everything on site - and smaller, mobile camera rigs - certainly helped with the transition,” admits Sven.

The final film and effects look really impressive. Although it’s not always easy to see the work that Volucap did. “If you slow it down frame by frame you will see something weird in the faces or walls moving like you’re in the matrix,” says Sven.

Filming the slow mo bullet effect in the underwater studio. Credit: Volucap.

It was an incredibly innovative and effective way of working, but certainly not cheap or necessarily right for every production. Volumetric filming requires a hell of a lot of data. “To give you an idea, on a normal set you’d probably have two or three cinema cameras with quite low resolution. On this film, with our systems, there was about 100 terabytes of data for just 20 minutes of filming, which is about 40 times more data per camera, and we were using 30 or 40 cameras,” says Sven. “So you really need a lot of knowledge about infrastructure of IT to handle it. The alternative is going to a lower resolution, but it will be too blurry and not good enough for feature films.”

Sven and his team at Volucap are getting more requests to build custom solutions for films and TV series after the success of The Matrix Resurrections. “We have even got a portable, smaller volumetric capture studio, which can be transported to different countries around the world and used underwater. It’s no longer a case of how to do it, but what are your requirements?” concludes Sven.