#1557: Apple Immersive Video Behind-the-Scenes & Overcoming Fears with “Adventure” Series Athlete
Ant Williams is a freediving athlete featured in the third episode of The Adventure Series titled "Ice Dive" that's co-produced by Apple and Atlantic Studios (formerly Atlantic Productions). In the episode, Williams attempts to swim a world record distance 182 meters under ice, and I wanted to get some additional behind-the-scenes context on his experience as well as what it was like to have the most intense month of his life condensed down into a 15-minute Apple Immersive Video. Williams is a sports psychologist who wanted to put his theories into practice by taking what he calls "positive, calculated risk-taking challenges" that allow him to deal with overwhelming anxiety, and overcome his fears, uncertainty, and self-doubt.
I also wanted to get some additional context on the production of the episode as Apple has otherwise been pretty tight-lipped about the series, launched with which launched with "Highlining" on the same day as the Apple Vision Pro launch on February 2, 2024. Apple Immersive Video is a different format than spatial video. Apple says "spatial videos are captured in 1080p at 30 frames per second in standard dynamic range," and these are what can be captured by either an iPhone or Apple Vision Pro, and they are displayed in a windowed frame where you see the stereoscopic effects. Apple describes Apple Immersive Video as "a remarkable storytelling format that leverages 3D video recorded in 8K with a 180-degree field of view and Spatial Audio to transport viewers to the center of the action."
Apple Immersive Video is much closer to what we've seen from the XR industry and VR 180 filmmakers from the past decade, and Apple's technology is likely derived from their 2020 acquisition of NextVR. NextVR focused on live stereoscopic broadcasts of sports events on VR headsets starting with the Samsung Gear VR and Oculus Rift in 2014.
A lot of the technical specifications of the Apple Immersive Video format have not been officially confirmed by Apple, but there are a couple of breadcrumbs that give us some more details. Thanks to iFixIt's breakdown of the Apple Vision Pro on February 7, 2024, then we know the microOLED display size is reported as "the lit area totals 3660 px by 3200 px." 360 Labs' Mike Rowell wrote a post on March 19, 2024 saying, "Apple Vision Pro’s screens are a whopping 3660 x 3200 pixels per eye. Although they haven’t made any official claims as to the FOV of the headset, 3rd party developers claim that it looks to be around 100° horizontal. With each screen having 3,660 horizontal pixels, this would mean that a 180° immersive experience would need about 6,000 x 6,000 pixels per eye to saturate the display. Apple’s own immersive experiences have been reported at being 4320x4320 per eye at 90fps and in HDR10."
The reporting of Apple's immersive experiences was detailed by Mike Swanson, who announced a spatial video tool on March 7, 2024 that leverages the Apple's AVFoundation to properly encode video into the "multiview extensions of the HEVC codec, known as MV-HEVC" format. Swanson says in his post, "I receive multiple messages and files every day from people who are trying to find the limits of what the Apple Vision Pro is capable of playing. You can start with the 4320×4320 per-eye 90fps content that Apple is producing and go up from there. I’ve personally played up to “12K” (11520×5760) per eye 360-degree stereo video at 30fps."
Another clue can be found in the Blackmagic URSA Cine Immersive camera that was announced on June 10, 2024, which says, "The sensor delivers 8160 x 7200 resolution per eye with pixel level synchronization and an incredible 16 stops of dynamic range, so cinematographers can shoot 90fps stereoscopic 3D immersive cinema content to a single file."
Incidentally, Currents director Jake Oleson told me that he used Swanson's tool to create his immersive film after shooting it in 8k on the Canon EOS R5 Camera Body with Canon's Dual Fisheye lens.
Netherlands