How Marvel’s ‘What If…?’ Became the Ideal Platform for ILM Immersive’s Interactive Storytelling
Following in the footsteps of ILM Immersive’s (formerly ILMxLAB) “Vader Immortal” game and the company’s virtual reality experiences “Star Wars: Secret of the Empire” and “Avengers: Damage Control,” “What If…? — An Immersive Story” offers fans of the Marvel Cinematic Universe an opportunity not just to enter the world of their favorite superheroes, but participate in an adventure with them.
Borrowing from the namesake Disney+ anthology television series (and Marvel Comics series that inspired it) that explores speculative timelines in the superhero multiverse, “An Immersive Story” uses Apple’s Vision Pro headset to create a universe where participants can interact with characters like the Watcher, Sorcerer Supreme Wong, and a handful of alternate-reality heroes (and villains). Best of all, they can recreate and use the powers of those characters to advance their own story. “What If…?” director and executive producer Dave Bushore says that their approach to the project, which was released for free May 30, was as free from limitations as the show that inspired it.
More from Variety
“The constraints were only how many times could we ask ‘What if?’ of ourselves,” Bushore tells Variety.
Led with trademark cantankerousness by Doctor Strange’s sorcerer counterpart Wong, a “What If…?” participant learns how to repeat the movements of the characters to open portals and utilize the power of the Infinity Stones. Bushore says that even though the story was untethered from the proper Marvel canon, they wanted to integrate characters that MCU fans knew — just in unique ways. “When we were developing this, it was ‘What’s it been in the past? What was it in the movies?’ And we took that [approach] with the teenage Hela character,” he says, referring to a pint-sized version of the “Thor” foe.
“To all of the characters, [she has] a different perspective,” he explains. “She comes at the story from a place of, ‘I don’t actually want to do hero stuff, I just want to get my friend back’.” Bushore says that he encouraged writers David Dong and Phil McCarty to take a similar approach with the Infinity Stones themselves. “The Soul Gem is an iconic piece of Marvel storytelling. So being able to take that and go, ‘let’s use it differently,’ was fun.”
In order to develop the hourlong, mixed-reality experience, Bushore worked hand in hand with ILM Immersive executive producer Shereif M. Fattouh, who previously worked on “Ralph Breaks VR” and “Damage Control,” experiences created for virtual reality environment, the Void. “Getting to the point where you had a headset that can do this crystal clear pass-through and detect your hands and not have controllers, we had been thinking about that for a long time,” says Fattouh. “So we’ve just been really excited about doing it narrative-focused versus a traditional game.”
Bushore says they employed classic storytelling techniques to set up the experience before inviting the viewer to become an active participant. “Traditional media, people have an understanding of — an intro, an outro, an action sequence,” he says. So what we did specifically was have 2D flat media that tells an emotional story to the characters that you are about to participate with, before you go into a three-dimensional space that you are now the main character in.”
Fattouh highlights one of the key differences between immersive, experiences and traditional media. “It is very odd to not be able to cut,” he says. The solution they discovered was to use cinematic vignettes — short narrative clips that unfold like “shards” from the shattering multiverse — that serve as gateways to interactive sequences. “We brought in the amazing team at Flying Bark that works on the actual [“What If…?”] show so that we were able to have traditional forms of the series in a ‘shard’,” he says. “So that you do have a lot of dramatic cuts, more classic cinematography that can take place to tell a story, but then when we’re in our AR and VR moments.”
Developing projects for the Void, and even the Oculus Quest and Rift headsets gave them an advantage when conceiving this project for the Apple Vision Pro, even if the Apple device’s absence of a handset or more traditional controller presented them with new challenges. “We knew the basics of what works well, what kind of gestures are comfortable. Because there’s a ton of stuff that’s like, ‘that’s going to be awesome,’ but you do it, and it didn’t give me the feeling that I wanted to have,” Fattouh remembers. “It’s fictionally right, but it’s not physically right.”
He says that it took a lot of trial and error to create a harmonious experience between the traditional and interactive elements. “The Time Stone moment is such a great [example],” Fattouh says. “Where Hela is, where Wong is, when the Time Stone comes out, what we’re asking you to do, the framing was in a very cinematic style even though it was interactive, to really craft an emotional beat with the traditional techniques of cinematic filmmaking.”
In the wake of ILMxLAB re-branding as ILM Immersive, many of the company’s previous interactive experiences effectively disappeared; though projects like “Avengers: Damage Control” were well-attended at Void locations and otherwise critically acclaimed, their complexity prevented them from being transferred (or translated) to consumer consoles such as the Oculus headsets used for “Vader Immortal.” Bushore and Fattouh suggest that “What If…?” not only offers participants with a multifaceted — and potentially update-able — experience that rewards multiple viewings, but just scratches at the surface of what they hope to do within the Marvel Cinematic Universe.
“There’s a lot there for folks that are interested to go back in,” says Fattouh. “But the possibilities are really endless. I mean, the method of this kind of content and how it’s distributed, it’s not a stretch to imagine that there can be things that get added.”
Adds Bushore, “when you release something into the wild, most of the times you can’t change it. We can change this. We can literally react to some degree in real time to what the audience is thinking about.”
Whether or not they will, of course, will likely come down to how many people actually get to try out the hour-long experience, which is currently available exclusively on the Apple Vision Pro. (In April 2024, Apple projected 400,000 to 450,000 first-year sales of the headset, whose $3,500 price tag limits access to the device, much less its content.) Bushore admits he’s unsure exactly what’s next for Marvel Studios in the VR space, but after trying to answer the question asked by “What If..,?,” he’s particularly bullish about the future of interactive storytelling.
“I grew up watching stories about being able to go into computer simulations, and now we have that capability,” Bushore says. “To put people in spaces with characters to make them feel things, it’s the core of why we tell stories in the first place. And I can’t wait to do it again.”
Best of Variety
'Blue Velvet,' 'Chinatown' and 'Fear and Loathing in Las Vegas Arrive on 4K in June
'House of the Dragon': Every Character and What You Need to Know About the 'Game of Thrones' Prequel
Sign up for Variety’s Newsletter. For the latest news, follow us on Facebook, Twitter, and Instagram.
