Mixed reality is still waiting for its "killer app," but you shouldn't have too long to wait. That's the argument Jesse Schell, head of Schell Games, made during his GDC 2024 panel "The Future of MR Experiences."
It's not controversial to say that mixed reality (MR) has a long way to go compared to virtual reality (VR). The Meta Quest 3 has seen steady sales but nothing on par with the VR-focused Quest 2, and the exorbitant Apple Vision Pro is best suited for enthusiasts. MR games have vastly improved, but developers are still experimenting with what works.
Schell claims he's "skeptical" that MR gaming will be the killer app to make XR headsets sell because "I want to go to amazing fantasy worlds, not my dingy living room." He argued that other use cases, such as 3D design or enterprise, are too niche to resonate with most consumers.
So what is Schell's vision of MR's future? During his GDC 2024 panel, he showed off a Schell Games tech demo of the popular kids' game Hot Potato. The twist was that the four players, wearing Quest 3 headsets in their respective living rooms, merged their homes into one wall-less gaming space. They could strategically throw the hot potato under fellow players' tables or couches to try and win.
To Schell, "merging spaces" is the first "killer app" for mixed reality. Not many people will bring a second Quest 3 to a friend's house for local multiplayer, but people used to social VR games like VRChat or great sports games like Racket Club would enjoy seeing friends' avatars in a stylized recreation of their actual rooms.
(For context, the Quest 3 cameras don't actually record your surroundings, but they do create and store a "mesh" layout that developers could theoretically share with other players, assuming you gave permission. Privacy will be an important consideration.)
Another future MR killer app is "physical activity games," Schell claims, specifically aimed towards kids in the 10–15 age group. He says the gaming industry "doesn't have enough respect" for kids as players, pointing to the massive popularity of VR "playgrounds" like Gorilla Tag despite its basic gameplay and graphics.
VR fitness is popular with adults for a reason. MR games that expand out of the current 3-meter playspace and let kids run around their living rooms — possibly with the added bonus of seeing friends in their own bedrooms — could be a real hit.
Mixed reality will "prioritize social connectivity over physical distance" when it comes to game design, Schell says.
Schell's other vision of MR experiences, geared more towards adults, is a new genre called "adaptive in-home storytelling," which would rely on AI to create NPCs that actually interact with your home layout. He says they'll knock on your front door, cook in your kitchen, or invite you to look in the attic for something.
He didn't go into further detail, but another GDC panel from Google on GenAI games suggests that these kinds of adaptive AI games are still a few years out.
His last futuristic, mixed-reality killer app has to do with photorealistic avatars. He believes that if we can get a more realistic version of Apple's "Persona" avatars for "eye-to-eye conversations," they'll become a more popular way to communicate than flat-screen Google Meet calls.
Schell Games (Until You Fall, I Expect You to Die 3, Among Us VR) specializes in VR games, but we're curious to see how the developer implements Schell's ideas for killer MR apps, especially its merged-home environments. It takes the Horizon Home concept and expands it to games where players are more likely to want to hang out for long periods.
We also liked how Schell compared the Quest 3 and Apple Vision Pro to Atari and IBM in 1981. Even though we pitted the Vision Pro against the Quest 3 like everyone else — including Mark Zuckerberg, who roasted Apple's headset — it's fair to say that they're trying to accomplish different things and can (in theory) both exist.
