VR has come a long, long way in the six short years since the first real mainstream headsets launched, but as Meta CEO Mark Zuckerberg told me and a virtual room of journalists at a special roundtable this past week, we still have a long way to go.
During the hour-long session, Zuckerberg and a panel of Meta researchers explained what it's going to take to pass the "visual Turing Test" in the best VR headsets.
The photo above showcases a wall of prototypes Meta's Reality Labs has been working on since the company bought Oculus back in 2014. While some of these prototypes have gone on to become consumer-grade headsets we all know — like the Oculus Quest 2 — most of them serve a sole purpose and are specifically designed to solve a single problem in VR.
Whether that's increasing the resolution of the display, widening the field of view, introducing retina-scorching levels of brightness, or just trying to make VR headsets smaller and lighter, each and every prototype is a piece of the puzzle that will lead to better products over the long-haul. But this kind of business costs a lot of money — $10 billion, to be exact — and it's this very reason that Zuckerberg and company took the time to explain exactly what Meta has been spending its money on.
The visual Turing Test
The Butterscotch prototype... creates a view of a virtual world that looks essentially perfect to the human eye.
Butterscotch might not be a name that you associate with VR — or any technology at all, for that matter — but its classic flavor inspired the views that make this first prototype so darn sweet. The Butterscotch prototype from Meta was designed to create a retinal resolution VR display that looks indistinguishable from the quality of display we've come to expect from a modern smartphone or TV.
It achieves the famed 60-pixels-per-degree measurement that's been sought after in VR for ages, creating a view of a virtual world that looks essentially perfect to the human eye. It's in every way as clear as you view the real world around you, but it has one huge issue: An extremely narrow field of view.
Imagine holding a butterscotch candy up to your eye and only seeing the world through that tiny circle. Perhaps that's where Meta got the name for this fascinating prototype, but it shows how much of a challenge optics engineers have laid out before them. In a nutshell, as the field of view (FoV) of a VR headset's lens increases — thereby giving you a wider view of the virtual world to better replicate the human eyeball — the perceived resolution of the display decreases.
To counteract that, Meta engineers had to develop a new type of lens that would provide a wide enough FoV to be comfortable, while still presenting this ultra-high-resolution vision of the virtual world.
Pancake lenses are named such because they work to "pancake" the space between the display and the physical lens itself.
But that lens is just one of many different types of lenses Meta's Reality Labs has been working on. As we've seen from leaks and other bits and pieces of information, the upcoming Apple VR and the Oculus Quest Pro headsets seem to have all the right things in common. Namely, in this case, are the pancake lenses that are being used in those upcoming headsets.
While it's another strangely sweet-sounding name for a piece of technology, pancake lenses are named such because they work to "pancake" the space between the display and the physical lens itself. Current-generation VR headsets like the Quest 2 and PlayStation VR have a lot of empty space between the lens and the actual display because of how the lenses work. Pancake lenses reduce that distance by bouncing light off of microscopic structures within the lens itself.
Going beyond the pancake lens is one Meta calls the "Holocake lens," yet another sweet-sounding name that's clearly painting a picture of how hungry Reality Labs engineers must regularly be. Holocake utilizes a combination of pancake lenses in addition to a holographic lens, removing the need for concave or convex lenses, which have a curved structure and take up more space.
With that breakthrough, Meta was able to piece together a headset that's thinner and lighter than anything they had made before, all without sacrificing visual quality. The downside to this particular prototype, the Holocake 2, is that it's a PC-tethered headset and can't be used on its own like a Quest 2.
Holocake 2 utilizes a combination of pancake lenses in addition to a holographic lens to make things really flat.
Seeing this headset on Zuckerberg's head made me realize two things. First, the visual similarities to the HTC Vive Flow — a 3DOF headset that has to be tethered to a smartphone — and second, just how much sleeker it is than anything on the market right now. This product, of course, won't see the light of day on a store shelf, but could very well spawn a series of future products from Meta that utilize its unique design.
One such possibility is the upcoming Oculus Quest Pro — better known as Project Cambria at this time — which seems to feature a similar enough design to this particular prototype.
Starburst, on the other hand, was designed to achieve one specific goal, and throw all other constraints out the window. As you can see above, Starburst isn't usable in any normal situation and certainly isn't what you would call a "shelf-worthy" product, but it does do one thing very well: Its display is supremely bright.
Starburst — oh, hey, another candy name — was designed to be as "bright as the world around you," said Zuckerberg in the presentation, and has a proper HDR display that can deliver a significantly wider range of brightness levels than any current VR headset can. Much like an HDR TV, this means bright areas in an image are very bright but don't drown out those lovely deep shadowed areas. It also means colors are far more realistic than you might see on a Quest 2's display, for example.
Mirror Lake combines key technologies from just about every headset prototype you see above.
Rounding everything out was the introduction to Mirror Lake, a device that breaks the mold in every way. Firstly, it's not the name of a dessert or candy, which tells you right away that something is very different about it. Second, is the fact that there's no actual physical prototype of Mirror Lake anywhere in Reality Labs — at least, not one that they showed to the press.
Mirror Lake combines key technologies from just about every headset prototype you see above — plus one other prototype headset called Half-Dome — to achieve a thin, light, and powerful VR headset the likes of which simply does not exist today. In addition to the hybrid Holocake lens and a bright, high-resolution display, Mirror Lake incorporates Half-Dome's unique "varifocal" electronic lenses.
Varifocal lenses solve the problem of the fixed-focus lenses used in VR today. While current lens tech works well for many situations, holding objects closely always results in a blurry image when attempting to use two eyes to focus. Varifocal lenses — like the ones used in the Half-Dome and Mirror Lake prototype headsets — physically focus on different parts of the image depending on a number of factors. That means up-close objects are actually clear to the viewer and the movement of objects can be more clearly followed.
Amazingly enough, Meta's latest varifocal lenses don't even need to mechanically focus. They do everything electronically, taking up virtually no space at all in a headset. Seeing all this in action was incredibly exciting, to say the least, and it makes me thankful that at least one company is willing to spend the big bucks it takes to deliver this kind of technology to a lowly peon like myself.
These prototypes might not see the light of day any time soon (or ever), but they all give me serious hope that the next generation of VR will deliver even more groundbreaking tech than we've already seen. The future is certainly very bright for the metaverse.
