- Analysis
- Technology
- Gadgets
Apple’s vision for mixed reality could make headsets mainstream
By Tim Biggs
With the Vision Pro, Apple is finally entering into a headset market that’s been slowly maturing for years, but hasn’t quite found its mainstream appeal.
We’ve had game-focused virtual reality headsets (VR), clear-lensed augmented reality glasses that project digital images into the real world (AR), goggles that can do both in so-called mixed reality (MR), and most recently the term “extended reality” has emerged as a blanket term for the whole lot (XR).
Apple’s device likely won’t be sold in Australia until late 2024, and with the American price revealed to be a shocking $US3500 ($5300), not many people will be in line to get one. But could its successors take off as a new popular form of computing? We can scrutinise the details Apple has provided, and take a look back at some of its previous big new product launches, to get an early idea of how things might go.
What exactly is the Vision Pro?
Looking like a pair of space age ski goggles, the Vision Pro is a lightweight computer you attach to your face. It has a stretchy cloth band to keep it on your head, and is powered via a cable attached either to a battery in your pocket or a more stationary source of electricity. It detects motion so it knows when you move your head, has cameras inside to track your eyes, and cameras outside to track your hands.
Most importantly it also has a pair of extremely high-resolution screens that each sit in front of one of your eyeballs, a pair of speakers that sit near your ears for spatial audio, a screen facing outwards so others can tell whether you’re aware of them or not, and external colour cameras to let you see the outside world.
The idea is that the space around you becomes the medium for computing, rather than a monitor or smartphone screen, and your body becomes the input device. So for example movies or videos can appear as floating 30-metre screens in your backyard, or you could place a number of apps around your kitchen table instead of sorting them as tabs on a laptop screen.
This all requires an interface that seems just as new and confusing now as the touchscreen was when we were used to buttons. In some cases menus and options are selected just by looking at them, and you can “click” by touching your thumb and forefinger together. Many examples of apps shown off by Apple have had users walking around and interacting with digital elements using both hands. But you can also use your voice or traditional computing tools like keyboards and touchpads.
How is it different to other headsets?
On paper, the Vision Pro is an extremely ambitious device that seems to be tackling issues that the likes of HTC and Meta have barely touched because of the immense costs involved. It’s a mixed reality headset (not that Apple would use an industry term it hadn’t coined itself; it calls it a “spatial computer”), but one that seems a few years ahead of most.
For a start, Apple has said the Vision Pro is a modular build that can be adjusted to exactly fit the head shape of different users, presumably after they’re measured at an Apple Store. Users with glasses will also have custom ZEISS lenses with their prescription installed.
The system has a total of 12 cameras, five sensors, and six microphones, with Apple’s custom processor crunching all that input and able to reflect the information on-screen within — again, according to Apple — 12 milliseconds.
The two postage-stamp-sized screens have a total pixel count of 23 million, as compared to something like 7 million on Meta’s Quest Pro. The cameras, sensors and invisible LEDs trained on your eyes not only track where you’re looking at all times and model your face for use as a virtual avatar, but display a version of your eyes on the outside of the display in real time.
All of this should — in theory — have the effect of cutting down many of the bugbears of headset design. Nausea should be lessened because of the speed and fidelity, while problems with fit and vision should also be decreased. People in your physical space being able to see your eyes should make things less uncanny. But it all comes at a very high cost, at least for this first generation device.
Less is known about Apple’s software, though the company has released a few videos that give an idea of how things will work. The goal of mixed reality is ultimately to give the user control over whether they’re fully immersed in a virtual world or merely bringing digital elements into their normal surroundings.
Apple is doing this with a dial at users’ temple that can move smoothly move between the two, which is a unique take. It has also shown how the device can intelligently blend the two on its own. For example, if you’re in a fully immersive mode and somebody approaches you to talk, that person will appear to cut into your digital reality.
Can Apple really convince people to use XR?
The early reaction to the Vision Pro has been very similar to the early reactions to past new product categories from Apple. Brand diehards sighed about how catastrophically expensive it is, tech junkies opined about how it compares to existing devices that have been around for years, and regular folks just didn’t see why anyone would want to use it.
The Apple Vision Pro.Credit: AP
A phone with a touchscreen? A big laptop with no keyboard? A little computer on your wrist? Laughable.
The thing is, and this is by no means a fresh observation, Apple has a healthy knack of perfectly timing its product launches. Usually, the idea is well seeded in the market and similar devices have been around long enough to provide a good idea of what works and what needs work. Apple then puts its design, engineering and marketing prowess to bear to launch a product that offers many consumer their first exposure to the technology.
It’s a formula that has worked for Apple with the smartphone, the tablet and the smartwatch, and just might work with the extended reality headset.
For both the iPhone and the Apple Watch, Apple lowered the price of the second generation version after the first model was deemed too expensive.
And for those two products, as well as the iPad, the company eventually introduced various models at different price points, as well as keeping older models in stock at discounted prices. The first generation devices were only ever bought by the most hardcore early adopters, and in retrospect were all viewed as clunky.
To take the iPhone as the most famous example. In 2007, the mobile phone market was established, but nowhere near mature. Brands were experimenting with touchscreens, some phones had slide-out keyboards while others had chunky joysticks, and they all had vastly different software features.
By 2011, the time of the iPhone 4S, almost all popular phones looked like the iPhone. A wider audience had come around to the design, infrastructure and telcos had evolved to better realise the promise of smartphones, and the popularity of the iPhone prompted Google’s Android to adopt touchscreens as standard, and unite many other phone-makers into a strong alternative.
Smartwatches have been around since at least 2012, and as with phones they seemed to develop in all directions at once while staying firmly the domain of the tech-obsessed. We had square and round designs, e-ink displays and LCDs, mechanical watches with little screens, all sorts of dials and crowns, and we were just starting to integrate heart sensors and fitness tracking when Apple arrived in 2015.
By the fifth generation Apple had worked out the kind of features that do and do not work on the watch, and had laser focused on its use for tracking fitness and wellbeing. It’s by far the most popular wearable device, but it also made a case for mobile fitness tracking generally.
Some early presentations related to the iPhone or the iPad seem cringeworthy in retrospect when it comes to extolling the virtues of the devices, and the same will be true for the Vision Pro. But nobody really remembers those things.
Instead, just like the persistent connectivity and cameras of the iPhone, or the health-tracking of the Apple Watch, many of us likely discover “the thing” within the first few generations of the Vision, which builds on everything the last decade of iterative headset design has been leading up to.
Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.