Apple's headset to use 3D operating system that looks like iOS
Apple's VR headset will feature a 3D operating system that will resemble iOS with hand and finger gesture recognition -- and can be used as a macOS display.
![A render of a potential Apple headset [AppleInsider]](https://photos5.appleinsider.com/gallery/52621-105127-51744-102531-51350-101493-50209-98618-49777-97523-apple-ar-headset-xl-xl-xl-xl-xl.jpg)
A render of a potential Apple headset [AppleInsider]
The long-rumored Apple AR headset could finally become a real product in 2023, but there are still many questions about how it will actually function. In a new breakdown of how the headset will work, Apple's attempt to take on Meta and others will rely on producing a great operating system experience, as well as high-specification hardware.
Driving the entire device will be an operating system that will have an iOS-like interface, according to Bloomberg. The operating system, xrOS, will be completely in 3D, and offer many of the same features as iOS and iPadOS, including a version of Safari, Photos, Mail, Messages, and Calendar.
The aim is to apparently offer an experience familiar to iPhone and iPad users.
To navigate the environment, hand, finger, and eye tracking will be used via several cameras and sensors, with gestures such as pinching their thumb and index finger used to trigger items. Eye tracking will allow users to select elements from a menu, without clumsily doing so with their hands.
Despite being a headset that covers the head, FaceTime will still be usable, with the headset realistically rendering a user's face and body in VR as an avatar. This would be wildly different from Meta's approach, which relies on cartoon-like avatars instead.
The realistic avatars will only be supported in one-on-one video chats due to the high amount of processing power required for it to work. For multiple people, additional users could be displayed as an icon or as Memoji.
A Digital Crown, like the Apple Watch version, will be used to switch users between AR and VR modes, so that they can gradually move between the two depending on how much of the real-world environment they need to see.
To further productivity, the headset will also be able to work as an external monitor for a Mac. The user's Mac display will be visible in VR, but control of the computer is still performed using a trackpad, mouse, or physical keyboard.
This sort of system is available elsewhere, including Meta's Horizon Workrooms, which also employs a passthrough system so you can access a Mac's keyboard and mouse.
To enter text into the system, users will still have an option to do it entirely in-headset. Technology is being made to allow users to type in midair with their hands on a virtual keyboard, however it apparently won't be ready for the initial launch of the device.
Siri can also be used to enter text via dictation.
An App Store will also be offered, complete with third-party software that could be installed, much like the iOS variant. The famous home screen will also make an appearance, compete with a grid of app icons and pinnable widgets. App Store
Powered by a variation of the M2 chip and a dedicated graphics processor called the "Reality Processor," the headset will have two ultra-resolution displays developed with Sony, as well as a collection of cameras to handle pass-through mode. To ensure the screen is visible to glasses users, Apple will offer custom lenses that can rest inside the enclosure itself.
Power is derived from a battery pack, which will be approximately the size of two iPhone 14 Pro Maxes stacked on each other, with power lasting for up to two hours.
However, a prototype headset has the battery pack designed to fit into a user's pocket, rather than being attached to the headset's headband or incorporated into the headset itself. This tethered approach may cause problems for users in the long run, but it will also reduce the weight of the headset sat on their heads.
Current speculation is for a launch unveiling the headset in early 2023, for release later in the year for around $3,000.
As part of Apple's promotional efforts, it may go down the route of the Apple Watch in creating a store-within-a-store, so customers can try out the headsets before buying it.
This could provide Apple with a byproduct of extra sales, as a customer may end up acquiring an iPad or other products from the store, even if they don't pay for the headset itself.
Read on AppleInsider
A render of a potential Apple headset [AppleInsider]
The long-rumored Apple AR headset could finally become a real product in 2023, but there are still many questions about how it will actually function. In a new breakdown of how the headset will work, Apple's attempt to take on Meta and others will rely on producing a great operating system experience, as well as high-specification hardware.
Driving the entire device will be an operating system that will have an iOS-like interface, according to Bloomberg. The operating system, xrOS, will be completely in 3D, and offer many of the same features as iOS and iPadOS, including a version of Safari, Photos, Mail, Messages, and Calendar.
The aim is to apparently offer an experience familiar to iPhone and iPad users.
To navigate the environment, hand, finger, and eye tracking will be used via several cameras and sensors, with gestures such as pinching their thumb and index finger used to trigger items. Eye tracking will allow users to select elements from a menu, without clumsily doing so with their hands.
Despite being a headset that covers the head, FaceTime will still be usable, with the headset realistically rendering a user's face and body in VR as an avatar. This would be wildly different from Meta's approach, which relies on cartoon-like avatars instead.
The realistic avatars will only be supported in one-on-one video chats due to the high amount of processing power required for it to work. For multiple people, additional users could be displayed as an icon or as Memoji.
A Digital Crown, like the Apple Watch version, will be used to switch users between AR and VR modes, so that they can gradually move between the two depending on how much of the real-world environment they need to see.
To further productivity, the headset will also be able to work as an external monitor for a Mac. The user's Mac display will be visible in VR, but control of the computer is still performed using a trackpad, mouse, or physical keyboard.
This sort of system is available elsewhere, including Meta's Horizon Workrooms, which also employs a passthrough system so you can access a Mac's keyboard and mouse.
To enter text into the system, users will still have an option to do it entirely in-headset. Technology is being made to allow users to type in midair with their hands on a virtual keyboard, however it apparently won't be ready for the initial launch of the device.
Siri can also be used to enter text via dictation.
An App Store will also be offered, complete with third-party software that could be installed, much like the iOS variant. The famous home screen will also make an appearance, compete with a grid of app icons and pinnable widgets. App Store
Powered by a variation of the M2 chip and a dedicated graphics processor called the "Reality Processor," the headset will have two ultra-resolution displays developed with Sony, as well as a collection of cameras to handle pass-through mode. To ensure the screen is visible to glasses users, Apple will offer custom lenses that can rest inside the enclosure itself.
Power is derived from a battery pack, which will be approximately the size of two iPhone 14 Pro Maxes stacked on each other, with power lasting for up to two hours.
However, a prototype headset has the battery pack designed to fit into a user's pocket, rather than being attached to the headset's headband or incorporated into the headset itself. This tethered approach may cause problems for users in the long run, but it will also reduce the weight of the headset sat on their heads.
Current speculation is for a launch unveiling the headset in early 2023, for release later in the year for around $3,000.
As part of Apple's promotional efforts, it may go down the route of the Apple Watch in creating a store-within-a-store, so customers can try out the headsets before buying it.
This could provide Apple with a byproduct of extra sales, as a customer may end up acquiring an iPad or other products from the store, even if they don't pay for the headset itself.
Read on AppleInsider
Comments
The Achilles heels to this proposal is the $3000 price tag, and that it isn’t something you take with you on the go.
for wide uptake, what you can do with it would have to be incredibly life changing for users. And arguably essential.
I find this really odd! They made the decision to have an external battery pack, but they design a battery pack that can only power the device for 2 hours. That's nuts? Weird. Well, surely that means there will be 3rd party packs.
One thing to note is that I find it implausible that they could have an M1 Max like SoC embedded in the goggles. That's a huge heat source. Plain ski goggles make people sweat. Add 50 W to it? No way. So perhaps the computing bits actually in the "battery pack" too. Once that happens, the whole deal of having a pocket sized battery pack only powering it for two hours makes a little more sense.
I can already feel myself feeling nauseous. rOS is better than xrOS.
I mean the display composition engine has been a 3D openGL (assume now Metal) scene since Leopard or slightly before. Allowing all the flying window and stacking effects driven by the OS. Apps stream textures to the scene of what is visible in their windows.
Seem like xrOS is just the next step with 2 cameras/eyes looking at the scene to set the depth location of the interface elements.
To me the question is can 3D content apps stream a model to the viewport of the app window so you can a stereo view of the model when connected to macOS apps.
one thing that strikes me as funny is the bit about getting to use your actual face in FaceTime calls —guess what, I already get to do that! In fact I’ve been on Zoom calls with dozens of people and their actual faces. So I’m not sure how much of a killer feature that is.
The rumored $3,000 price is just an Apple marketing mind game like the rumors before the iPad was released, When this device comes at out $1,500, the tech press will marvel at the bargain and more people will be able to afford it. Your overall premise is, in my view, correct, that it won't be the next big thing at least until a cheaper more mass-market version is released. (Maybe it will be a subscription device?)
Also Apple is so uniquely positioned to pull this off…hardware/software integration, computing efficiency (huge here for weight and heat issues), plus its primesense acquisition and ability to integrate real world motion as a simulacra.
Plus I will say it again, spectator sports will be the thing that will be unexpected boon.
Afterall if people can put this on their head, its not a stretch to wear VR wear. And imagine how great it would be to be immersed in game action?