Hands on with visionOS -- using the Apple Vision Pro operating system

AppleInsider

The Apple Vision Pro simulator is in the latest beta of Xcode. AppleInsider walks through various apps and the all-encompassing UI of Apple's new spatial computing headset, set to arrive in 2024.

Vision Pro

We got our first look at Vision Pro at Apple Park during WWDC 2023 where we brought you detailed insight into the headset, including a closeup of the hardware.

And now, with the release of the developer software for the headset, we can show you the interface of Vision Pro and its visionOS software.

Navigating visionOS

As you launch into visionOS, you'll see your Home Screen of apps. As you've probably seen from Apple's various marketing videos, it looks like a spatial version of the Apple Watch grid with circular app icons.

Subscribe to AppleInsider on YouTube

There's a dock down the left-hand side to get to other pages on your Home Screen. One page has apps, the second has FaceTime contacts, and the third has different environments you can immerse yourself.

Environments in visionOS

There are 14 environments you can choose from, including Haleakala, Yosemite, sky, spring light, Joshua Tree, Lake Vrangla, Mount Hood, summer light, fall light, the moon, beach, white sands, and winter light.

These environments can replace your living room and transport you as you read, meditate, or work.

Floating near the top, there is a small caret icon that opens up device controls. The caret was exceptionally annoying while using the simulator as we'd inadvertently tap it on accident while trying to work within other applications.

Using the device is better, though, as you have to look at the caret to activate it.

The first icon in the Control Center takes you back to the Home Screen -- a task also accomplished by the Digital Crown. The second icon lets you switch the environmental light.

The last two icons were for Control Center and Notification Center.

Vision Pro Control Center

Control Center hosts toggles for your usual connectivity methods -- AirPlay, Wi-Fi, Bluetooth, airplane mode, Do Not Disturb, volume -- but also has Spotlight search, a Now Playing widget, and guest mode.

Apple mentioned guest mode at the device's launch as an easy way to let others try your Vision Pro headset. Just enable it and set a passcode that will be good for five minutes.

Guest mode on Vision Pro

Assuming they put the passcode in within five minutes, they can test the headset without access to your private data.

When you open an app, it can be closed by hovering over the small dot below the open window. If you select the adjacent bar, you can move the windows around.

Windows can move forward, backward, up, down, or anywhere you need them to move. Multiple windows can be opened and stacked atop one another with the ones that are in the background having decreased opacity.

Using visionOS

In the simulator, the first thing we did was jump into Safari. We loaded up AppleInsider which felt very similar to the iPad. You could hover over the unified search bar where the multiple open tabs would appear.

Keyboard on visionOS

Typing is done either via a connected Bluetooth keyboard or you can use the on-screen keyboard that appears to float below the window. Just like windows, the keyboard can be closed or repositioned.

Emoji keyboard on visionOS

Sure enough, Apple has an emoji keyboard in visionOS too. Perfect for using iMessage, email, or social.

We also loaded up Freeform in visionOS. It was cool to see how you could draw on the massive whiteboard in the virtual living room.

Freeform in visionOS

The Freeform usability is much better here than on iOS, almost like Apple originally developed it with Vision Pro in mind. Like a full-sized virtual whiteboard.

Some stock apps are on Apple Vision Pro now

In the first beta of visionOS, we got our first look at what apps Apple hopes to include with Vision Pro. Many are not available this far ahead of release.

Currently, Apple has working versions of Freeform, Safari, Photos, and Files. These are all native to the visionOS experience.

Compatible apps for visionOS

Some apps are merely ported versions directly from iPadOS or iOS. These are listed as compatible apps and right now include Calendar, Maps, News, Reminders, and Shortcuts.

Compatible apps worked fine in the simulator but they didn't scale the same. They had a locked aspect ratio as you resized them.

Apple News can change orientation

They also have a little icon in the corner where you can rotate them from landscape to horizontal like you were turning your iPad.

In the simulator, Apple shows icons for Books, Clock, Podcasts, and Stocks -- although they aren't available just yet. On the Vision Pro mini-site, Apple highlights the entire iWork suite, Notes, Messages, Apple TV, and Meditation.

Rumors say there are a lot more apps on the way for Vision Pro, with Apple working on an app for Tai Chi, Nike workouts, yoga, and more. Surely we'll see more as we approach release.

Apple Vision Pro simulator is just the start of a long trip

This is still only a simulator running the very first beta of Apple's new operating system. It will change drastically over the coming years.

That being said, it is impressive. Compared to the limited hands-on time Apple offered in Cupertino, developers and users can play around in visionOS as long as they'd like.

The potential is clear. We're excited to see what Apple and third-party developers will ready by the time Vision Pro starts shipping in 2024.

Read on AppleInsider

[email protected]

Hands? I see what you did there.

darkvader

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

KTR

I can see the mapping companies using this 

alandail

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

pretty much the only difference is being able to run Xcode. That's the only reason I carry my MacBook with me when I travel.

mknelson

Thank you for filling in the missing detail. The announcements yesterday didn't mention the simulator as far as I saw.

melgross

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

I don’t know. We’re seeing more Mac apps come to iOS anyway. Some of the iOS versions are better and some need more updating to competely get there. I’m not concerned though. We’ll get new apps that can’t be done in iOS or MacOS. But it will take time. This won’t be an instant fully featured thing with every app we want. That figures. Look at the iPhone. The first year it only had the built in apps. Next year, with the App Store, it had 512 more. Now it has 2 million, with a million for the iPad. Who could have predicted that the first year the phone came out?

So I think we need to sit back and not get riled up about what might be there or what might not be there. Apparently Apple won’t be able to sell more than about 400,000 the first year anyway. I’ll try to get one, but likely won’t be able to.

tpurdy

It was cool to see people / user accounts. Hopefully this makes it to iPad someday too

chutzpah

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

They already showed it mirroring a Mac screen.  Obviously it won't run Mac apps on-device without modification.

And there's plenty of scope for it to be useful without running Mac apps.  Silly trolling.

StrangeDays

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

We already know you can look at a nearby Mac and pull them into the VP’s spatial computing.  As a desktop, or with the VP as a virtual display, and you can arrange each app window. 

https://www.macrumors.com/2023/06/06/apple-vision-pro-mac/

MplsP

StrangeDays said:

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

We already know you can look at a nearby Mac and pull them into the VP’s spatial computing.  As a desktop, or with the VP as a virtual display, and you can arrange each app window. 

https://www.macrumors.com/2023/06/06/apple-vision-pro-mac/

Ultimately @darkvader is right. Of course the same could be said about any new product.  VR has been ‘the next big thing’ for years but hasn’t really progressed much beyond gaming and other niche uses. 

Apple’s headset is not just VR, though and there is a lot of potential. In typical Apple fashion they put work into integrating it with macs and the rest of their lineup and focused a lot on the interface and usability. Hopefully it keeps progressing. 

I don’t have a use and can’t justify $3,500 for one but I’d love to try one. 

Edit: I haven’t read anything but I assume you can pair it with AirPods? The spatial audio and improved audio quality would be an awesome compliment to the headset. 

tht

MplsP said:

StrangeDays said:

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

We already know you can look at a nearby Mac and pull them into the VP’s spatial computing.  As a desktop, or with the VP as a virtual display, and you can arrange each app window. 

https://www.macrumors.com/2023/06/06/apple-vision-pro-mac/

Ultimately @darkvader is right. Of course the same could be said about any new product.  VR has been ‘the next big thing’ for years but hasn’t really progressed much beyond gaming and other niche uses. 

Apple’s headset is not just VR, though and there is a lot of potential. In typical Apple fashion they put work into integrating it with macs and the rest of their lineup and focused a lot on the interface and usability. Hopefully it keeps progressing. 

I don’t have a use and can’t justify $3,500 for one but I’d love to try one. 

Edit: I haven’t read anything but I assume you can pair it with AirPods? The spatial audio and improved audio quality would be an awesome compliment to the headset. 

Heh, the first time I used a VR headset was in 1991. 

You can pair it with AirPods. You can see it in the intro video when the woman is on the airplane. It will likely work with all headphones that at least support spatial audio. Perhaps a limiter is headphones may need to support higher sample rate rate Bluetooth connections to keep latency at a minimum.

Apple's pass-through VR tech stack looks like it has crossed a vaguely defined level where mass market adoption is possible. The eye and hand tracking looks to have solved a baseline level of UI interaction, like touch and capacitive touchscreens for smartphones, while the R2 and realtime OS frameworks to drive frame rates have addressed motion sickness for a larger portion of the market than existing solutions.

The rest is driving down pricing. I'm thinking it's addressing a potential market size of the combined tablet and PC market where people can use it for entertainment and computing. The utility is there to overcome the difficulties with it being a heavy goggle set, and people will want to wear it even with all its bulkiness.

Apple however is too... opinionated. It really needs to have Terminal.app, just like iPadOS should have, and even iOS. Don't care how they do it, like running DarwinOS in it's own VM, but it is an important part of computing that they need to have on all their computing platforms. tvOS no, but visionOS? Yes.

It probably also needs a wired way to feed Mac displays into the headset to reduce latency. There will be a class of apps that M2, M3, etc, SoCs can't run, and letting a Mac splay all its apps all over the views inside visionOS will be necessary. One workflow that they can do is using a Mac's display and a virtual display inside the headset.

Japhey

Anybody know if that virtual keyboard supports swipe typing? 

Off topic, I really wish I had taken more panoramas over the years. 

Japhey

tht said:

MplsP said:

StrangeDays said:

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

We already know you can look at a nearby Mac and pull them into the VP’s spatial computing.  As a desktop, or with the VP as a virtual display, and you can arrange each app window. 

https://www.macrumors.com/2023/06/06/apple-vision-pro-mac/

Ultimately @darkvader is right. Of course the same could be said about any new product.  VR has been ‘the next big thing’ for years but hasn’t really progressed much beyond gaming and other niche uses. 

Apple’s headset is not just VR, though and there is a lot of potential. In typical Apple fashion they put work into integrating it with macs and the rest of their lineup and focused a lot on the interface and usability. Hopefully it keeps progressing. 

I don’t have a use and can’t justify $3,500 for one but I’d love to try one. 

Edit: I haven’t read anything but I assume you can pair it with AirPods? The spatial audio and improved audio quality would be an awesome compliment to the headset. 

Heh, the first time I used a VR headset was in 1991. 

You can pair it with AirPods. You can see it in the intro video when the woman is on the airplane. It will likely work with all headphones that at least support spatial audio. Perhaps a limiter is headphones may need to support higher sample rate rate Bluetooth connections to keep latency at a minimum.

Apple's pass-through VR tech stack looks like it has crossed a vaguely defined level where mass market adoption is possible. The eye and hand tracking looks to have solved a baseline level of UI interaction, like touch and capacitive touchscreens for smartphones, while the R2 and realtime OS frameworks to drive frame rates have addressed motion sickness for a larger portion of the market than existing solutions.

The rest is driving down pricing. I'm thinking it's addressing a potential market size of the combined tablet and PC market where people can use it for entertainment and computing. The utility is there to overcome the difficulties with it being a heavy goggle set, and people will want to wear it even with all its bulkiness.

Apple however is too... opinionated. It really needs to have Terminal.app, just like iPadOS should have, and even iOS. Don't care how they do it, like running DarwinOS in it's own VM, but it is an important part of computing that they need to have on all their computing platforms. tvOS no, but visionOS? Yes.

It probably also needs a wired way to feed Mac displays into the headset to reduce latency. There will be a class of apps that M2, M3, etc, SoCs can't run, and letting a Mac splay all its apps all over the views inside visionOS will be necessary. One workflow that they can do is using a Mac's display and a virtual display inside the headset.

Just out of curiosity, do you remember the name of that VR unit from ’91? Because I used to go to the old Best Products around the same time to play around with a floor model they had, but I just can’t remember who made it. 

tht

Japhey said:

tht said:

MplsP said:

StrangeDays said:

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

We already know you can look at a nearby Mac and pull them into the VP’s spatial computing.  As a desktop, or with the VP as a virtual display, and you can arrange each app window. 

https://www.macrumors.com/2023/06/06/apple-vision-pro-mac/

Ultimately @darkvader is right. Of course the same could be said about any new product.  VR has been ‘the next big thing’ for years but hasn’t really progressed much beyond gaming and other niche uses. 

Apple’s headset is not just VR, though and there is a lot of potential. In typical Apple fashion they put work into integrating it with macs and the rest of their lineup and focused a lot on the interface and usability. Hopefully it keeps progressing. 

I don’t have a use and can’t justify $3,500 for one but I’d love to try one. 

Edit: I haven’t read anything but I assume you can pair it with AirPods? The spatial audio and improved audio quality would be an awesome compliment to the headset. 

Heh, the first time I used a VR headset was in 1991. 

You can pair it with AirPods. You can see it in the intro video when the woman is on the airplane. It will likely work with all headphones that at least support spatial audio. Perhaps a limiter is headphones may need to support higher sample rate rate Bluetooth connections to keep latency at a minimum.

Apple's pass-through VR tech stack looks like it has crossed a vaguely defined level where mass market adoption is possible. The eye and hand tracking looks to have solved a baseline level of UI interaction, like touch and capacitive touchscreens for smartphones, while the R2 and realtime OS frameworks to drive frame rates have addressed motion sickness for a larger portion of the market than existing solutions.

The rest is driving down pricing. I'm thinking it's addressing a potential market size of the combined tablet and PC market where people can use it for entertainment and computing. The utility is there to overcome the difficulties with it being a heavy goggle set, and people will want to wear it even with all its bulkiness.

Apple however is too... opinionated. It really needs to have Terminal.app, just like iPadOS should have, and even iOS. Don't care how they do it, like running DarwinOS in it's own VM, but it is an important part of computing that they need to have on all their computing platforms. tvOS no, but visionOS? Yes.

It probably also needs a wired way to feed Mac displays into the headset to reduce latency. There will be a class of apps that M2, M3, etc, SoCs can't run, and letting a Mac splay all its apps all over the views inside visionOS will be necessary. One workflow that they can do is using a Mac's display and a virtual display inside the headset.

Just out of curiosity, do you remember the name of that VR unit from ’91? Because I used to go to the old Best Products around the same time to play around with a floor model they had, but I just can’t remember who made it. 

I don’t remember. It was powered by an SGI workstation and had thin, wired gloves. It used hand tracking with the gloves to navigating the 3D view. 

Xed

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

Kinda like how the iPad is completely useless and stupid because it's not simply running macOS? 🙄

Xed

MplsP said:

StrangeDays said:

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

We already know you can look at a nearby Mac and pull them into the VP’s spatial computing.  As a desktop, or with the VP as a virtual display, and you can arrange each app window. 

https://www.macrumors.com/2023/06/06/apple-vision-pro-mac/

Ultimately @darkvader is right. Of course the same could be said about any new product.  VR has been ‘the next big thing’ for years but hasn’t really progressed much beyond gaming and other niche uses. 

Apple’s headset is not just VR, though and there is a lot of potential. In typical Apple fashion they put work into integrating it with macs and the rest of their lineup and focused a lot on the interface and usability. Hopefully it keeps progressing. 

I don’t have a use and can’t justify $3,500 for one but I’d love to try one. 

Edit: I haven’t read anything but I assume you can pair it with AirPods? The spatial audio and improved audio quality would be an awesome compliment to the headset. 

No, he's not even close to being right. Note that he said Apple Vision Pro will be "completely useless and stupid" if it doesn't run macOS apps. Note that Apple's demo showed an OS built around the HW and apps designed for that specific VR and AR usage, not simply Mac apps.

Xed

melgross said:

darkvader said:

You know, this thing could actually be useful (if it runs Mac apps) or completely useless and stupid (if it doesn't). 

I wonder which one will happen?

I don’t know. We’re seeing more Mac apps come to iOS anyway. Some of the iOS versions are better and some need more updating to competely get there. I’m not concerned though. We’ll get new apps that can’t be done in iOS or MacOS. But it will take time. This won’t be an instant fully featured thing with every app we want. That figures. Look at the iPhone. The first year it only had the built in apps. Next year, with the App Store, it had 512 more. Now it has 2 million, with a million for the iPad. Who could have predicted that the first year the phone came out?

So I think we need to sit back and not get riled up about what might be there or what might not be there. Apparently Apple won’t be able to sell more than about 400,000 the first year anyway. I’ll try to get one, but likely won’t be able to.

Doing the same thing as a Mac app doesn't make them Mac apps. Safari was first on macOS. The same for Mail, Calendar, etc. but these aren't simply "Mac apps" because they have the same name and provide the same core function as the Mac app. They are designed for that device type. You can now run actual iOS and iPadOS apps on your M-series Macs and at one point could run iPhone apps on your iPad, but that feature was deprecated a long time ago in favor of apps designed for the iPad.

CheeseFreeze

I get pretty much all Apple products and their use-cases. They all succeed up to a certain degree.

The laptop and the watch are clearly serving a purpose and target audience. The iPad does (amazing for kids!) as it’s positioned for light computing. The iPad Pro doesn’t so much (laptop prices for a crippled experience) but still makes sense. 

What I can’t figure out is the Vision Pro. It wants to be a true spatial computer. Okay, so let’s walk that out. Do we need one? At $3500 I expect a high-end device, but it has an M2. It takes much more eye strain to look at two screens mounted right in front of your head (ironically, iPhone/iPad will come with a feature warning users to keep distance between your eyes and screen). It also doesn’t seem to be as practical as using a mouse and keyboard. I’m not going to develop or make music on it for more than 10 minutes; the usability and barrier to entry doesn’t seem to be there, let alone the cost.
It is positioned by Apple to compete with normal computers, because they didn’t announce any app to prove otherwise. 

Gaming then? Nah, that seems to be covered by the $500 Meta Quest 3. 

I just don’t see any reason for this device to exist. There is no ‘killer app’ or ‘killer use case’.

Why did Apple announce this?

Xed

CheeseFreeze said:

I get pretty much all Apple products and their use-cases. They all succeed up to a certain degree.

The laptop and the watch are clearly serving a purpose and target audience. The iPad does (amazing for kids!) as it’s positioned for light computing. The iPad Pro doesn’t so much (laptop prices for a crippled experience) but still makes sense. 

What I can’t figure out is the Vision Pro. It wants to be a true spatial computer. Okay, so let’s walk that out. Do we need one? At $3500 I expect a high-end device, but it has an M2. It takes much more eye strain to look at two screens mounted right in front of your head (ironically, iPhone/iPad will come with a feature warning users to keep distance between your eyes and screen). It also doesn’t seem to be as practical as using a mouse and keyboard. I’m not going to develop or make music on it for more than 10 minutes; the usability and barrier to entry doesn’t seem to be there, let alone the cost.
It is positioned by Apple to compete with normal computers, because they didn’t announce any app to prove otherwise. 

Gaming then? Nah, that seems to be covered by the $500 Meta Quest 3. 

If you think gaming will be better on Meta then you haven't looked at its specs.

I just don’t see any reason for this device to exist. There is no ‘killer app’ or ‘killer use case’.

Why did Apple announce this?

How exactly do you get this into the hands of developers with an SDK while also keeping it completely quiet?

chutzpah

CheeseFreeze said:

What I can’t figure out is the Vision Pro. It wants to be a true spatial computer. Okay, so let’s walk that out. 

Ok, let's.

Do we need one? 

No.  No one said you need one.  You don't need any Apple products to function perfectly well in life.

At $3500 I expect a high-end device, but it has an M2.

It also has a lot of other things.  A lot of other high end things.  2 >4k micro displays.  More cameras than I care to count.  A dedicated chip to control those camera inputs.  If you want to strap an M2 Ultra and associated heat sink to your face in a 2kg package then you'll have to wait.

The Snapdragon in the Quest Pro wouldn't even be able to run the VisionPros screens, and the HoloLens uses an Atom.  An Atom!  And you're complaining about an M2?  It's a great chip.

It takes much more eye strain to look at two screens mounted right in front of your head 

It does?  Who says?

 It also doesn’t seem to be as practical as using a mouse and keyboard. 

Why not?  Also, you can use a mouse a keyboard with it, so do that if you really need that practicality.

I’m not going to develop or make music on it for more than 10 minutes; the usability and barrier to entry doesn’t seem to be there, let alone the cost.

The barrier to entry and cost doesn't seem to be there?  Well that's great!  Otherwise, maybe try it before you decide so clearly how long you'll use it for?

It is positioned by Apple to compete with normal computers, because they didn’t announce any app to prove otherwise.

Ok?  You don't think 3D experience as a feature differentiates it?  Or cinema screens in a device you can pack in hand luggage?  Bit weird that you can't since they made quite a big deal about it.

Gaming then? Nah, that seems to be covered by the $500 Meta Quest 3. 

Have you tried the Quest 3?  It definitely doesn't "cover it", and I doubt Meta thinks so either, otherwise the Quest Pro wouldn't exist.  It's not bad for an entry level product, but it's the cheapest device they could make, and it shows.

I just don’t see any reason for this device to exist. There is no ‘killer app’ or ‘killer use case’.

It'd be nice to have one, for sure.  But the Watch didn't have one, it was realised somewhat later in Fitness.  The iPad didn't have one either, still doesn't really, though that doesn't stop it from being a fine device.  Hell the iPhone didn't even have a clear killer use case at launch, that came somewhat later with the app store and the possibilities that opened up.

Why did Apple announce this?

Because Apple gonna do what Apple gonna do.  No point whining about it, just enjoy the show.