New visionOS 2 updates are more significant than Apple initially implied
Apple's unveiling of visionOS 2 ended with its bento image summarizing its features. Here's what that graphic said, and what it means for the Apple Vision Pro.
An Apple Vision Pro user in front of the visionOS bento graphic
Like its other operating systems, Apple finished its exploration of visionOS 2 and its inbound changes with a bento graphic. Named after the compartmentalized food container, the bento image has been Apple's go-to for showing as many features as possible about a product or service.
While it does highlight a large number of features, it doesn't do so in detail. Instead, it's only got a few words on each.
Apple's 'Bento' image for visionOS 2.0
Here's what's coming in visionOS 2.0
Create spatial photos
Under visionOS 2, users can create spatial photos from the Photos app. Machine learning converts a 2D image to a 3D version, viewable as a spatial photo.
We've tried this already, and it's impressive. It's not the same as taking a photo natively on Apple Vision Pro or on an iPhone 15 Pro, but those pictures you took a long time ago can get a very nice 3D treatment.
Bora Bora environment
Users get a new view for the Apple Vision Pro. The Bora Bora environment is a beach scene with blue waters and distant mountainous views.
Multiview in TV app
Viewers of Major League Soccer and Major League Baseball can watch multiple games simultaneously. This is already a feature of tvOS and iPadOS, with it being an expansion to the headset.
-xl-xl.jpg)
Multiview of MLB and MLS will be available. Apple previewed it showing basketball.
Five different games can be viewed in five separate views, with a main view switchable with games in four smaller views.
Travel Mode on trains
Previously, as we'd discussed, Apple Vision Pro travel mode didn't work at all on trains. Apple took that feedback to heart, apparently.
Train support in Travel Mode will allow the Apple Vision Pro to be used while on the railways of the world.
New Apple Immersive Video
More Apple Immersive Video titles will be arriving in the future. These are 180-degree field-of-view films that leverage 3D 8K video.
The existing roster of footage includes a lot of action shorts, including a Parkour run in Paris. During the event, Apple teased more is on the way from Red Bull, and what appears to be a submarine-based one, which we will get our sub service vet Mike Wuerthele to assess for you when it's available.
Apple also took a moment to reveal work by Blackmagic Design to create en end-to-end solution for producing Apple Immersive Video content.
Re-arrangeable Home View
Originally, users of the Apple Vision Pro couldn't change where icons were placed in the Home View. In vision OS 2, users can rearrange the apps to wherever they want.
Volumetric APIs
APIs for developers to help improve the development of apps that can run side-by-side with others in a shared space. This includes fixed and dynamic scale states, resizing volumes with a SwiftUI scene modifier, and affixable ornaments.
TabletopKit
A framework for developers to more easily make collaborative experiences based around a table. It can handle the manipulation of cards and pieces, placements and layouts, and defining game boards.
This can be an extremely useful addition for game developers modelling board games for a spatial computing environment.
Enterprise APIs
New APIs that provide more enhanced sensor access and increased control, specifically for enterprise app creation. This includes access to the main camera, spatial barcode and QR code scanning, and use of the Neural Engine.
Mac Virtual Display enhancements
Mac Virtual Display will be expandable into an ultrawide screen that wraps around the user. It will be the equivalent of having two 4K-resolution displays side-by-side.
The new Mac Virtual Display will be a larger screen in visionOS 2
For Mac users, this will mean they will benefit from a larger contiguous workspace when using their headset.
This is not arriving with visionOS 2.0, but instead, later in the year.
New gestures
Users can quickly access the Home View with their hands, see the date and time, control volume, and check the battery level. Control Center can also be more easily opened, so notifications and frequently-used items can be quickly used.
An open face-up palm followed by a pinch can summon the Home View. Rotating the hand from this position so it is face down displays a floating clock next to the user's thumb.
While in this open palm face-down position, the user can pinch to summon a Control Center pop-up they can scroll through to access more features. It also includes the battery and Wi-Fi signal level, the date and time, and the volume control.
HealthKit
HealthKit, Apple's framework for collecting and managing health-related data points of users, is coming to visionOS.
Recent Guest User
Guest User provides a way for others to try out the Apple Vision Pro. Recent Guest User can save the most recent guest's eye and hand data, held on-device for up to 30 days.
By doing this, the guest user the data relates to can skip the setup process and go back to experiencing the headset the next time they wear it. This will significantly reduce the time from donning the headset to actually using it, and is closer to a shared single headset amongst multiple family members.
Keyboard breakthrough
If you're immersed in an Environment, visionOS will make it so that the Magic Keyboard or MacBook keyboard is still visible to the user.
Keyboard breakthrough will show your keyboard while in an immersive environment.
Users won't have to blindly fumble around to find their keyboard, nor break immersion to use it.
Follow your breathing
The Mindfulness app follows the user's breathing. The Apple Vision Pro senses the user's breathing rate and matches the visualizations within a Mindfulness session.
AppleInsider previously reported on respiration tracking in visionOS, with several references discovered in the software ahead of WWDC.
SharePlay for Photos
SharePlay in the Photos app can be used with other users you're communicating with via their spatial Persona. This lets them see your photos and spatial photos.
Mouse support
You can connect a Bluetooth mouse or a Magic Mouse to the Apple Vision Pro. This may seem like an odd addition for a gesture-heavy device, but it does make sense to those who work using the headset.
For example, if you're editing a text document in an app, some will find it easier to navigate the 2D file's text using a mouse instead of using their fingers.
AirPlay Receiver
Users can enable the Apple Vision Pro as an AirPlay Receiver. Content from an iPhone or iPad on the same network can be viewed from within the headset.
This gives Apple Vision Pro users an opportunity to see what others share with them using AirPlay from their devices, without necessarily having to take off the headset to view a nearby AirPlay-enabled screen.
Read on AppleInsider
Comments
I wonder if there is teh option to fix a window to be relative to the headset, meaning it will always be visible, even turning ones head.
I have a medical app idea, that would absolutely need this.
As I recall, there was something in place to prevent using Vision Pro while in motion, because it was inevitable that some moron would try to use it while driving. I would guess that this just allows that to be a setting.
The VisionOS, being spatial aware, tries to keep what you see "fixed" in space. So if you are in a moving vehicle, everything you see would zoom past you as it tried to stay where it originally was. Travel model makes the vehicle the fixed space, so what you see stays fixed with that instead of the larger world space.
I trust that they will come up with a solution to the menu bar for the Mac Virtual Display. If you're working on an app placed to the right, but need to go all the way to the top-left to access the File or Edit menus, that would be super annoying. I found that annoying on large displays already, so I hope that they address that before release. Maybe the menu bar floats above the currently-active window instead of being attached to the top of the full display space?
https://www.youtube.com/watch?v=IRxPMhbxWi0
https://github.com/alvr-org/alvr
https://www.youtube.com/watch?v=U1R8r4BJwPo
https://www.reddit.com/r/VisionPro/comments/1bmn6gd/getting_started_with_alvr_on_vision_pro/
I’ve noticed with 2.0, when moving about, a window pops up now to ask if you are traveling. One of the issues people on planes complained of is not setting travel mode before they leave the gate. Once in the air, they couldn’t use the VP. Unless I missed it, the “are you traveling” pop-up is now and a nice addition.
On another note - Bora Bora. We’ve been waiting for this one to come out for a while. Now that it’s here, have to say I now use Environments. It’s SO RELAXING there are times I just sit there and veg out. Great when there’s a nice breeze outside. Feels like you are really there. Get a little sand box to complete the feeling. Amazing work that one.