New visionOS 2 updates are more significant than Apple initially implied

Posted:
in visionOS edited June 12

Apple's unveiling of visionOS 2 ended with its bento image summarizing its features. Here's what that graphic said, and what it means for the Apple Vision Pro.

Man wearing a VR headset, surrounded by colorful app icons and widgets in the background.
An Apple Vision Pro user in front of the visionOS bento graphic



Like its other operating systems, Apple finished its exploration of visionOS 2 and its inbound changes with a bento graphic. Named after the compartmentalized food container, the bento image has been Apple's go-to for showing as many features as possible about a product or service.

While it does highlight a large number of features, it doesn't do so in detail. Instead, it's only got a few words on each.

visionOS 2.0 features including spatial photos, volumetric APIs, immersive video, airway receiver, new gestures, health monitoring, and more displayed in a grid format.
Apple's 'Bento' image for visionOS 2.0



Here's what's coming in visionOS 2.0

Create spatial photos



Under visionOS 2, users can create spatial photos from the Photos app. Machine learning converts a 2D image to a 3D version, viewable as a spatial photo.

We've tried this already, and it's impressive. It's not the same as taking a photo natively on Apple Vision Pro or on an iPhone 15 Pro, but those pictures you took a long time ago can get a very nice 3D treatment.

Bora Bora environment



Users get a new view for the Apple Vision Pro. The Bora Bora environment is a beach scene with blue waters and distant mountainous views.

Multiview in TV app



Viewers of Major League Soccer and Major League Baseball can watch multiple games simultaneously. This is already a feature of tvOS and iPadOS, with it being an expansion to the headset.

Screen displaying multiple live basketball games with players in action, featuring teams like Golden State Warriors, Los Angeles Lakers, New York Knicks, Dallas Mavericks, and more.
Multiview of MLB and MLS will be available. Apple previewed it showing basketball.



Five different games can be viewed in five separate views, with a main view switchable with games in four smaller views.

Travel Mode on trains



Previously, as we'd discussed, Apple Vision Pro travel mode didn't work at all on trains. Apple took that feedback to heart, apparently.

Train support in Travel Mode will allow the Apple Vision Pro to be used while on the railways of the world.

New Apple Immersive Video



More Apple Immersive Video titles will be arriving in the future. These are 180-degree field-of-view films that leverage 3D 8K video.

The existing roster of footage includes a lot of action shorts, including a Parkour run in Paris. During the event, Apple teased more is on the way from Red Bull, and what appears to be a submarine-based one, which we will get our sub service vet Mike Wuerthele to assess for you when it's available.

Apple also took a moment to reveal work by Blackmagic Design to create en end-to-end solution for producing Apple Immersive Video content.

Re-arrangeable Home View



Originally, users of the Apple Vision Pro couldn't change where icons were placed in the Home View. In vision OS 2, users can rearrange the apps to wherever they want.

Volumetric APIs



APIs for developers to help improve the development of apps that can run side-by-side with others in a shared space. This includes fixed and dynamic scale states, resizing volumes with a SwiftUI scene modifier, and affixable ornaments.

TabletopKit



A framework for developers to more easily make collaborative experiences based around a table. It can handle the manipulation of cards and pieces, placements and layouts, and defining game boards.

This can be an extremely useful addition for game developers modelling board games for a spatial computing environment.

Enterprise APIs



New APIs that provide more enhanced sensor access and increased control, specifically for enterprise app creation. This includes access to the main camera, spatial barcode and QR code scanning, and use of the Neural Engine.

Mac Virtual Display enhancements



Mac Virtual Display will be expandable into an ultrawide screen that wraps around the user. It will be the equivalent of having two 4K-resolution displays side-by-side.

Person wearing a VR headset, sitting at a table with multiple virtual screens displaying video editing software, notes, and data in a modern, well-lit living room.
The new Mac Virtual Display will be a larger screen in visionOS 2



For Mac users, this will mean they will benefit from a larger contiguous workspace when using their headset.

This is not arriving with visionOS 2.0, but instead, later in the year.

New gestures



Users can quickly access the Home View with their hands, see the date and time, control volume, and check the battery level. Control Center can also be more easily opened, so notifications and frequently-used items can be quickly used.

An open face-up palm followed by a pinch can summon the Home View. Rotating the hand from this position so it is face down displays a floating clock next to the user's thumb.

While in this open palm face-down position, the user can pinch to summon a Control Center pop-up they can scroll through to access more features. It also includes the battery and Wi-Fi signal level, the date and time, and the volume control.

HealthKit



HealthKit, Apple's framework for collecting and managing health-related data points of users, is coming to visionOS.

Recent Guest User



Guest User provides a way for others to try out the Apple Vision Pro. Recent Guest User can save the most recent guest's eye and hand data, held on-device for up to 30 days.

By doing this, the guest user the data relates to can skip the setup process and go back to experiencing the headset the next time they wear it. This will significantly reduce the time from donning the headset to actually using it, and is closer to a shared single headset amongst multiple family members.

Keyboard breakthrough



If you're immersed in an Environment, visionOS will make it so that the Magic Keyboard or MacBook keyboard is still visible to the user.

App icons displayed against a background of a keyboard and mountainous landscape with a sunrise sky. Icons include TV, Music, Safari, Photos, Notes, Mail, Messages, and App Store.
Keyboard breakthrough will show your keyboard while in an immersive environment.



Users won't have to blindly fumble around to find their keyboard, nor break immersion to use it.

Follow your breathing



The Mindfulness app follows the user's breathing. The Apple Vision Pro senses the user's breathing rate and matches the visualizations within a Mindfulness session.

AppleInsider previously reported on respiration tracking in visionOS, with several references discovered in the software ahead of WWDC.

SharePlay for Photos

SharePlay

in the Photos app can be used with other users you're communicating with via their spatial Persona. This lets them see your photos and spatial photos.

Mouse support



You can connect a Bluetooth mouse or a Magic Mouse to the Apple Vision Pro. This may seem like an odd addition for a gesture-heavy device, but it does make sense to those who work using the headset.

For example, if you're editing a text document in an app, some will find it easier to navigate the 2D file's text using a mouse instead of using their fingers.

AirPlay Receiver



Users can enable the Apple Vision Pro as an AirPlay Receiver. Content from an iPhone or iPad on the same network can be viewed from within the headset.

This gives Apple Vision Pro users an opportunity to see what others share with them using AirPlay from their devices, without necessarily having to take off the headset to view a nearby AirPlay-enabled screen.



Read on AppleInsider

Comments

  • Reply 1 of 17
    robin huberrobin huber Posts: 3,991member
    What does Travel Mode do exactly?
    VictorMortimer
  • Reply 2 of 17
    cpsrocpsro Posts: 3,212member
    How about modifying Apple's bento image to show just the features available with the first developer beta?
    williamlondon
  • Reply 3 of 17
    mjpbuymjpbuy Posts: 22member
    Travel Mode, I guess, is while traveling your windows don't get left behind.  The Apple Vision fixes windows in space that stay in place relative to the room you are in, not to the headset...
    I wonder if there is teh option to fix a window to be relative to the headset, meaning it will always be visible, even turning ones head.
    I have a medical app idea, that would absolutely need this.
    beowulfschmidtjas99Bart Y
  • Reply 4 of 17
    What does Travel Mode do exactly?
    Nothing 
    williamlondon
  • Reply 5 of 17
    What does Travel Mode do exactly?

    As I recall, there was something in place to prevent using Vision Pro while in motion, because it was inevitable that some moron would try to use it while driving.  I would guess that this just allows that to be a setting.
  • Reply 6 of 17
    What does Travel Mode do exactly?
    Vision Pro is spatially aware and keeps windows in the space where they were opened. So for example I can watch a movie in my living room, get up and walk to my office and answer an email then return to my living room and resume watching the move. Both the movie and email apps stay open but they are in different locations. This becomes an issue if your physical location is moving as your app windows would all be left behind. Travel mode compensates for the movement and holds the app windows in a static place while. 

    It also helps avoid motion sickness. Motion sickness happens when what you are looking at  doesn’t match the movement your body is feeling. The AVP will sense the movement and adjust what you are looking at to match what you are feeling. 
    edited June 13 gilly33jas99Bart Y
  • Reply 7 of 17
    csimmonscsimmons Posts: 101member
    I’m surprised that nobody is talking more about the keyboard breakthrough and improved virtual display capabilities, since these features alone would make a smaller, cheaper, stand-alone Apple Vision Display headset a viable product. 
    jas99Bart Y
  • Reply 8 of 17
    9secondkox29secondkox2 Posts: 2,904member
    Kitchen sink stuff. 

    Sounds like an internal alarm went off and Tim may have given some ultimatums to make this thing compelling. End result: Great for customers. 
  • Reply 9 of 17
    coolfactorcoolfactor Posts: 2,294member
    What does Travel Mode do exactly?

    The VisionOS, being spatial aware, tries to keep what you see "fixed" in space. So if you are in a moving vehicle, everything you see would zoom past you as it tried to stay where it originally was. Travel model makes the vehicle the fixed space, so what you see stays fixed with that instead of the larger world space.
    gilly33thtBart Y
  • Reply 10 of 17
    coolfactorcoolfactor Posts: 2,294member

    I trust that they will come up with a solution to the menu bar for the Mac Virtual Display. If you're working on an app placed to the right, but need to go all the way to the top-left to access the File or Edit menus, that would be super annoying. I found that annoying on large displays already, so I hope that they address that before release. Maybe the menu bar floats above the currently-active window instead of being attached to the top of the full display space?
    edited June 13
  • Reply 11 of 17
    Kitchen sink stuff. 

    Sounds like an internal alarm went off and Tim may have given some ultimatums to make this thing compelling. End result: Great for customers. 
    Or, this is like any other product Apple has released and they are doing natural iterations to improve it. 

    But after you countless bad takes on AVP I can see why you are wanting to spin it that way. 
    williamlondonmacgui9secondkox2mattinozBart Y
  • Reply 12 of 17
    MacProMacPro Posts: 19,787member
    Now if I could just use it to run MS Flight Simulator!
  • Reply 13 of 17
    thttht Posts: 5,544member
    Seems all but inevitable that iPhone virtual display and integration is coming to visionOS and iPadOS. 

    This would be one of those features that keeps you wearing goggles as the resolution of the Vision Pro displays aren’t there yet, especially at 15” focal distances. 

    They have to add Terminal.app or a DarwinOS/XNU VM app to both visionOS and iPadOS. 
  • Reply 14 of 17
    9secondkox29secondkox2 Posts: 2,904member
    Kitchen sink stuff. 

    Sounds like an internal alarm went off and Tim may have given some ultimatums to make this thing compelling. End result: Great for customers. 
    Or, this is like any other product Apple has released and they are doing natural iterations to improve it. 

    But after you countless bad takes on AVP I can see why you are wanting to spin it that way. 
    You mean honest takes. And no, since mommy and daddy aren’t here to correct you, apple hasn’t historically thrown the kitchen sink in v2 products. Case in point: ipad j taking 14 years to get a native calculator and calendar taking years to finally communicate with reminders. Talk about bad takes…
    edited June 14
  • Reply 15 of 17
    ohcomeonohcomeon Posts: 31member
    Still no download for Apple TV content?
  • Reply 16 of 17
    MarvinMarvin Posts: 15,406moderator
    MacPro said:
    Now if I could just use it to run MS Flight Simulator!
    According the following, it can be done using ALVR installed onto the headset using testflight, most Steam VR games are supported, including Half-Life Alyx:

    https://www.youtube.com/watch?v=IRxPMhbxWi0
    https://github.com/alvr-org/alvr
    https://www.youtube.com/watch?v=U1R8r4BJwPo

    https://www.reddit.com/r/VisionPro/comments/1bmn6gd/getting_started_with_alvr_on_vision_pro/
  • Reply 17 of 17
    What does Travel Mode do exactly?
    Keeps windows/apps you open in place. Because the VP is spacially aware, when you open a window/app in a particular space, it keeps it there. If you are traveling in a plane, car, train, you may not be able to use the headset unless travel mode is on. Example - you open apps, because you are moving, they disappear behind you before you can do anything with them. 

    I’ve noticed with 2.0, when moving about, a window pops up now to ask if you are traveling. One of the issues people on planes complained of is not setting travel mode before they leave the gate. Once in the air, they couldn’t use the VP. Unless I missed it, the “are you traveling” pop-up is now and a nice addition.

    On another note - Bora Bora. We’ve been waiting for this one to come out for a while. Now that it’s here, have to say I now use Environments. It’s SO RELAXING there are times I just sit there and veg out. Great when there’s a nice breeze outside. Feels like you are really there. Get a little sand box to complete the feeling. Amazing work that one.  
Sign In or Register to comment.