Apple has reportedly approached Sony to partner on Apple Vision Pro controllers

AppleInsiderAppleInsider
Posted:
in Apple Vision Pro edited December 2024

A new report claims Apple has approached Sony in an effort to bring handheld game controller support -- and presumably more games -- to the Apple Vision Pro.

Sleek black virtual reality headset resting on a white surface, with blurred purple background.
VR hand controller support could revitalize the software market for the Apple Vision Pro.
Bloomberg

reports that a future update to visionOS will bring support for third-party hand controllers, including Sony's PlayStation VR2 controllers. Apple has also reached out to third-party developers in an effort to bring more games to the Apple Vision Pro, with support for controllers as an inducement.

Traditional game controllers used with the Xbox and PlayStation are currently supported in the Apple Vision Pro, but only for Apple Arcade games. Bringing support for VR controllers, and encouraging developers to support more VR games for the platform, would likely increase sales of Apple's headset.



Supporting hand controllers could augment Apple's own eye-and-hand movement detection, attracting more development for games and other applications. Bloomberg has also speculated that Apple could go as far as helping fund more programs for the Apple Vision Pro in an effort to develop more momentum for the product.

The Apple Vision Pro has sold modestly, as might be expected from an all-new product category without an existing mature market. Apple is believed to have sold around 370,000 units in the first three quarters of sales, with another 50,000 units expected by the end of 2024.

More importantly, people who own an Apple Vision Pro are not using it as much as the company had expected, indicating a need to beef up software options for it. The company has until recently been focused on bringing its own Apple Pencil-like "wand" controller to the Apple Vision Pro for more precise types of work, but now sees value in adapting VR hand controllers to the platform as well.

Rumor Score: Possible

Read on AppleInsider

Comments

  • Reply 1 of 9
    MarvinMarvin Posts: 15,515moderator

    A new report claims Apple has approached Sony in an effort to bring handheld game controller support -- and presumably more games -- to the Apple Vision Pro. 

    VR hand controller support could revitalize the software market for the Apple Vision Pro.

    Top 10 VR games on Playstation reported by Sony for 2023 are on the following page:

    https://www.roadtovr.com/sony-reveals-the-10-most-downloaded-psvr-2-games-in-2023/

    These all probably work better with controllers, many simulate holding weapons or objects. The PSVR2 controllers are a bit bulky though:



    There are more compact ones that do full finger tracking:

    https://eteexr.com/



    Apple could design their own compact controllers like these with full tracking and touch controls. It would be good if they could have wristbands that could track interaction with any passive objects. Then people could hold plastic replicas of any object.

    Since games are already made to support the Sony controllers and millions of people already own them, it would help to have support for them too.

    There are controllers here designed for AVP that have a nicer design, Apple could probably buy this company out or just sell their products in store:

    https://www.surreal-interactive.com/
    https://www.kickstarter.com/projects/surreal-interactive/surreal-touch-1-controllers-for-apple-vision-pro





    If Apple partners with Valve for SteamVR, AVP will be compatible with thousands of games, initially via streaming and then with native ports.
    watto_cobra
  • Reply 2 of 9
    mpantonempantone Posts: 2,274member
    One very important thing hand VR controllers offer over controller-free hand gestures is haptic feedback.

    For gaming this can be pretty important depending on the title. Even if you are grabbing something, the game developer can program some sort of feedback.

    There are other controller features like gyroscopes, adaptive triggers and more. These are some of the same things that people playing with mouse & keyboard miss out on. There's really no way to implement an adaptive trigger for a standard keyboard and mouse combination.

    Gyroscopic input is also extremely useful for certain situations. Many driving (and some flight) simulation games in VR utilize a gamepad as the handheld input device that essentially functions as a steering wheel. If you are just using hand gestures, you don't have a credible input experience just by grabbing some image projected in air.

    While controller-less gesturing might be very freeing in some VR situations, often there's a benefit to having some sort of haptic feedback especially if you are simulating grabbing some object with your hand.

    AVP has a looooooooong way to go before it escapes being a niche device. Even the controllers for my old Oculus Rift S VR HMD have vibration motors (just like a gamepad).
    edited December 2024 watto_cobra
  • Reply 3 of 9
    apple4thewinapple4thewin Posts: 341member
    Wow can you imagine the PS 6 comes out and the psvr3 is actually an Apple headset 🤯. The main gripe I hear about the psvr2 controllers is that is was made for smaller hands, but the adaptive triggers are great so if Apple and Sony create a new controller for both the VP and PSVR it would possibly be the new industry standard (I have a very positive outlook in the future for unlikely things)
    watto_cobra
  • Reply 4 of 9
    thttht Posts: 5,767member
    Hopefully, the deal primarily involves Sony porting PS5 games to Vision Pro and Mac.  :D

    This looks like the slow incremental addition of Vision features Apple has been doing for the past year, filling out the feature set for visionOS. A nice to have, another reason to use the AVP, but Apple will have to subsidize games for awhile.

    A very long road to go. They still need to add:

    1. iPhone Virtual Display or iPhone Mirroring for visionOS
    2. Xcode, Terminal, and the XQuartz equivalent.
    3. Update the hardware. They really should update the AVP to M4, 16/32 GB, 0.5/1/2 TB, 12 MP main view cameras, WiFi 7, etc.
    4. Continue with Continuity features as much as possible. 
    5. Subsidize content.
    apple4thewinpulseimageswatto_cobra
  • Reply 5 of 9
    chasmchasm Posts: 3,654member
    tht said:
    A very long road to go. They still need to add:

    1. iPhone Virtual Display or iPhone Mirroring for visionOS
    2. Xcode, Terminal, and the XQuartz equivalent.
    3. Update the hardware. They really should update the AVP to M4, 16/32 GB, 0.5/1/2 TB, 12 MP main view cameras, WiFi 7, etc.
    4. Continue with Continuity features as much as possible. 
    5. Subsidize content.
    Good analysis.

    1. This first item is incredibly likely to arrive within the next few months.
    2. I would guess Terminal is not coming soon, but Xcode/XQuartz should be.
    3. I think that’s going to be announced at WWDC. On the outside not very different-looking, but with all or most of what you describe.
    4. Amen
    5. This is an area that is sorta new for Apple in terms of giving other big companies bribe money for development, but I concur that if they want new content fast, that needs to happen.
    watto_cobra
  • Reply 6 of 9
    danoxdanox Posts: 3,479member
    tht said:
    Hopefully, the deal primarily involves Sony porting PS5 games to Vision Pro and Mac.  :D

    This looks like the slow incremental addition of Vision features Apple has been doing for the past year, filling out the feature set for visionOS. A nice to have, another reason to use the AVP, but Apple will have to subsidize games for awhile.

    A very long road to go. They still need to add:

    1. iPhone Virtual Display or iPhone Mirroring for visionOS
    2. Xcode, Terminal, and the XQuartz equivalent.
    3. Update the hardware. They really should update the AVP to M4, 16/32 GB, 0.5/1/2 TB, 12 MP main view cameras, WiFi 7, etc.
    4. Continue with Continuity features as much as possible. 
    5. Subsidize content.
    Number five is dead on arrival, Apple is not going to pay somebody else for game content if they don’t want to use or release product for their platforms and if they do decide to do something, they should be looking in the 3-D graphics area and in the professional CAD area i.e. AutoCad, Revit, and Navis Manage the current hardware is more than capable, particularly the MacBook Pro laptops and the Mac Studios both of which are ideal for on the job site use due to performance, size and power usage.

    No I don’t think Apple would do this either for the same reason but one can only hope….
    Alex_Vwatto_cobra
  • Reply 7 of 9
    thttht Posts: 5,767member
    danox said:
    tht said:
    Hopefully, the deal primarily involves Sony porting PS5 games to Vision Pro and Mac.  :D

    This looks like the slow incremental addition of Vision features Apple has been doing for the past year, filling out the feature set for visionOS. A nice to have, another reason to use the AVP, but Apple will have to subsidize games for awhile.

    A very long road to go. They still need to add:

    1. iPhone Virtual Display or iPhone Mirroring for visionOS
    2. Xcode, Terminal, and the XQuartz equivalent.
    3. Update the hardware. They really should update the AVP to M4, 16/32 GB, 0.5/1/2 TB, 12 MP main view cameras, WiFi 7, etc.
    4. Continue with Continuity features as much as possible. 
    5. Subsidize content.
    Number five is dead on arrival, Apple is not going to pay somebody else for game content if they don’t want to use or release product for their platforms and if they do decide to do something, they should be looking in the 3-D graphics area and in the professional CAD area i.e. AutoCad, Revit, and Navis Manage the current hardware is more than capable, particularly the MacBook Pro laptops and the Mac Studios both of which are ideal for on the job site use due to performance, size and power usage.

    No I don’t think Apple would do this either for the same reason but one can only hope….
    Yeah. Hope springs eternal. I didn't mean games only either. Subsidize everything.

    They need to subsidize workstation software for macOS, visionOS, and even iPadOS. It just isn't naturally going to come to macOS. They always eschewed the market, but the hardware is so good now that they have a chance, but it needs to be kickstarted. They really didn't have the hardware before, or hardware in the manner they like before, to serve this niche. Now, a 128 GB M4 Max and presumably a 256 GB M4 Ultra would make for great machines to do GPU and NPU compute.

    For gaming, going the SteamDeck route is basically their only option now? Seems hopeless to have games programmed to use macOS APIs, and they will have to be able translate Windows games in order to at least develop a set of users who buys games, spends money in games to attract real ports or have games developed natively.

    For the AVP, content includes things like graphic novels, videos, audio, and apps. The hardware needs to be updated though. M4 32 GB RAM should really be the base model now. Drop the M2 16 GB RAM model to $3000. Next major revision of the AVP has to improve wearability. It has to be trivial to put on and off. It has to be light enough, breathable enough to wear all the time. It would take off unless someone can just slip it on in a second, half it hanging around your neck like headphones when it is not worn, etc.
    watto_cobra
  • Reply 8 of 9
    danoxdanox Posts: 3,479member
    tht said:
    danox said:
    tht said:
    Hopefully, the deal primarily involves Sony porting PS5 games to Vision Pro and Mac.  :D

    This looks like the slow incremental addition of Vision features Apple has been doing for the past year, filling out the feature set for visionOS. A nice to have, another reason to use the AVP, but Apple will have to subsidize games for awhile.

    A very long road to go. They still need to add:

    1. iPhone Virtual Display or iPhone Mirroring for visionOS
    2. Xcode, Terminal, and the XQuartz equivalent.
    3. Update the hardware. They really should update the AVP to M4, 16/32 GB, 0.5/1/2 TB, 12 MP main view cameras, WiFi 7, etc.
    4. Continue with Continuity features as much as possible. 
    5. Subsidize content.
    Number five is dead on arrival, Apple is not going to pay somebody else for game content if they don’t want to use or release product for their platforms and if they do decide to do something, they should be looking in the 3-D graphics area and in the professional CAD area i.e. AutoCad, Revit, and Navis Manage the current hardware is more than capable, particularly the MacBook Pro laptops and the Mac Studios both of which are ideal for on the job site use due to performance, size and power usage.

    No I don’t think Apple would do this either for the same reason but one can only hope….
    Yeah. Hope springs eternal. I didn't mean games only either. Subsidize everything.

    They need to subsidize workstation software for macOS, visionOS, and even iPadOS. It just isn't naturally going to come to macOS. They always eschewed the market, but the hardware is so good now that they have a chance, but it needs to be kickstarted. They really didn't have the hardware before, or hardware in the manner they like before, to serve this niche. Now, a 128 GB M4 Max and presumably a 256 GB M4 Ultra would make for great machines to do GPU and NPU compute.

    For gaming, going the SteamDeck route is basically their only option now? Seems hopeless to have games programmed to use macOS APIs, and they will have to be able translate Windows games in order to at least develop a set of users who buys games, spends money in games to attract real ports or have games developed natively.

    For the AVP, content includes things like graphic novels, videos, audio, and apps. The hardware needs to be updated though. M4 32 GB RAM should really be the base model now. Drop the M2 16 GB RAM model to $3000. Next major revision of the AVP has to improve wearability. It has to be trivial to put on and off. It has to be light enough, breathable enough to wear all the time. It would take off unless someone can just slip it on in a second, half it hanging around your neck like headphones when it is not worn, etc.
    I can’t disagree Apple’s present trajectory with Apple Silicon will take it closer to convergence to Nvidia. 

    https://www.youtube.com/watch?v=5dhuxRF2c_w @ 12:50 says it all from a power usage standpoint Apple is on the right path with their designs (the PC users in the comments section are hopping mad I believe they know the convergence is coming soon M5?).
    watto_cobra
  • Reply 9 of 9
    alandailalandail Posts: 776member
    mpantone said:
    One very important thing hand VR controllers offer over controller-free hand gestures is haptic feedback.

    For gaming this can be pretty important depending on the title. Even if you are grabbing something, the game developer can program some sort of feedback.

    There are other controller features like gyroscopes, adaptive triggers and more. These are some of the same things that people playing with mouse & keyboard miss out on. There's really no way to implement an adaptive trigger for a standard keyboard and mouse combination.

    Gyroscopic input is also extremely useful for certain situations. Many driving (and some flight) simulation games in VR utilize a gamepad as the handheld input device that essentially functions as a steering wheel. If you are just using hand gestures, you don't have a credible input experience just by grabbing some image projected in air.

    While controller-less gesturing might be very freeing in some VR situations, often there's a benefit to having some sort of haptic feedback especially if you are simulating grabbing some object with your hand.

    AVP has a looooooooong way to go before it escapes being a niche device. Even the controllers for my old Oculus Rift S VR HMD have vibration motors (just like a gamepad).
    Haptics, higher speed/accuracy in tracking, and ability to track out of the view of the headset are all reasons to support controllers.

    So many fitness and gaming apps for other platforms that won't work on Vision Pro without controllers, including racket games, bow and arrow games, etc. It's frustrating to own a Vision Pro and have to use a quest 3 for fitness/gaming. Even an app like synth riders, which does work on a Vision Pro, and looks much better on a vision pro, is far more playable on quest because of the controllers.
    watto_cobra
Sign In or Register to comment.