Apple considered VR controllers & rings for Vision Pro, but didn't like the ideas

Comments

• Reply 1 of 8

  Xed Posts: 2,690member

  July 2, 2023 2:34PM

  Just like with the Apple Pencil and the iPad, I don't expect an auxiliary device will be the default use for Vision Pro, but I do expect it to be exist. There are simply too many use cases where even the most advanced finger tracking can't do everything one might need in a VR or AR environment.

  
• Reply 2 of 8

  retrogusto Posts: 1,121member

  July 2, 2023 5:20PM

  The eye+gesture control appears to be one of the best things about Vision Pro, but I can see something like wireless haptic gloves with small haptic clickers built into the fingertips coming out eventually, either from Apple or a third party.

  
• Reply 3 of 8

  tht Posts: 5,536member

  July 2, 2023 9:03PM

  retrogusto said:

  The eye+gesture control appears to be one of the best things about Vision Pro, but I can see something like wireless haptic gloves with small haptic clickers built into the fingertips coming out eventually, either from Apple or a third party.

  The hand tracking pushes against the idea of having hand controllers, especially for Apple. The current implementation is not fast enough for fast twitch games, but if Apple continues to improve hand tracking, I think the will continue to view it as the best option. Say, the latency on hand tracking gets down to 5 ms (200 Hz). If it gets that fast, you won't need controllers to play a fast twitch game.
  
  Camera and IR frame rates would have to go up to 400 to 500 Hz. So, probably a doubling to tripling from where the VP is at now. It isn't going to be cheap computationally or power efficient, but I would definitely prefer them to push hand and eye tracking to the point below human perception. It would benefit all applications, not just games.

  
• Reply 4 of 8

  Mariner8 Posts: 33member

  July 3, 2023 4:00AM

  I’ll take another run at this, expanding a bit on a previous post. 

  
  

  Think about the thumb-finger tap as a secondary ‘single’ switch, locking in or activating a primary selection that’s made by eye-gaze, timing (e.g. capturing the moment when a target comes into view or moves into the crosshairs, or when a rolling highlight hits the desired selection in a drop-down or matrix menu).  

  
  

  Consider that *any* single-switch has exactly the same function and can accomplish the same purpose as a thumb-finger tap.  Under what circumstances would this be useful?  Well, if your fingers don’t work but your toes still do, that would be useful (try it now).  If your fingers and toes don’t work but your tongue still does, that would be useful (try making a little ‘tetch’ sound with your tongue now).  

  
  

  There are scores of single-switches that have been developed and used by people with disabilities for over 40 years.  Think about Steven Hawking writing A Brief History of Time with a single switch (Walt and Ginger Waltoz built that solution to run on an off-the-shelf laptop instead of a $7,000 AAC device).  Run a Google or Chat GPT search on the phrase [disability single-switch interface]. 

  
  

  Speech-to-text and text-to-speech were developed in the disability field and then adopted for use by ‘the rest of us’.  The first full-fledged Dragon Dictate system cost many thousands of dollars, and its successors are now a free component of every major operating system, including devices in the $50-$100 range. I just filled in a YouTube search on my TV set by speaking into an Apple TV controller. 
  
  Think about sound-activated switches, sip-and-puff switches, proximity and capacitance switches (you may have used one the last time you got on an elevator), voice-powered switches, and coming up, EEG switches. The interesting thing here is that *any* single switch can be linked to any device through a universal port (there’s at least one on the device that you’re using to read this right now). Apple in particular has added dozens of accessibility and switching features to its standard iOS and OS systems.  Adding that capacity to the Vision Pro should be a cinch. 

  
  

  David Wetherow 
  The Star Raft Project 

  
  

  
  

  edited July 2023
• Reply 5 of 8

  beowulfschmidt Posts: 2,232member

  July 3, 2023 12:12PM

  tht said:

  retrogusto said:

  The eye+gesture control appears to be one of the best things about Vision Pro, but I can see something like wireless haptic gloves with small haptic clickers built into the fingertips coming out eventually, either from Apple or a third party.

  The hand tracking pushes against the idea of having hand controllers, especially for Apple. The current implementation is not fast enough for fast twitch games, but if Apple continues to improve hand tracking, I think the will continue to view it as the best option. Say, the latency on hand tracking gets down to 5 ms (200 Hz). If it gets that fast, you won't need controllers to play a fast twitch game.
  
  Camera and IR frame rates would have to go up to 400 to 500 Hz. So, probably a doubling to tripling from where the VP is at now. It isn't going to be cheap computationally or power efficient, but I would definitely prefer them to push hand and eye tracking to the point below human perception. It would benefit all applications, not just games.

  
  I would expect the vast majority of current fast twitch gamers are going to insist on either a controller or keyboard/mouse.  The advantage of such is that there is no uncertainty at all as to which control is being pressed, even if one's hand might be in a slightly different position.  I'm not sure hand tracking will have that kind of precision and lack of uncertainty anytime soon.  I do think it's a solvable problem, but the current generation of gamers isn't going to want to throw away all the work they've put into developing muscle memory, especially since Apple is still showing only lukewarm interest in any kind of high end, fast twitch games.  Of course, based on some numbers I saw recently, the mobile game market is larger than all the console markets put together, so I'm not sure they really need "high end" games.
  

  
• Reply 6 of 8

  eriamjh Posts: 1,681member

  July 3, 2023 1:14PM

  I don’t think the finger touch is meant to replace controllers.   Doesn’t apple show someone using a controller in their own promo shots?   

  
  

  This is about stand-alone use.   Apple can easily add BT click-mouse device-like support, voice-select, tongue-select, and many other methods.   

  
  

  It’s the dawn of a new age.   Let some light shine before running off into the dimness of it.  

  
• Reply 7 of 8

  jahblade Posts: 159member

  July 3, 2023 2:32PM

  I am truly excited to see what third-party hardware accessories developers come up with. Very exciting times to come!

  
• Reply 8 of 8

  Xed Posts: 2,690member

  July 9, 2023 11:55AM

  chutzpah said:

  Xed said:

  ATLMacFan said:

  Ya’ll deleted my comment about your typo, but left the typo.
  
  ”Apple reportedly company considered.”
  
  Again, I ask, where are your editors?

  I've never understood why someone would make an account and post just to point out a fucking typo. Did you understand what was being stated? Do you really get a feeling of superiority from noting the typo? Do you go to sleep thinking "I could make a better rumors website than AppleInsider because I wouldn't ever publish typos"?

  Let's all tolerate slopping editing in online journalism.

  Then send a private message to the editor if the error is that egregious to you—I’ve done that when something was woefully incorrect that it altered how the piece reads—but to create an account and then come find the typo in the article is fucking pathetic. Now the thread is permanently jacked because some you had to note that the foolish author wrote ‘teh ’ instead of ‘the’, for example. And for what? So you can prove how you are superior to the author? At least the author is contributing something..

  edited July 2023