Apple Vision Pro may help users navigate with directional audio cues

Posted:
in General Discussion edited July 25

The forthcoming Apple Vision Pro, regular iPhones, or even the Apple Car, could use spatial audio to get users to turn toward where a sound seems to be coming from.

Detail from the patent showing a rather slim Apple Vision Pro
Detail from the patent showing a rather slim Apple Vision Pro



We're already used to Siri telling us when to turn left or right, whether it's through CarPlay, our AirPods, or just our regular iPhone. That means we're also used to podcasts being paused and music volume dipping, but now Apple wants to improve on all of this -- and maybe when we're just walking around, too.

From not interrupting us, to giving us better directions, Apple Vision Pro and more Apple devices in the future may intentionally alter where what we're listening to appears to be coming.

"Spatial Audio Navigation," is a newly-granted patent that sets out to describe how audio can be used to guide users in the real world or with AR and Mixed Reality (MR).

"Conventional navigation applications may provide vocal directions such as 'turn left' or 'turn right' to direct users to destinations," says the patent application. "However, a user, for example while walking or riding a bike, may be listening to an audio source (e.g., music, an audio book, a phone call, etc.) while using a navigation application, and may prefer for the audio not to be interrupted by vocal directions."

"In addition," it continues, "when on a phone call, vocal directions can interfere with the conversation, and/or the conversation can interfere with the vocal direction."

So instead Apple proposes using "directionality and distance of sound played through a binaural audio device to provide navigational cues to the user." So long as they are listening in stereo -- over Vision Pro's headphones or car speakers, for example -- then Apple can "thus use the spatial location of the apparent source of a sound to guide the user in a certain direction."

"For example, to guide the user on a path while listening to music, the apparent source of the music may be placed in front of the user to guide the user along the path, and moved to the side of the user to prompt the user to make a turn on the path," says the application.

"Thus, the music is not interrupted, and [this provides] a subtler method to convey navigational information than conventional navigation applications," it continues.

There's more to it than making sure you can listen to the AppleInsider podcast uninterrupted, however. Apple believes that this spatial audio system is safer.

"In addition, psychologically a user may tend to assume vocal directions are correct and thus follow the directions without much thought, potentially causing accidents," it says. "By using directionality and distance of sound as an audio cue to guide a user instead of vocal directions to tell a user, it is up to the user to determine if it is safe to follow the directional audio cue."

"A voice does not tell the user to do something (e.g., 'turn left' or 'turn right'); instead, the user is following the directional audio cue," it continues. "When following something such as a directional audio cue, a different psychology comes into play than when listening to spoken commands."

Detail from the patent showing a user turning to the side and all the spatial audio adjusting to match
Detail from the patent showing a user turning to the side and all the spatial audio adjusting to match



The rest of the patent application is concerned with the process of altering the audio's position in stereo, or changing its volume. "[Other] aspects of the audio may be attenuated to affect the virtual directionality and distance of the audio, including but not limited to frequency and reverberation," it says.

This application is credited to six inventors, including Avi Bar-Zeev and Rahul Nair, both of whom have previously worked on patent applications regarding tracking a user's gaze in Apple AR.

Read on AppleInsider

Comments

  • Reply 1 of 10
    I hope this is less annoying than it sounds. Even the Apple Watch taps and shakes can get annoying after a bit.
    beowulfschmidt
  • Reply 2 of 10
    Not sure how this would work out patent wise as there are already several companies that have similar 3D spatial audio effects, one of them being Audeze who have a gaming headphone that pinpoints the sound in game to an actual spot whether it’s behind or to the side etc.

    secondly, it’ll be really annoying for me if the sound keeps fading left to right or back to front giving the impression there something wrong with the speakers or the playback. I don’t have any issues with my music volume lowering slightly while I’m given a couple of seconds of direction. But never say never.  Will be interesting to see what they come up with
    williamlondonFileMakerFeller
  • Reply 3 of 10
    SpamSandwichSpamSandwich Posts: 33,407member
    Can we all agree they’re not going to call them “Apple Glass” since everyone already saw Google Glass come and go?

    ”Apple Glasses” at dead simple minimum.
  • Reply 4 of 10
    mayflymayfly Posts: 385member
    This is a huge reveal, it makes it apparent that Apple expects its customers will wear their headset while walking about, probably even outside, and that Apple embraces the idea.

    There is no way that it is safe to walk around while wearing them. There are obstructions everywhere, that our normal field of vision picks up without us even knowing it. People will be tripping over things and running into things, being attacked from behind because they can't hear what's around them when playing music using ANC. And wearing a $3,500 piece of highly visible jewelry that's going to be the number one target for thieves. People are being robbed, beaten and killed for $200 shoes and jackets already. Better to disable them so users have to take them off when moving.

    You'd think people are too smart to impair their sight and hearing when navigating about, but history tells us this is not the case.
    grandact73
  • Reply 5 of 10
    Sounds better with a glasses type than the vision pro. Imagine biking with that headset it sounds like an accident waiting to happen. I do wonder if Apple will have safe guards in place to stop people from walking down a street or riding a bike with VP on?
    williamlondon
  • Reply 6 of 10
    22july201322july2013 Posts: 3,494member
    mayfly said:
    ...Apple expects its customers will wear their headset while walking about, probably even outside, and that Apple embraces the idea.
    There is no way that it is safe to walk around while wearing them. 
    Or it could be a feature intended to assist the blind. Much the same way as Apple touted how FaceTime can help the deaf. This sort of detail can probably be omitted from the patent, to give competitors less information.

    If someone stole an Apple Vision Pro from a blind person would that be "robbing him blind?"
    FileMakerFeller
  • Reply 7 of 10
    mayflymayfly Posts: 385member
    mayfly said:
    ...Apple expects its customers will wear their headset while walking about, probably even outside, and that Apple embraces the idea.
    There is no way that it is safe to walk around while wearing them. 
    Or it could be a feature intended to assist the blind. Much the same way as Apple touted how FaceTime can help the deaf. This sort of detail can probably be omitted from the patent, to give competitors less information.

    If someone stole an Apple Vision Pro from a blind person would that be "robbing him blind?"
    LOL! But why would a blind person buy something called "Vision"Pro for $5,000 (with sales tax)? They could get a seeing eye dog who would perform way better. You'd be expected to pay $150 for the first dog, and $50 for each subsequent dog. Sure, you have to feed and care for him or her, but a VisionPro won't love you back or help you meet people.
  • Reply 8 of 10
    22july201322july2013 Posts: 3,494member
    mayfly said:
    mayfly said:
    ...Apple expects its customers will wear their headset while walking about, probably even outside, and that Apple embraces the idea.
    There is no way that it is safe to walk around while wearing them. 
    Or it could be a feature intended to assist the blind. Much the same way as Apple touted how FaceTime can help the deaf. This sort of detail can probably be omitted from the patent, to give competitors less information.

    If someone stole an Apple Vision Pro from a blind person would that be "robbing him blind?"
    LOL! But why would a blind person buy something called "Vision"Pro for $5,000 (with sales tax)? They could get a seeing eye dog who would perform way better. You'd be expected to pay $150 for the first dog, and $50 for each subsequent dog. Sure, you have to feed and care for him or her, but a VisionPro won't love you back or help you meet people.
    Also, if the user is both blind and deaf, the device could signal the wearer via a taptic engine of issues.

    Hey, half my ideas are duds, but at least I come up with new ideas.
    mayfly
  • Reply 9 of 10
    dewmedewme Posts: 5,126member
    mayfly said:
    mayfly said:
    ...Apple expects its customers will wear their headset while walking about, probably even outside, and that Apple embraces the idea.
    There is no way that it is safe to walk around while wearing them. 
    Or it could be a feature intended to assist the blind. Much the same way as Apple touted how FaceTime can help the deaf. This sort of detail can probably be omitted from the patent, to give competitors less information.

    If someone stole an Apple Vision Pro from a blind person would that be "robbing him blind?"
    LOL! But why would a blind person buy something called "Vision"Pro for $5,000 (with sales tax)? They could get a seeing eye dog who would perform way better. You'd be expected to pay $150 for the first dog, and $50 for each subsequent dog. Sure, you have to feed and care for him or her, but a VisionPro won't love you back or help you meet people.
    I get the joke, but the $150 cost of a seeing eye dog from is way off the mark.  From what I’ve read the cost of training, vet care, and feeding a seeing eye dog is somewhere between $15,000 - $30,000 USD. Of course this cost is not passed on to the sight impaired individual receiving the dog. I think these trained dogs are amazing, as I’ve seen firsthand with a colleague’s seeing eye dog, but the cost of training them is very expensive. The same thing holds for police, military, and especially personal protection dogs that can cost as much as $75,000 for a fully trained dog. 

    I’m all for baking AI-based assistance and augmentation into consumer products, but it’ll be a while before artificial intelligence comes close to performing at the broad and highly adaptive intelligence level of a real brain, even a dog brain or a crow brain. I would imagine that even a bat’s brain has far superior spatial audio processing than anything Apple has demonstrated in a lab. 

    With new cutting edge technology it’s more important in my opinion to focus on the emerging capabilities and potential than the initial cost and price. The cost and price will usually come down and the performance will usually improve to the point where the price/performance ratio makes it possible for most everyone to afford the product in question. The initial cost of commercially available LEDs in the early 1960s was around $200 each. I fully expect that Apple’s Vision Pro, if commercially successful, will result in a large number of derivatives that are much less costly and with higher performance and greater capabilities. Compare the price/performance and form factor of the original iMac to the latest version. The current Vision Pro is the equivalent of the original iMac for a whole new product category.
  • Reply 10 of 10
    mayflymayfly Posts: 385member
    "Of course this cost is not passed on to the sight impaired individual receiving the dog."

    That's partially the point, though, isn't it. Do you suppose Apple will subsidize the VisionPro for blind people?

    And I don't know if you've ever owned a golden retriever, but I have (pet, not service dot), and again, your VisionPro isn't going to love you, or be your friend, or help you make friends. True, you have to care for him or her, and that costs money. But you have to buy software for the VP, and that costs money too. And blind or not, I'd rather spend $5,000 a year on my goldie than $5,000 (with tax) on a headset.
    edited July 26
Sign In or Register to comment.