Apple researching HomePod-like audio for future MacBook Pro

Posted:
in General Discussion edited June 2020
Apple could manipulate traditional speakers into creating sound that appears to come from somewhere else, such as one from around a MacBook user instead of at the keyboard, to give a better audio experience.




Apple is continuing to work on improving sound quality from speakers, in particular those on MacBook Pro laptops or similar. In a new patent filing, the company describes how an audio track can be processed in such a way that it appears to be coming from "elsewhere in an indoor outdoor space [rather] than directly from a loudspeaker."

"Audio signal processing for virtual acoustics can greatly enhance a movie, a sports even, a videogame or other screen viewing experience, adding to the feeling of "being there," says Apple in US patent application number 10,524,080, entitled "System to move a virtual sound away from a listener using a crosstalk canceler."

"Various known audio processing algorithms, executed by digital processors, modify one or more recorded, synthesized, mixed or otherwise produced digital audio signals," it continues, "in such a way as to position a virtual source according to modeling that is based on human perception of sound, including the role of ear acoustics, other reflecting and absorbing surfaces, distance and angle of source, and other factors."

Detail from patent showing part of how audio can be made to appear to surround the user
Detail from patent showing part of how audio can be made to appear to surround the user


While the majority of the documentation describes the loudspeakers on a laptop, Apple claims that this patent covers virtual sound from any such audio source -- including headphones.

"In the case of headphones, specially processed audio signals (binaural rendering) are sent to left and right ears of a listener without the crosstalk that is inevitably received by the ears when listening to stereo loudspeakers," it says.

Apple says that using "sound wave cancellation in the air surrounding the listener," the audio can be controlled.

"For viewers and listeners that prefer loudspeakers, for example those that may be built into a laptop computer, a crosstalk canceler is employed in some virtual acoustic systems to produce sounds from multiple loudspeakers in such a way that for example a 'left' audio signal is predominantly heard only at the left ear of the listener, and a 'right' audio signal is predominantly heard only at the right ear of the listener," it says.

"This allows the left and right audio signals to contain spatial cues that enable a virtual sound to be 'positioned' at a desired location between the loudspeakers."

Detail from patent showing direct and reflected audio from speakers
Detail from patent showing direct and reflected audio from speakers


The six inventors credited with this work have between them almost 50 prior patents in related fields, chiefly for Apple but also for Sennheiser Electronics. Martin E. Johnson alone has 38 previous patents, ranging from "correction of unknown audio content" and "Method and aparatis for estimating talker distance."

This latest patent, filed on December 31, 2019, follows two from November which are to do with converting headphones into speakers and providing better sound isolation.

Apple has also recently filed a patent for headphones that are able to detect which ear they've been put on, which would affect how audio's left and right channels would be routed.
«1

Comments

  • Reply 1 of 24
    lkrupplkrupp Posts: 10,557member
    Lots of things get experimented with but few actually come to market. Why? Because of market realities, ROI, engineering obstacles, manufacturing problems, etc. I always start to giggle when someone starts out a post with, “It would be a simple matter for Apple to do this or that, make this or that, enable this or that.” It shows the utter cluelessness of the one posting.
    cornchipwatto_cobraRayz2016
  • Reply 2 of 24
    lkrupp said:
    Lots of things get experimented with but few actually come to market. Why? Because of market realities, ROI, engineering obstacles, manufacturing problems, etc. I always start to giggle when someone starts out a post with, “It would be a simple matter for Apple to do this or that, make this or that, enable this or that.” It shows the utter cluelessness of the one posting.
    About a week there was a rumor that Apple may be working on a Mac SKU dedicated to gaming.  It could come in the form of a laptop or iMac.  This patent and that rumor may go hand-in-hand

    https://www.patentlyapple.com/patently-apple/2019/12/rumor-apple-plans-to-launch-a-new-high-end-mac-in-2020-designed-to-address-the-esports-market.html
    watto_cobra
  • Reply 3 of 24
    zimmiezimmie Posts: 651member
    "Sonic holography" has been a thing since the 70s or so. It's pretty cool. Gives you the perception of a wider soundstage without the physical distance between speakers to support it. I have a Carver C-4000 preamp (and an M-400 cube!) from the 80s which can do some pretty amazing things.
    cornchipwatto_cobra
  • Reply 4 of 24
    Mike WuertheleMike Wuerthele Posts: 6,858administrator
    lkrupp said:
    Lots of things get experimented with but few actually come to market. Why? Because of market realities, ROI, engineering obstacles, manufacturing problems, etc. I always start to giggle when someone starts out a post with, “It would be a simple matter for Apple to do this or that, make this or that, enable this or that.” It shows the utter cluelessness of the one posting.
    About a week there was a rumor that Apple may be working on a Mac SKU dedicated to gaming.  It could come in the form of a laptop or iMac.  This patent and that rumor may go hand-in-hand

    https://www.patentlyapple.com/patently-apple/2019/12/rumor-apple-plans-to-launch-a-new-high-end-mac-in-2020-designed-to-address-the-esports-market.html
    We don't think so. The source that PA used for its report is unreliable, and then PA spun the limited data well out of proportion to what was being claimed.
    edited December 2019 watto_cobra
  • Reply 5 of 24
    dewmedewme Posts: 5,335member
    zimmie said:
    "Sonic holography" has been a thing since the 70s or so. It's pretty cool. Gives you the perception of a wider soundstage without the physical distance between speakers to support it. I have a Carver C-4000 preamp (and an M-400 cube!) from the 80s which can do some pretty amazing things.
    Yeah. Though not geared towards music reproduction active sonar systems have been using transmit beam forming and beam steering since the 1950s. What is being described here is a direct adaptation of the exact same acoustic principles used by active sonars for sound transmission, and all array based sonars (and the brains of living creatures that hear) for sound reception. Very cool stuff. 
    watto_cobra
  • Reply 6 of 24
    ...having just been through the exercise of 'Smart TV' and how proprietary such has become I keep hoping AirPlay2 will offer true decoding for surround sound via wifi & bluetooth, perhaps even ATMOS. I too have discrete analog audiophile equipment (cubes) connected with Airport Express that may be tough to better with anything soundbar, not to mention the environmental and cost aspects of replacing perfectly functional gear... I continue to hope Apple may resurrect Airport and Time Capsule too, sigh...
    edited December 2019 watto_cobra
  • Reply 7 of 24
    I've long wondered when someone would merge head tracking (obtained from the camera) to modify sound sent to headphones to lock the sound source to a world frame of reference. At present, your headphone's sound stage moves with your head, and that breaks immersion. Locking the stage to a world reference would go a long way towards fooling your brain into thinking there actually is a stage. When using a laptop's world referenced speakers, psychoacoustic processing could incorporate knowledge of head orientation to alter the way sound is processed to give the impression of a world referenced 3D audio space.

    People have experimented with head tracking to produce 3D effects on 2D displays...



    I imagine there's something to be gained by using head tracking to alter audio as well.
    edited December 2019 watto_cobraRayz2016
  • Reply 8 of 24
    zimmiezimmie Posts: 651member
    dewme said:
    zimmie said:
    "Sonic holography" has been a thing since the 70s or so. It's pretty cool. Gives you the perception of a wider soundstage without the physical distance between speakers to support it. I have a Carver C-4000 preamp (and an M-400 cube!) from the 80s which can do some pretty amazing things.
    Yeah. Though not geared towards music reproduction active sonar systems have been using transmit beam forming and beam steering since the 1950s. What is being described here is a direct adaptation of the exact same acoustic principles used by active sonars for sound transmission, and all array based sonars (and the brains of living creatures that hear) for sound reception. Very cool stuff. 
    My understanding is active phased array sonar is more like parametric speakers. I don't think that's what Apple is talking about here, but maybe!
    watto_cobra
  • Reply 9 of 24
    MacProMacPro Posts: 19,718member
    Talking of HomePods ... Siri still can't access my iTunes Match from any HomePods we have, instead always attempting to open Apple Music which I don't subscribe to.  All other devices including Car Play and Apple TV 4K using the remote's mic have no such problem.  Everything is up to date.  Has anyone else seen this problem?
    cornchipwatto_cobra
  • Reply 10 of 24
    dewmedewme Posts: 5,335member
    zimmie said:
    dewme said:
    zimmie said:
    "Sonic holography" has been a thing since the 70s or so. It's pretty cool. Gives you the perception of a wider soundstage without the physical distance between speakers to support it. I have a Carver C-4000 preamp (and an M-400 cube!) from the 80s which can do some pretty amazing things.
    Yeah. Though not geared towards music reproduction active sonar systems have been using transmit beam forming and beam steering since the 1950s. What is being described here is a direct adaptation of the exact same acoustic principles used by active sonars for sound transmission, and all array based sonars (and the brains of living creatures that hear) for sound reception. Very cool stuff. 
    My understanding is active phased array sonar is more like parametric speakers. I don't think that's what Apple is talking about here, but maybe!
    It all ties back to the principle of the superposition of (sound) waves, so both approaches share the same underlying principles. Nobody is copying anyone else, they’re simply all taking advantage of a common knowledge base of the physics of acoustics. 

    What Apple is doing is very similar to an approach one might take when designing a target simulator for a passive sonar array, where you’d have to model both the distance and angle between the sound source (target) and the receiving array. Doing this properly requires fairly intimate knowledge of the acoustic processing characteristics of the receiver, including both the “array” and receiver signal processing. This really highlights how far Apple has progressed towards combining their understanding of how their technology best interacts with innate human sensory abilities. 

    The physics parts are relatively straightforward because they are based on sound underlying principles. The human side of their solution is where Apple really shines. Like I said, very cool stuff. 
    watto_cobra
  • Reply 11 of 24
    zoetmbzoetmb Posts: 2,654member
    Instead of fooling around with this stuff and destroying the integrity of the original recordings by manipulating phase and other techniques, Apple should simply try to come up with a way to add more bass to the internal speakers (if this is for gaming apps, then it's fine).   I know the new MBP is supposed to have better bass, but at the Apple store, I couldn't hear any difference, although as usual it was unbelievably noisy in the store.   (When Siri was first released, I couldn't make it work in the store either, because it was so noisy, Siri couldn't understand what I was saying).   This is one part of the Apple store design that they got wrong:  all hard surfaces, no absorption, lots of echo.   
    watto_cobra
  • Reply 12 of 24
    MacPro said:
    Talking of HomePods ... Siri still can't access my iTunes Match from any HomePods we have, instead always attempting to open Apple Music which I don't subscribe to.  All other devices including Car Play and Apple TV 4K using the remote's mic have no such problem.  Everything is up to date.  Has anyone else seen this problem?
    I have the same issue and Apple Support confirmed the bug to me a couple weeks ago.  It has to do with the multi-user features in iOS 13.3.  Hopefully 13.4 will finally fix the issue.   Turn off  "recognise my voice" in the Home app and you will be able to access your playlists again. 
    watto_cobraMacPro
  • Reply 13 of 24
    MplsPMplsP Posts: 3,911member
    This seems neat, but in reality it would be difficult to accomplish. The HomePod has an array of speakers and microphones that let it tailor the sound while a MacBook only has a couple of speakers in two locations. I like Bsimpsen's idea about using head tracking to position the sound, though.
    watto_cobra
  • Reply 14 of 24
    badmonkbadmonk Posts: 1,286member
    zimmie said:
    "Sonic holography" has been a thing since the 70s or so. It's pretty cool. Gives you the perception of a wider soundstage without the physical distance between speakers to support it. I have a Carver C-4000 preamp (and an M-400 cube!) from the 80s which can do some pretty amazing things.
    i am envious, the Carver Cube amplifier from the early 80 was my first techno-lust item.  I wonder if it influenced Steve Jobs to make the Apple Cube computer
    cornchipwatto_cobra
  • Reply 15 of 24
    MplsP said:
    This seems neat, but in reality it would be difficult to accomplish. The HomePod has an array of speakers and microphones that let it tailor the sound while a MacBook only has a couple of speakers in two locations. I like Bsimpsen's idea about using head tracking to position the sound, though.
    Though having multiple speakers might make things easier, there's a fair bit of "magic" that can be done with only two. Our brains, using only two ears, locate sound sources in 3D space using a variety of cues. Sounds coming from behind us have reduced high frequency content because they're blocked by the pinna. That can be simulated from the front. We discern left and right not only by relative amplitude, but also by phase delay. Sound from a source on the right will arrive at the left ear later than at the right ear. We also sense the echoes of sound sources against other objects, to get a sense of the surroundings. That subconscious understanding of the environment can help resolve ambiguity in the sound we hear, allowing us to more accurately locate the actual source. Hearing is a marvelously complex thing, and we're probably still a long way from understanding it well. Nevertheless, what we do understand allows us to do some very interesting things with just two speakers.

    When we're uncertain about the location of a sound, we turn our heads. This is where head tracking comes in. By understanding how a real 3D sound stage would be perceived as we turn our heads, and also understanding how the laptop's two speakers will sound as we turn our heads, a sufficiently complete psychoacoustic model should be able to fool us into comprehending a 3D aural environment that's not actually there.

    Many years ago, a friend invited me to listen to a little experiment he'd rigged up, using a white noise source and a pair of headphones attached to a position encoder hanging from a microphone stand. As I turned my head, his device altered the amplitude and delay of the noise to each ear. The result, a bit clunky, produced a very clear perception that the noise source was in a fixed location behind me. Though we both understood the underlying mechanism of that perception, we still found it "magical".

    Ultimately, you don't want all that psychoacoustic noodling to sound like magic so much as to sound real. That won't be easy.
    watto_cobra
  • Reply 16 of 24
    dewmedewme Posts: 5,335member
    zoetmb said:
    Instead of fooling around with this stuff and destroying the integrity of the original recordings by manipulating phase and other techniques, Apple should simply try to come up with a way to add more bass to the internal speakers (if this is for gaming apps, then it's fine).   I know the new MBP is supposed to have better bass, but at the Apple store, I couldn't hear any difference, although as usual it was unbelievably noisy in the store.   (When Siri was first released, I couldn't make it work in the store either, because it was so noisy, Siri couldn't understand what I was saying).   This is one part of the Apple store design that they got wrong:  all hard surfaces, no absorption, lots of echo.   
    I agree that the listener should always have the option of hearing the sound as close to how it was originally produced as long as the reproduction system supports this capability. However some folks do like to mess around with the reproduction to suit their personal preferences or as we’ve seen with faux stereo reproductions, to exploit customer perceptions. 

    What Apple is describing seems much more focused on gaming and virtual/augmented reality. 

    Getting good undistorted bass from small form factor devices with small speakers is a tough nut to crack because of the physics involved, especially notebook computers that have virtually unlimited placement options. It might more sense for Apple to build an optional companion wireless subwoofer that pairs with the MacBook for certain scenarios. You probably wouldn’t want to carry it everywhere with you, but at your desk or in your family room, maybe bring it along. 
    watto_cobra
  • Reply 17 of 24
    entropysentropys Posts: 4,152member
    While no doubt fun stuffing around with MBP speakers, which are already better than the competition anyway, Apple’s priorities IMHO laptopwise should be:

    • getting the MBP 16 keyboard into the rest of the laptop line ASAP
    • improving significantly battery life
    • remembering that MBPs and MBAs used for work or school in a variety of locations might need a USB-A port (and maybe HDMI).

    Reflecting back over the last decade, I put it to you all the late 2010 MBA and it’s subsequent iterations was the best Mac ever, possibly even more significant than the PowerBook 100 series-yes please forgive my blasphemy). Like the original power PowerBook , the MBA changed laptop computing, everyone else copied it, it featured in every movie when a laptop was needed, and was well priced.
    Then it got Ived.  
    edited December 2019 MplsP
  • Reply 18 of 24
    cpsrocpsro Posts: 3,192member
    With MacBook Pro 16 speakers, sound already often seems to come from somewhere else. Because of its larger display and vastly better audio, the MBP16 has superseded the iPad 12.9 for my portable video viewing. (This makes me think Apple is now obligated to make larger iPads with improved speakers.)
    @Zimmie: Yes, Carver's Sonic Holography is/was fantastic. The sweet spot for listening is unfortunately just too confining.
    watto_cobra
  • Reply 19 of 24
    k2kwk2kw Posts: 2,075member
    entropys said:
    While no doubt fun stuffing around with MBP speakers, which are already better than the competition anyway, Apple’s priorities IMHO laptopwise should be:

    • getting the MBP 16 keyboard into the rest of the laptop line ASAP
    • improving significantly battery life
    • remembering that MBPs and MBAs used for work or school in a variety of locations might need a USB-A port (and maybe HDMI).

    Reflecting back over the last decade, I put it to you all the late 2010 MBA and it’s subsequent iterations was the best Mac ever, possibly even more significant than the PowerBook 100 series-yes please forgive my blasphemy). Like the original power PowerBook , the MBA changed laptop computing, everyone else copied it, it featured in every movie when a laptop was needed, and was well priced.
    Then it got Ived.  
    I mainly agree with you although I would prefer HDMI and , SD card reader, and MagSafe being added to the MBPin the 2020 model.  
    MplsP
  • Reply 20 of 24
    fastasleepfastasleep Posts: 6,408member
    lkrupp said:
    Lots of things get experimented with but few actually come to market. Why? Because of market realities, ROI, engineering obstacles, manufacturing problems, etc. I always start to giggle when someone starts out a post with, “It would be a simple matter for Apple to do this or that, make this or that, enable this or that.” It shows the utter cluelessness of the one posting.
    So an imaginary quote that you just made up shows the utter cluelessness of the one posting? Hmm.
    watto_cobraMplsP
Sign In or Register to comment.