Apple invention adjusts audio based on a display's orientation, user positioning
A patent application published by the U.S. Patent and Trademark Office on Thursday reveals Apple is actively researching methods to ensure a video's multi-channel audio is always played back correctly, no matter how a user holds the device.
Source: USPTO
Apple's patent filing for "Orientation-based audio" describes a system that leverages a mobile device's internal sensors, including accelerometers, gyroscope and cameras, among others, to synchronize audio output from multiple speakers.
According to one embodiment, the orientation of video being displayed on a device is used to determine the relative position of speakers to the left and right of the screen. Stereo audio is then routed to the correct speakers.
In another embodiment, a device is determined to be rotating based on various sensor data. The information is used to dynamically switch, or route, the audio to the appropriate speakers. For example, if an iPad mini user moves from viewing a movie with the tablet positioned home-button-down to home-button-up, the right and left audio channels would be switched to compensate for the 180-degree rotation.
The invention can be applied to devices with more than two speakers, and examples include implementations with three or more outputs.
Powering the system is an audio processing router that first matches audio channels of the source to the device's speaker configuration.
If the device is not locked to a particular mode, like "portrait" or "landscape," the processor will actively reroute audio depending on what orientation the video is being displayed. An alternative mode of operation bases audio output on user settings rather than a predefined map.
When rotation or other movement is perceived, the processor creates a new map of the device and redirects audio accordingly. User customization settings can limit the switching to certain angles, such as a 90-degree or full 180-degree rotation.
Finally, some embodiments call for the application of equalization techniques to normalize audio according to a user's perception. For example, if a device is moved laterally away from a user, gain can be adjusted to compensate. Sensors, such as accelerometers, cameras and infrared presence detectors can be used to determine a user's movement.
In one example, if a portion of the user's body, such as the head, appears smaller, it is likely that they moved away from the device. In this situation, gain or volume would be increased to normalize audio.
Time delays can also be used strategically to syncing audio from one set of speakers with a second if they are not in line with a user's head.
While Apple's current iOS device lineup holds only one product with stereo loudspeakers, the iPad mini, the tech may be implemented in future iterations of existing models. Alternatively, the filing's assets can be applied to MacBook Pros or other hardware Apple has yet to introduce. The technology would be well-suited for a dedicated television set, a product long rumored to be in the offing.
Apple's orientation-based audio patent application was first filed for in 2011 and credits Martin E. Johnson, Ruchi Goel and Darby E. Hadley as its inventors.
Source: USPTO
Apple's patent filing for "Orientation-based audio" describes a system that leverages a mobile device's internal sensors, including accelerometers, gyroscope and cameras, among others, to synchronize audio output from multiple speakers.
According to one embodiment, the orientation of video being displayed on a device is used to determine the relative position of speakers to the left and right of the screen. Stereo audio is then routed to the correct speakers.
In another embodiment, a device is determined to be rotating based on various sensor data. The information is used to dynamically switch, or route, the audio to the appropriate speakers. For example, if an iPad mini user moves from viewing a movie with the tablet positioned home-button-down to home-button-up, the right and left audio channels would be switched to compensate for the 180-degree rotation.
The invention can be applied to devices with more than two speakers, and examples include implementations with three or more outputs.
Powering the system is an audio processing router that first matches audio channels of the source to the device's speaker configuration.
Alongside speaker mapping, the processor can mix what would normally be center channel audio across a set of two speakers if a third is not present. Gain is also controlled to compensate for the "missing speaker," while in another embodiment the channel may be suppressed entirely.Typically, although not necessarily, the audio processing router attempts to ensure that left and right channel audio outputs (whether front or back) are sent to speakers on the left and right sides of the device, respectively, given the device's current orientation. Thus, front and rear left channel audio may be mixed and sent to the left speaker(s) while the front and rear right channel audio may be mixed and sent to the right speaker(s).
If the device is not locked to a particular mode, like "portrait" or "landscape," the processor will actively reroute audio depending on what orientation the video is being displayed. An alternative mode of operation bases audio output on user settings rather than a predefined map.
When rotation or other movement is perceived, the processor creates a new map of the device and redirects audio accordingly. User customization settings can limit the switching to certain angles, such as a 90-degree or full 180-degree rotation.
Finally, some embodiments call for the application of equalization techniques to normalize audio according to a user's perception. For example, if a device is moved laterally away from a user, gain can be adjusted to compensate. Sensors, such as accelerometers, cameras and infrared presence detectors can be used to determine a user's movement.
In one example, if a portion of the user's body, such as the head, appears smaller, it is likely that they moved away from the device. In this situation, gain or volume would be increased to normalize audio.
Time delays can also be used strategically to syncing audio from one set of speakers with a second if they are not in line with a user's head.
While Apple's current iOS device lineup holds only one product with stereo loudspeakers, the iPad mini, the tech may be implemented in future iterations of existing models. Alternatively, the filing's assets can be applied to MacBook Pros or other hardware Apple has yet to introduce. The technology would be well-suited for a dedicated television set, a product long rumored to be in the offing.
Apple's orientation-based audio patent application was first filed for in 2011 and credits Martin E. Johnson, Ruchi Goel and Darby E. Hadley as its inventors.
Comments
This is what - addressing real problems. Very nice. When I watch videos with my niece in landscape orientation and when she holds the iPad, her palm covers the speakers.
This is nice. This is nice.
Not sure that this patent passes the common sense rule. Seems obvious to use orientation sensors to intelligently route audio to the appropriate speaker. Likewise for the camera sensing the size of your head.
Still, stereo speaker in the iPad then? Not too bothered, but it'd be a nice addition. In contrast to Ireland I've found the iPad speakers to be very good for a device of its size. Both volume and audio quality are as good, or better than I need them to be already.
EDIT: Ooh, I didn't think about the implication for multiple speakers meaning that your palm is less likely to be covering it up. That's definitely a win.
The other thing that would be really nice and would probably only take about 5 minutes to code would be orientation sensitive volume controls. It's more intuitive this way:
-.........+
When held in landscape view with the home button on either the left or right side.
+
.
.
.
.
-
When held in portrait view with the home button at either the top or bottom.
Not sure about that. Controls changing position could be pretty unintuitive for some people too, if your physical memory is of the form "button closest to the corner is volume up, button furthest from the corner is volume down". Depends what the more common thought process is, I guess.
The article refers to the speakers "switching" channels. That leaves out a typical form of Apple elegance.
When the device is physically rotated, the picture doesn't just abruptly switch to the new orientation, it "spins" into position. The audio should, too. There should be a crossfade from speaker to speaker, not an instantaneous switch.
Cool idea, especially if the audio rotation is synchronised with the spinning of the picture. Could be a bit weird to experience though, I think I'd need to try it before I know if I'll like it
I see a nice hypnosis or tripping app there. A 3D vortex comes to mind.
Oh, and I second Crowley. Good idea.
Maybe I am mistaken, but doesn't existing smart phones already have similar functionality? The HTC One comes to mind.
I'd rather see them dropping the volume indicator altogether. There's nothing wrong with my hearing, so I don't need visual feedback. It's strange for Apple to focus on simplicity, yet they intrude the screen with that volume overlay.
Still, you make a valid point here.