Apple's 'audio hyperlink' tech can control devices with inaudible sonic pulses

Posted:
in General Discussion edited January 2014
An Apple patent filing published by the U.S. Patent and Trademark Office on Thursday describes a method of encoding hyperlinks into an audio stream, offering content creators the ability to link to other media, or control a device's user interface, with a song or podcast.

Audio Hyperlink
Illustration of audio hyperlink from one audio track to another. | Source: USPTO


Titled "Audio Hyperlinking," Apple's invention is similar in function to the current "enhanced podcast" technique used to add images, URLs and other content to a file encoded in the AAC audio format. Instead of wrapping the assets as chapter markers or metadata, however, the invention's hyperlinks are encoded directly into the source audio stream.

From the patent filing's background:
For example, many podcast programs identify other resources that might be of interest to the listener, such as other podcasts or web material, there is currently no way for a podcast to provide a hyperlink to such other resources in the audio content of the podcast.
In one embodiment, a hyperlink indicator and accompanying instruction set can be embedded in the audio stream as either an audible or inaudible tone. As noted in the application, these tones can be grouped into sequences that are detected and decoded by the playback framework.

Once the audio hyperlink is identified, the device "traverses" the hyperlink based on information provided in the decoded data to change its user interface, displaying information about the link, notify the listener of its presence, or take any other action as instructed. Some embodiments don't require user input, meaning the device will execute instructions based on predetermined settings or preferences.

Audio Hyperlink
Flowchart of audio hyperlink encoding.


Certain hyperlinks can pause currently playing audio and take the listener to another position in the track, to another media file, or a Web content. Alternatively, the hyperlink can lead to images, URLs and other media. Basically, the audio hyperlink acts as a traditional text-based hyperlink, but is encoded in audio data.

A decaying pulse amplitude can also be applied to limit the time in which a hyperlink is active. In this case, a user may have to touch the device's screen, use voice commands, or interact with physical buttons to follow the link. Once the signal reaches a certain threshold, the hyperlink becomes inaccessible.

The filing notes that, like enhanced podcasts, audio hyperlinks can be accompanied by markers that notate their location on the track. This data would not be interleaved with the audio stream, but may be stored as associated metadata instead.

Audio Hyperlink


Solutions similar to Apple's described invention are already in use, though only specialized programs can take advantage of the technology. For example, during a recent concert tour, Lady Gaga performed a song that included inaudible tones only electronic devices can distinguish. An accompanying service called Sonic Notify translated those high frequency tones into a type of hyperlink that led smartphones to webpages and images.

It is unclear what Apple has planned for the invention, but such technology can theoretically be applied to songs with rich interactive content, or for ad placement in tracks played via the forthcoming iTunes Radio.

Apple's audio hyperlink patent application was first filed for in 2012 and credits Samir Gehani as its inventor.

Comments

  • Reply 1 of 19
    philboogiephilboogie Posts: 7,675member
    Interesting tech. Probably copied by the competition as 'obvious'.

    For its implementation, iTunes Radio comes to mind. Easily disable ads if user paid the $25/y iTunes Match.
  • Reply 2 of 19
    olalauolalau Posts: 2member
    They invented Chirp! (http://www.chirp.io)
  • Reply 3 of 19
    dugbugdugbug Posts: 283member


    So when my dog barks I have a new ad :)

     

  • Reply 4 of 19
    irelandireland Posts: 17,798member
    philboogie wrote: »
    Interesting tech. Probably copied by the competition as 'obvious'.

    Don't become bitter.
  • Reply 5 of 19
    philboogiephilboogie Posts: 7,675member
    ireland wrote: »
    philboogie wrote: »
    Interesting tech. Probably copied by the competition as 'obvious'.

    Don't become bitter.

    Oops. I came across like that? Off to my iPad for some CandyCrush...
  • Reply 6 of 19
    weejockweejock Posts: 32member
    Inaudible sounds controlling my device automatically? My mind instantly jumped to thoughts of clever hackers creating secret audio-viruses to take control of my phone,

    I know that the reality will be much less sinister, with permissions being asked for, prompts to click on, settings to only automatically allow links from trusted sources etc.. This sounds pretty cool; it'll be interesting to see how this could be developed through iTunes radio as that grows.
  • Reply 7 of 19
    MacProMacPro Posts: 19,718member
    weejock wrote: »
    Inaudible sounds controlling my device automatically? My mind instantly jumped to thoughts of clever hackers creating secret audio-viruses to take control of my phone,

    I know that the reality will be much less sinister, with permissions being asked for, prompts to click on, settings to only automatically allow links from trusted sources etc.. This sounds pretty cool; it'll be interesting to see how this could be developed through iTunes radio as that grows.

    I must be feeling dark this morning! Lol

    My first thought was to wonder if this could be used for ownership verification. I could imagine it would be simple enough to tag music as purchased from iTunes with the buyers' Apple ID. Of course the question them becomes how easy is it remove or modify once in the stream.

    The humorist in me also imagined the news station being changed on smart TVs. ;)
  • Reply 8 of 19
    Damn I thought of using something similar a while ago.... but only as a mean to make a cheap wireless controller... i.e. different frequencies for direction and buttons.
  • Reply 9 of 19

    Quote:

    Originally Posted by PhilBoogie View Post



    Interesting tech. Probably copied by the competition as 'obvious'.



     


    This is nothing new and was more or less copied from the competition, it's called stealthy audio watermarking.


     


    Quote:

    Originally Posted by digitalclips View Post





    I must be feeling dark this morning! Lol



    My first thought was to wonder if this could be used for ownership verification. I could imagine it would be simple enough to tag music as purchased from iTunes with the buyers' Apple ID. Of course the question them becomes how easy is it remove or modify once in the stream.



     


    Yes this can be used for DRM and can be nearly impossible to remove from the audio.


    A good example of this is Cinavia, which can control playback and display messages encoded into the audio.

  • Reply 10 of 19
    robin huberrobin huber Posts: 3,949member
    "Clap on! Clap off!"
  • Reply 11 of 19
    droidftwdroidftw Posts: 1,009member


    Chirp! better lawyer up because they're about to get their asses handed to them for copying Apple's innovation.

  • Reply 12 of 19
    imt1imt1 Posts: 87member

    Quote:

    Originally Posted by Chrispoe View Post


    This is nothing new and was more or less copied from the competition, it's called stealthy audio watermarking.


     


    Yes this can be used for DRM and can be nearly impossible to remove from the audio.


    A good example of this is Cinavia, which can control playback and display messages encoded into the audio.



    This is nothing like Stealth watermarking. This is also nothing like Cinavia, which is strictly a stealth audio watermarking for blue-ray copy protection. This is not a watermark. This is a way to insert a hyperlink into an audio stream, which a user can click on to get more information. The action the user takes depends on what happens next.  Cinavia does not allow for user intervention either. It will only detect if the disk is an original or being played on the correct device. If not the disk stops and the user gets a prompt. This all assumes a Cinavia encoded blue ray with a Blue Ray player containing Cinavia technology.


     


    I would also have to imagine that the way the signal is embedded via Apple is utilizing different methodology as well. Cinavia is stealth and is done via phase manipulation. 


     


    Chirp is different as well. Chirp sends you info (File/photo/etc) to its server and then "Chirps" a specific audio sequence, which another iPhone running Chirp listens and translates and then goes to the server and retrieves the file. 

  • Reply 13 of 19
    droidftwdroidftw Posts: 1,009member
    Hopefully, for those companies, Apple agrees with you. As we all know, Apple takes infringement on their patents very seriously.
  • Reply 14 of 19
    ash471ash471 Posts: 705member

    Quote:

    Originally Posted by Chrispoe View Post


    This is nothing new and was more or less copied from the competition, it's called stealthy audio watermarking.


     


    Yes this can be used for DRM and can be nearly impossible to remove from the audio.


    A good example of this is Cinavia, which can control playback and display messages encoded into the audio.



    What? Not novel?  Are you kidding?  This is awesome technology and it has nothing to do with watermarking.  Apple is trying to create a responsive system, not authenticate the source of the audio.  


     


    You obviously don't understand invention.  Invention comes from putting known parts together in a way that creates a product that performs a new function or solves a problem.  The European patent office has a good term for this.  They require a "technical advancement." Based on what we know about the prior art, Apple's hyperlinking is clearly innovative. 


     


    The Cinavia might be closer prior art since the audio actually performs a function, but it still isn't hyperlinking.


     


    I very much doubt that Apple has interest in DRM.  Apple convinced the record industry to make iTunes DRM free.

  • Reply 15 of 19
    An Ask Patents user submitted a prior-art-request for Apple Audio Hyperlinking Patent Application here:
    --> http://patents.stackexchange.com/q/4745/1767 ...10 minutes of your time can help narrow this US patent application before it becomes an issued patent. Follow @askpatents on twitter to help.
  • Reply 16 of 19


    A user at AskPatents.com submitted a prior art request for Apple's "Audio Hyperlinking" Patent Application discussed above.


     


    Anyone can help knockout this patent application:  http://patents.stackexchange.com/q/4745/1767


     


    10 minutes of your time can help narrow US patent applications before they become patents. Follow @askpatents on twitter to help.

  • Reply 17 of 19
    It learns your surroundings and then knows more about you, if this is ever implanted I hope audio sense or whatever has off button to conserve battery.
  • Reply 18 of 19


    From Wikipedia:http://en.wikipedia.org/wiki/CueCat


    :CRQ (a word play on "see our cue") is software developed by Digital Convergence intended to convert "cues" from television signals and the :CueCat bar code reader into URLs. The television technology was launched in 1996 on the television series Net Talk Live! and made its network debut on NBC during its "Must See TV" programming and used a computer sound card to decode a tone and launch a web site.

  • Reply 19 of 19
    @chrispoe: I used to work for a company that employed this technology for watermarking: making personalised copies of albums before they were released, for anti-piracy reasons. It was possible to render the pulses un-decodeable by vari-speeding the audio 0.1%, which is basically inaudible. That was 2005, not sure if the technology has advanced since then.
Sign In or Register to comment.