Apple patent could bring polyphonic tuning to Logic Pro

Posted:
in General Discussion edited January 2014
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent for polyphonic note detection, or the ability to parse out single pitches and tones from a plurality of voices, a technology that may make its way to an expected version of the company's Logic Pro music editing software.

Polyphonic Note Detection
Source: USPTO


Aptly-named "Polyphonic note detection," U.S. Patent No. 8,309,834 describes a method in which a polyphonic audio signal is converted and sampled against a reference note to distinguish individual voices.

Prevailing technology is limited to detecting single notes, such as a person singing or the picking of a single string on a guitar, which can be used for a number of audio applications like tuning, music creation and pitch correction. With polyphonic detection, a user would be able to tune all strings on a guitar with a single strum.

While there are certain algorithms on the market for digital audio workstations, such as Melodyne from Celemony, Apple has yet to include the feature in its own Logic DAW.

The patented method is clearly aimed at software, as it calls for a processor-enabled solution that breaks down a portion of a polyphonic audio signal by first converting it from a time domain to a frequency domain. Peak frequencies are detected using either a reference signal or a decibel threshold, and if a predetermined number of integer-interval harmonic partials are detected, a note is recorded. In other words, the system relies on harmonics, or harmonic partials, which are the basic result of resonant sound waves created by an instrument or human voice.

Fundamental frequencies, or the lowest frequency off of which harmonics and partials are based, act as a guide to which integer-interval harmonic partials are tested. If a fundamental frequency is detected, a note will be registered. The deviation from a perfect integer-interval can be tracked to find the inharmonicity, or distance from the nearest harmonic, in a polyphonic signal.

Three peaks at integer-interval harmonic partial frequencies must be detected to notate a fundamental frequency.

Peak Frequency
Illustration of a polyphonic audio signal and its peak integer-interval harmonic partials.
Note fundamental frequency threshold at 30 dB.


When the system finishes detecting the particular notes, it can determine what chord is being played, if any. For example, if the notes detected are C#, E, G#, it would be determined that the polyphonic signal is a C sharp minor chord.

In one embodiment, the detection process described can repeated every 256 samples, or every 5.8 milliseconds for a CD-quality recording, which has a standard sampling frequency of 44,100 Hz. Implementations of the invention are also described in which MIDI notes are used for reference against the processed signal.

According to the invention, the process described can be repeated "until each note in the polyphonic audio signal has been detected."

Apple's patent can be executed by software-only, hardware-only, or a combination of both, meaning it has the possibility of appearing in an upcoming version of Logic. Beyond rumors, there has been little evidence that Apple's DAW will feature polyphonic note detection, however rival software does offer similar solutions as plug-ins or wholly-licensed technology.

Comments

  • Reply 1 of 20
    TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone.. I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?
  • Reply 2 of 20
    MarvinMarvin Posts: 15,309moderator
    If this will be like the following software, it would offer a lot of contol over audio editing:

    http://www.engadget.com/2011/06/25/spectral-layers-promises-to-let-you-edit-audio-photoshop-style/
  • Reply 3 of 20

    Quote:

    Originally Posted by edubong View Post



    TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone.. I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?


    The patent will be for the precise method to create the end result, not the end result itself. There are many ways to skin a cat, and each method of skinning can be patented.

  • Reply 4 of 20
    gtbuzzgtbuzz Posts: 129member
    Could it be for specific voice detection out of several voices ?
  • Reply 5 of 20

    Quote:

    Originally Posted by GTBuzz View Post



    Could it be for specific voice detection out of several voices ?


    I was thinking similar.

  • Reply 6 of 20


    I know a few people who will go on a music-making spree if this is true… 


     


    image

  • Reply 7 of 20
    drblankdrblank Posts: 3,385member

    Quote:

    Originally Posted by edubong View Post



    TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone.. I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?


    The only way they can do this is by using a different algorithm.  The end result is the same, but the method in how it's done might be different.

  • Reply 8 of 20
    drblankdrblank Posts: 3,385member

    Quote:

    Originally Posted by edubong View Post



    TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone.. I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?


    Now, once you have several different methods of doing this polyphonic tuning, some might be more advanced to allow more notes maybe, or maybe more accuracy, so the end result is that one might work a little better or give more control.  These are just POSSIBILITIES when comparing the different methods.


     


    The other aspect might be which would be which requires less processing due to how it's done.


     


    This is just off the top of my head any possible differences between one method and the other.


     


    I haven't tried either method, or have analyzed them, but it may end up that one is better than the other.


     


    Obviously, TC Electronics doesn't make a Logic Pro style app that's as commonly used, so TC Electronics could or does make a plug-in much like AutoTune or Melodyne.  But Apple would just have this possibly as a plug-in or just embedded in Logic Pro.


     


    Personally, I don't or wouldn't use these pitch correction software, nor do I recommend the usage of it, since I think it's a cheap trick to use for singers that really can't sing, but as a special effect it might be cool.


     


    From a purist point of view, any manipulation of a performance can degrade the overall recording, but unfortunately, that's how most of the popular music is being produced these days because they promote singers that can't sing.  You know, the Justin Bieber type singers.

  • Reply 9 of 20
    Let's hope they came up with a different method from Celemony's existing DNA - http://www.celemony.com/cms/index.php?id=dna - which is amazing but rather expensive.

    Also, there's a big difference between simply spotting the notes in polyphonic audio (the upcoming Ableton Live 9 will do that and spit out MIDI information) and allowing you to manipulate the polyphonic audio itself, which is what Celemony's DNA does. This story makes it sound like Apple is getting a patent on a way to do the former, which is less fun...
  • Reply 10 of 20
    moxommoxom Posts: 326member
    Come on Apple - just release Logic Pro X already!!!
  • Reply 11 of 20

    Quote:

    Originally Posted by drblank View Post


    Now, once you have several different methods of doing this polyphonic tuning, some might be more advanced to allow more notes maybe, or maybe more accuracy, so the end result is that one might work a little better or give more control.  These are just POSSIBILITIES when comparing the different methods.


     


    The other aspect might be which would be which requires less processing due to how it's done.


     


    This is just off the top of my head any possible differences between one method and the other.


     


    I haven't tried either method, or have analyzed them, but it may end up that one is better than the other.


     


    Obviously, TC Electronics doesn't make a Logic Pro style app that's as commonly used, so TC Electronics could or does make a plug-in much like AutoTune or Melodyne.  But Apple would just have this possibly as a plug-in or just embedded in Logic Pro.


     


    Personally, I don't or wouldn't use these pitch correction software, nor do I recommend the usage of it, since I think it's a cheap trick to use for singers that really can't sing, but as a special effect it might be cool.


     


    From a purist point of view, any manipulation of a performance can degrade the overall recording, but unfortunately, that's how most of the popular music is being produced these days because they promote singers that can't sing.  You know, the Justin Bieber type singers.



     


    Interesting to know that you actually get a patent for your own recipe no matter if it is the same dish.


    Sad how technology is being abused by the industry to fabricate "artists" as they please... lucky us there are still real talents like the black keys (just an example) who make it all analog and old school.. sounds so much better.


    First time I recall almost throwing up with an auto tuned / vocoded song has to be Cher. yikes!


     

  • Reply 12 of 20

    Quote:

    Originally Posted by drblank View Post


    Personally, I don't or wouldn't use these pitch correction software, nor do I recommend the usage of it, since I think it's a cheap trick to use for singers that really can't sing, but as a special effect it might be cool.


     


    From a purist point of view, any manipulation of a performance can degrade the overall recording, but unfortunately, that's how most of the popular music is being produced these days because they promote singers that can't sing.  You know, the Justin Bieber type singers.



    You're missing out, IMHO. First of all, monophonic or polyphonic tuning can not only be used for correcting vocals, but has tons of other valuable production and artistic applications, like mangling, say, pre-recorded guitar or piano recordings to different keys for experimentation and usage in other than originally planned productions. Sure, those pianos & guitars could be recorded again, and sometimes are after first using the treated audio as a guide, but often can be used just like that, saving time, money & effort. Or the tuning/correction might bring a *desired* artifact-ful effect that the producer *purposely* wants to use.


     


    Secondly, I know and have worked with numerous great singers who are amazing live and in the studio, but who occasionally are a little bit flat in the end of one big note here, and soft one there (which is only apparent when you get to listen to the recording over and over again and when you have it against dead-on synthesizers, for eg.) and in 9 out of 10 singers they're perfectly happy to have their tiny offs nudged in tune. It's not a cheap trick and doesn't make them bad singers at all, but it's the reality of today's music world where "things need to be perfect". I personally don't tune everything "straight", but try to have things in tune without it sounding unnatural, nor do I use T-Pain etc style gargling forced correction, that's just not my thing.


     


    BTW, you're not really on a steady ground with your "case" using Justin Bieber as an example of a singer that can't sing. Because that dude can. Did when he wasn't yet know, and still can.




  • Reply 13 of 20
    chabigchabig Posts: 641member

    Quote:

    Originally Posted by edubong View Post



    I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?


    It doesn't matter if there are similar products on the market. If this works differently and is unique, then Apple's method is patentable. Ideas and concepts are not patentable. Implementations are.

  • Reply 14 of 20
    john.bjohn.b Posts: 2,742member

    Quote:

    Originally Posted by edubong View Post



    TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone..


     


    Too bad TC can't be bothered to support open tunings...

  • Reply 15 of 20

    Quote:

    Originally Posted by Darude View Post


    Secondly, I know and have worked with numerous great singers who are amazing live and in the studio, but who occasionally are a little bit flat in the end of one big note here, and soft one there (which is only apparent when you get to listen to the recording over and over again and when you have it against dead-on synthesizers, for eg.) and in 9 out of 10 singers they're perfectly happy to have their tiny offs nudged in tune. It's not a cheap trick and doesn't make them bad singers at all, but it's the reality of today's music world where "things need to be perfect". I personally don't tune everything "straight", but try to have things in tune without it sounding unnatural, nor do I use T-Pain etc style gargling forced correction, that's just not my thing.



     


    Darude? You the musician from Finland? I actually have one of your songs. Nice to see you posting here.

  • Reply 16 of 20


    Celemony has had polyphonic pitch correction called DNA in their Melodyne editor for over a year. I've retuned guitars, vocals and strings with it. It works great. Celemony also has a program that fixes "Wow and Flutter".


    Again Apple is playing catch up.

  • Reply 17 of 20


    This is true.


    I created a string quartet out od multiple violin takes. By adjusting the formant I created realist violas and Cellos from the violins. This was in Melodyne.

  • Reply 18 of 20
    v5vv5v Posts: 1,357member

    Quote:

    Originally Posted by scottglasel View Post


    Celemony has had polyphonic pitch correction called DNA in their Melodyne editor for over a year. I've retuned guitars, vocals and strings with it. It works great. Celemony also has a program that fixes "Wow and Flutter".


    Again Apple is playing catch up.



     


    The Melodyne "app" approach is a PITA though. It's a nuisance having to "sample" audio into Melodyne before I can do anything with it.

  • Reply 19 of 20
    v5vv5v Posts: 1,357member


    [EDIT:] I decided to delete my comment because so far this is just a patent that hasn't even become a product so there's no way to know what it will or won't do. My comment came across like I have some clue what it will be capable of doing, and since I obviously don't, I take it back.

  • Reply 20 of 20

    Quote:

    Originally Posted by SpamSandwich View Post


     


    Darude? You the musician from Finland? I actually have one of your songs. Nice to see you posting here.



    Guilty as charged ;)  I've been a lurker for years, but haven't been compelled to post before :)

Sign In or Register to comment.