Apple patent could bring polyphonic tuning to Logic Pro
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent for polyphonic note detection, or the ability to parse out single pitches and tones from a plurality of voices, a technology that may make its way to an expected version of the company's Logic Pro music editing software.
Source: USPTO
Aptly-named "Polyphonic note detection," U.S. Patent No. 8,309,834 describes a method in which a polyphonic audio signal is converted and sampled against a reference note to distinguish individual voices.
Prevailing technology is limited to detecting single notes, such as a person singing or the picking of a single string on a guitar, which can be used for a number of audio applications like tuning, music creation and pitch correction. With polyphonic detection, a user would be able to tune all strings on a guitar with a single strum.
While there are certain algorithms on the market for digital audio workstations, such as Melodyne from Celemony, Apple has yet to include the feature in its own Logic DAW.
The patented method is clearly aimed at software, as it calls for a processor-enabled solution that breaks down a portion of a polyphonic audio signal by first converting it from a time domain to a frequency domain. Peak frequencies are detected using either a reference signal or a decibel threshold, and if a predetermined number of integer-interval harmonic partials are detected, a note is recorded. In other words, the system relies on harmonics, or harmonic partials, which are the basic result of resonant sound waves created by an instrument or human voice.
Fundamental frequencies, or the lowest frequency off of which harmonics and partials are based, act as a guide to which integer-interval harmonic partials are tested. If a fundamental frequency is detected, a note will be registered. The deviation from a perfect integer-interval can be tracked to find the inharmonicity, or distance from the nearest harmonic, in a polyphonic signal.
Three peaks at integer-interval harmonic partial frequencies must be detected to notate a fundamental frequency.
Illustration of a polyphonic audio signal and its peak integer-interval harmonic partials.
Note fundamental frequency threshold at 30 dB.
When the system finishes detecting the particular notes, it can determine what chord is being played, if any. For example, if the notes detected are C#, E, G#, it would be determined that the polyphonic signal is a C sharp minor chord.
In one embodiment, the detection process described can repeated every 256 samples, or every 5.8 milliseconds for a CD-quality recording, which has a standard sampling frequency of 44,100 Hz. Implementations of the invention are also described in which MIDI notes are used for reference against the processed signal.
According to the invention, the process described can be repeated "until each note in the polyphonic audio signal has been detected."
Apple's patent can be executed by software-only, hardware-only, or a combination of both, meaning it has the possibility of appearing in an upcoming version of Logic. Beyond rumors, there has been little evidence that Apple's DAW will feature polyphonic note detection, however rival software does offer similar solutions as plug-ins or wholly-licensed technology.
Source: USPTO
Aptly-named "Polyphonic note detection," U.S. Patent No. 8,309,834 describes a method in which a polyphonic audio signal is converted and sampled against a reference note to distinguish individual voices.
Prevailing technology is limited to detecting single notes, such as a person singing or the picking of a single string on a guitar, which can be used for a number of audio applications like tuning, music creation and pitch correction. With polyphonic detection, a user would be able to tune all strings on a guitar with a single strum.
While there are certain algorithms on the market for digital audio workstations, such as Melodyne from Celemony, Apple has yet to include the feature in its own Logic DAW.
The patented method is clearly aimed at software, as it calls for a processor-enabled solution that breaks down a portion of a polyphonic audio signal by first converting it from a time domain to a frequency domain. Peak frequencies are detected using either a reference signal or a decibel threshold, and if a predetermined number of integer-interval harmonic partials are detected, a note is recorded. In other words, the system relies on harmonics, or harmonic partials, which are the basic result of resonant sound waves created by an instrument or human voice.
Fundamental frequencies, or the lowest frequency off of which harmonics and partials are based, act as a guide to which integer-interval harmonic partials are tested. If a fundamental frequency is detected, a note will be registered. The deviation from a perfect integer-interval can be tracked to find the inharmonicity, or distance from the nearest harmonic, in a polyphonic signal.
Three peaks at integer-interval harmonic partial frequencies must be detected to notate a fundamental frequency.
Illustration of a polyphonic audio signal and its peak integer-interval harmonic partials.
Note fundamental frequency threshold at 30 dB.
When the system finishes detecting the particular notes, it can determine what chord is being played, if any. For example, if the notes detected are C#, E, G#, it would be determined that the polyphonic signal is a C sharp minor chord.
In one embodiment, the detection process described can repeated every 256 samples, or every 5.8 milliseconds for a CD-quality recording, which has a standard sampling frequency of 44,100 Hz. Implementations of the invention are also described in which MIDI notes are used for reference against the processed signal.
According to the invention, the process described can be repeated "until each note in the polyphonic audio signal has been detected."
Apple's patent can be executed by software-only, hardware-only, or a combination of both, meaning it has the possibility of appearing in an upcoming version of Logic. Beyond rumors, there has been little evidence that Apple's DAW will feature polyphonic note detection, however rival software does offer similar solutions as plug-ins or wholly-licensed technology.
Comments
http://www.engadget.com/2011/06/25/spectral-layers-promises-to-let-you-edit-audio-photoshop-style/
Quote:
Originally Posted by edubong
TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone.. I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?
The patent will be for the precise method to create the end result, not the end result itself. There are many ways to skin a cat, and each method of skinning can be patented.
Quote:
Originally Posted by GTBuzz
Could it be for specific voice detection out of several voices ?
I was thinking similar.
I know a few people who will go on a music-making spree if this is true…
Quote:
Originally Posted by edubong
TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone.. I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?
The only way they can do this is by using a different algorithm. The end result is the same, but the method in how it's done might be different.
Quote:
Originally Posted by edubong
TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone.. I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?
Now, once you have several different methods of doing this polyphonic tuning, some might be more advanced to allow more notes maybe, or maybe more accuracy, so the end result is that one might work a little better or give more control. These are just POSSIBILITIES when comparing the different methods.
The other aspect might be which would be which requires less processing due to how it's done.
This is just off the top of my head any possible differences between one method and the other.
I haven't tried either method, or have analyzed them, but it may end up that one is better than the other.
Obviously, TC Electronics doesn't make a Logic Pro style app that's as commonly used, so TC Electronics could or does make a plug-in much like AutoTune or Melodyne. But Apple would just have this possibly as a plug-in or just embedded in Logic Pro.
Personally, I don't or wouldn't use these pitch correction software, nor do I recommend the usage of it, since I think it's a cheap trick to use for singers that really can't sing, but as a special effect it might be cool.
From a purist point of view, any manipulation of a performance can degrade the overall recording, but unfortunately, that's how most of the popular music is being produced these days because they promote singers that can't sing. You know, the Justin Bieber type singers.
Also, there's a big difference between simply spotting the notes in polyphonic audio (the upcoming Ableton Live 9 will do that and spit out MIDI information) and allowing you to manipulate the polyphonic audio itself, which is what Celemony's DNA does. This story makes it sound like Apple is getting a patent on a way to do the former, which is less fun...
Quote:
Originally Posted by drblank
Now, once you have several different methods of doing this polyphonic tuning, some might be more advanced to allow more notes maybe, or maybe more accuracy, so the end result is that one might work a little better or give more control. These are just POSSIBILITIES when comparing the different methods.
The other aspect might be which would be which requires less processing due to how it's done.
This is just off the top of my head any possible differences between one method and the other.
I haven't tried either method, or have analyzed them, but it may end up that one is better than the other.
Obviously, TC Electronics doesn't make a Logic Pro style app that's as commonly used, so TC Electronics could or does make a plug-in much like AutoTune or Melodyne. But Apple would just have this possibly as a plug-in or just embedded in Logic Pro.
Personally, I don't or wouldn't use these pitch correction software, nor do I recommend the usage of it, since I think it's a cheap trick to use for singers that really can't sing, but as a special effect it might be cool.
From a purist point of view, any manipulation of a performance can degrade the overall recording, but unfortunately, that's how most of the popular music is being produced these days because they promote singers that can't sing. You know, the Justin Bieber type singers.
Interesting to know that you actually get a patent for your own recipe no matter if it is the same dish.
Sad how technology is being abused by the industry to fabricate "artists" as they please... lucky us there are still real talents like the black keys (just an example) who make it all analog and old school.. sounds so much better.
First time I recall almost throwing up with an auto tuned / vocoded song has to be Cher. yikes!
Quote:
Originally Posted by drblank
Personally, I don't or wouldn't use these pitch correction software, nor do I recommend the usage of it, since I think it's a cheap trick to use for singers that really can't sing, but as a special effect it might be cool.
From a purist point of view, any manipulation of a performance can degrade the overall recording, but unfortunately, that's how most of the popular music is being produced these days because they promote singers that can't sing. You know, the Justin Bieber type singers.
You're missing out, IMHO. First of all, monophonic or polyphonic tuning can not only be used for correcting vocals, but has tons of other valuable production and artistic applications, like mangling, say, pre-recorded guitar or piano recordings to different keys for experimentation and usage in other than originally planned productions. Sure, those pianos & guitars could be recorded again, and sometimes are after first using the treated audio as a guide, but often can be used just like that, saving time, money & effort. Or the tuning/correction might bring a *desired* artifact-ful effect that the producer *purposely* wants to use.
Secondly, I know and have worked with numerous great singers who are amazing live and in the studio, but who occasionally are a little bit flat in the end of one big note here, and soft one there (which is only apparent when you get to listen to the recording over and over again and when you have it against dead-on synthesizers, for eg.) and in 9 out of 10 singers they're perfectly happy to have their tiny offs nudged in tune. It's not a cheap trick and doesn't make them bad singers at all, but it's the reality of today's music world where "things need to be perfect". I personally don't tune everything "straight", but try to have things in tune without it sounding unnatural, nor do I use T-Pain etc style gargling forced correction, that's just not my thing.
BTW, you're not really on a steady ground with your "case" using Justin Bieber as an example of a singer that can't sing. Because that dude can. Did when he wasn't yet know, and still can.
Quote:
Originally Posted by edubong
I don't understand how apple can get a patent for an invention that has been already invented and turned into a real product that's been out for years. how does this work?
It doesn't matter if there are similar products on the market. If this works differently and is unique, then Apple's method is patentable. Ideas and concepts are not patentable. Implementations are.
Quote:
Originally Posted by edubong
TC electronic of denmark already makes a hardware polyphonic tuner, called polytune... even comes as an app for iPhone..
Too bad TC can't be bothered to support open tunings...
Quote:
Originally Posted by Darude
Secondly, I know and have worked with numerous great singers who are amazing live and in the studio, but who occasionally are a little bit flat in the end of one big note here, and soft one there (which is only apparent when you get to listen to the recording over and over again and when you have it against dead-on synthesizers, for eg.) and in 9 out of 10 singers they're perfectly happy to have their tiny offs nudged in tune. It's not a cheap trick and doesn't make them bad singers at all, but it's the reality of today's music world where "things need to be perfect". I personally don't tune everything "straight", but try to have things in tune without it sounding unnatural, nor do I use T-Pain etc style gargling forced correction, that's just not my thing.
Darude? You the musician from Finland? I actually have one of your songs. Nice to see you posting here.
Celemony has had polyphonic pitch correction called DNA in their Melodyne editor for over a year. I've retuned guitars, vocals and strings with it. It works great. Celemony also has a program that fixes "Wow and Flutter".
Again Apple is playing catch up.
This is true.
I created a string quartet out od multiple violin takes. By adjusting the formant I created realist violas and Cellos from the violins. This was in Melodyne.
Quote:
Originally Posted by scottglasel
Celemony has had polyphonic pitch correction called DNA in their Melodyne editor for over a year. I've retuned guitars, vocals and strings with it. It works great. Celemony also has a program that fixes "Wow and Flutter".
Again Apple is playing catch up.
The Melodyne "app" approach is a PITA though. It's a nuisance having to "sample" audio into Melodyne before I can do anything with it.
[EDIT:] I decided to delete my comment because so far this is just a patent that hasn't even become a product so there's no way to know what it will or won't do. My comment came across like I have some clue what it will be capable of doing, and since I obviously don't, I take it back.
Quote:
Originally Posted by SpamSandwich
Darude? You the musician from Finland? I actually have one of your songs. Nice to see you posting here.
Guilty as charged