Apple Watch 'particle cloud' pairing method likely revealed in new patent
Lost amidst a sea of flashy hardware technologies introduced with Apple Watch was a new image-based pairing technique distinguished by a "particle cloud" effect. On Tuesday, Apple received a patent detailing what appears to be the bedrock of this unique pairing procedure, hinting at its use in future products.
The U.S. Patent and Trademark Office awarded Apple sequential U.S. Patent Nos. 9,022,291 and 9,022,292, both titled "Invisible optical label for transmitting information between computing devices," laying the foundation for machine readable codes like those seen with Apple Watch. Specifically, the invention outlines a method by which data-rich optical labels are encoded and decoded in a manner imperceptible to end users.
According to the patent, an optical label is used to transmit data from one electronic device to another, much like quick response codes (QR codes) that are themselves descendants of UPC barcodes. In fact, Apple's invention supports traditional 2D machine readable schemes, but applies a display process that makes the matrices invisible against a static or dynamic background.
Source: USPTO
The invention relies heavily on human pyschovisual perception. By encoding and coloring a specific portion of a matrix in two alternating frames, Apple can control how a user perceives that area's temporal chrominance, or essentially the color of the picture.
For example, first and second portions of frame A are encoded using two colors, perhaps blue and magenta. Third and forth portions generated on frame B correspond to the first and second portions of frame A are applied opposing colors orange and green. Alternating between the two frames, the temporal average equates to a lack of color, or gray. To ensure chroma space changes go unnoticed, frames alternate at frequencies of 60 fps to 120 fps and beyond.
Assigning temporal averages selectively across a matrix allows it to blend in with a variety of background images, from icons to wallpapers. For example, a portion of the alternating frames can be encoded to output a temporal average of blue to match the blue section of an image, while another area may correspond to an orange section.
Since the human eye is more sensitive to changes in luminance, or brightness, than chrominance, color changes to the specially encoded optical label will likely go unnoticed. For an imaging sensor, however, chrominance changes are easily distinguishable. By capturing encoded frames at a different rate than what is being displayed by a transmitting device, a recipient device is able to to pick out the underlying matrix for decoding and processing.
Finally, displaying matrices in alternating frames enables embeds in moving images, a major departure from static black-and-white or color QR codes.
It looks like Apple already applied its newly granted patent with Apple Watch. As seen in the image at the top of this article, Apple is using a visual pairing method to link Watch with iPhone. During the setup process, owners are instructed to use an iPhone to capture images of a dynamically generated optical code -- resembling a roiling Oort cloud -- displayed on their Watch.
As Apple's invention is designed to trick the human eye, it is difficult to prove inclusion in Apple Watch's pairing process. Taking the shipping version's pairing speed, accuracy and sophistication into account, however, Apple's invisible optical label technology seems like a good fit.
Apple's invisible optical label patents were filed for in July 2014 and credit Rudolph van der Merwe and Samuel G. Noble as inventors.
The U.S. Patent and Trademark Office awarded Apple sequential U.S. Patent Nos. 9,022,291 and 9,022,292, both titled "Invisible optical label for transmitting information between computing devices," laying the foundation for machine readable codes like those seen with Apple Watch. Specifically, the invention outlines a method by which data-rich optical labels are encoded and decoded in a manner imperceptible to end users.
According to the patent, an optical label is used to transmit data from one electronic device to another, much like quick response codes (QR codes) that are themselves descendants of UPC barcodes. In fact, Apple's invention supports traditional 2D machine readable schemes, but applies a display process that makes the matrices invisible against a static or dynamic background.
Source: USPTO
The invention relies heavily on human pyschovisual perception. By encoding and coloring a specific portion of a matrix in two alternating frames, Apple can control how a user perceives that area's temporal chrominance, or essentially the color of the picture.
For example, first and second portions of frame A are encoded using two colors, perhaps blue and magenta. Third and forth portions generated on frame B correspond to the first and second portions of frame A are applied opposing colors orange and green. Alternating between the two frames, the temporal average equates to a lack of color, or gray. To ensure chroma space changes go unnoticed, frames alternate at frequencies of 60 fps to 120 fps and beyond.
Assigning temporal averages selectively across a matrix allows it to blend in with a variety of background images, from icons to wallpapers. For example, a portion of the alternating frames can be encoded to output a temporal average of blue to match the blue section of an image, while another area may correspond to an orange section.
Since the human eye is more sensitive to changes in luminance, or brightness, than chrominance, color changes to the specially encoded optical label will likely go unnoticed. For an imaging sensor, however, chrominance changes are easily distinguishable. By capturing encoded frames at a different rate than what is being displayed by a transmitting device, a recipient device is able to to pick out the underlying matrix for decoding and processing.
Finally, displaying matrices in alternating frames enables embeds in moving images, a major departure from static black-and-white or color QR codes.
It looks like Apple already applied its newly granted patent with Apple Watch. As seen in the image at the top of this article, Apple is using a visual pairing method to link Watch with iPhone. During the setup process, owners are instructed to use an iPhone to capture images of a dynamically generated optical code -- resembling a roiling Oort cloud -- displayed on their Watch.
As Apple's invention is designed to trick the human eye, it is difficult to prove inclusion in Apple Watch's pairing process. Taking the shipping version's pairing speed, accuracy and sophistication into account, however, Apple's invisible optical label technology seems like a good fit.
Apple's invisible optical label patents were filed for in July 2014 and credit Rudolph van der Merwe and Samuel G. Noble as inventors.
Comments
To me, this patent seems to be something different... Something in the realm of 3D, virtual reality. I see an Apple app that scans an environment and recognizes encoded information that once decoded can be used to display information on Apple devices. Indoor mapping, treasure hunts, guided tours, furniture shopping, etc.
I did get a nervous feeling reading about encoded information that the human eye cannot see though. Even though the eye cannot "see" the information, the information is there and can be recognized on a subconscious level. That felt Orwellian to me.
I did get a nervous feeling reading about encoded information that the human eye cannot see though. Even though the eye cannot "see" the information, the information is there and can be recognized on a subconscious level. That felt Orwellian to me.
Don't worry, I am sure they coded a 'Don't sweat it' message in there also, so all good.
Thanks!
While reading the actual patent, I learned I was on target with Bluetooth and AirDrop, but off target with the 3D, virtual reality stuff. Oh well.
I could also see this as an in-person method for anonymous and secure funds transfer, a la Bitcoin or other blockchain-based transactions, including private contracts. Avoiding the Internet, email or texting ensures greater security and privacy.
But set-up was easy and smooth nonetheless.
I hope it is saying 'Buy AAPL now' every few seconds
This is what separates Apple from other companies. I think of CurrentC as a prime example. Instead of coming up with something secure (considering they're dealing with customer data and funds), they use chose to use those stupid QR codes.
So true (about the separation). Were the watches being paired via QR codes, for example, I could see someone generating a QR code with the data encoded to possibly pair their phone with the watch of someone near by? Something as simple as messing with friends... or something a lot more sinister. (Can an Apple Watch be paired with multiple phones though?)
Generating a QR code is a hell of a lot easier then an animated 'particle cloud' rendering generated on the fly, probably containing/encoding data (time/location/device orientation) that's being constantly updated and validated.
I did get a nervous feeling reading about encoded information that the human eye cannot see though. Even though the eye cannot "see" the information, the information is there and can be recognized on a subconscious level. That felt Orwellian to me.
Nothing to worry about, move along.
I can not remember the movie, but in the movie information was being moved about using embedded information in a picture. A person looking at the photo could not tell there was hidden information until the image was digitized and then translated was the information finally reveal. This is kind of what Apple is doing embedding information in a picture of sorts.
Honestly. patents should not be granted on things which Hollywood thought up first in a movie. Like cell phones, the inventor of the cell phone publicly stated he got the idea from watch Star Trek back in the 60"s
I can not remember the movie, but in the movie information was being moved about using embedded information in a picture. A person looking at the photo could not tell there was hidden information until the image was digitized and then translated was the information finally reveal. This is kind of what Apple is doing embedding information in a picture of sorts.
Honestly. patents should not be granted on things which Hollywood thought up first in a movie. Like cell phones, the inventor of the cell phone publicly stated he got the idea from watch Star Trek back in the 60"s
But an idea and its actual implementation, I would argue, are two totally separate things though. Not in defense of, but just as a distinction, there is a big enough difference in the patent that Apple has been granted (& not assuming that you're saying its should not have been granted), and the method in which a static image could be digitized and 'read'. Nothing is going to prevent someone else from encoding information into a pictures (static or not) (because to your point, it's been done before).
Just thought of this: at some point in Skyfall (Bond Film) there is the the scene where they were looking at what seemed like a mismosh of information (was it like a 'particle cloud' of sorts as well?) and it ends up being a 3D map of the London (?) tube system?
A walkie talkie vs a cell phone? Cell (tower based) vs. (short/long wave) radio?
Just one persons opinion.
Honestly. patents should not be granted on things which Hollywood thought up first in a movie. Like cell phones, the inventor of the cell phone publicly stated he got the idea from watch Star Trek back in the 60"s
They had car phones and handie-talkies from the 30's. Did Star Trek show a system in which low power cells reuse frequencies, and the cell phone is dynamically handed off between cells? Did Star Trek show a voice communication system where the cell towers communicate on an embedded data channel to the handset, controlling when the phone is handed off? Did Star Trek show a telephone system where cell towers send all calls to a central mobile telephone switching office, which switches the voice path between towers, allowing calls to travel over wide areas without being dropped? Did Star Trek show a telephone system where idle phones listen to a control channel that is shared over a subset of the cells for incoming calls?
The cell phone isn't so obvious, is it now? Clearly it was a Hollywood idea.
This is the problem with laypeople talking about patents. They don't understand how hard it is to make technology work.
What's a layperson?
im not so sure about the science behind that notion. im of the impression that there is no way you can perceive the embedded barcode or whatever it is.
"I can not remember the movie, but in the movie information was being moved about using embedded information in a picture."
movie Contact used this to imbed information in video image
er, no...it's not the ideas that get patented ("flying car"), it's the implementation of how it works that gets patented (details of your anti-grav propulsion engine).
just dreaming up some shit and declaring it prior art doesnt hold any water. ideas are free...ideas are the easy part. implementation is the hard part.
You may be thinking of the movie Contact, in which a video signal contained 3-D instructions to help build an interdimensional travel device.
What's a layperson?
A prostitute.
Makes me wonder about the entire universe, encoded with information that our human senses cannot perceive.
The universe is a giant quantum computer.
The answer is 42.