Apple's camera-toting Apple Watch band employs facial recognition tech for clear FaceTime ...
Apple is investigating methods of incorporating camera hardware into its Apple Watch product lineup, focusing specifically on systems that avoid the inherent pitfalls of embedding image capturing equipment in a device not well-suited to the task.
Source: USPTO
According to a patent granted to Apple by the U.S. Patent and Trademark Office on Tuesday, the tech giant is looking to broaden the feature set of its wearable by incorporating a novel camera system capable of automatically cropping in on subject matter, tracking objects like a user's face and generating angle-adjusted avatars for FaceTime calls.
Apple's U.S. Patent No. 10,129,503 for an "Image-capturing watch" details a hardware and software solution that makes a camera-toting Apple Watch not only feasible, but useful.
As noted in the property's language, such designs would allow users to be less reliant on their iPhones, perhaps ditching the handset altogether in certain situations. With a camera-enabled Watch, users would be able to leave bulky handheld equipment behind when playing sports, working out or performing other strenuous activities.
A workable smartwatch camera solution is difficult to execute. Simply graphing a traditional handheld image capture system, like an iPhone camera, onto Apple Watch would be a user experience nightmare. Owners would be hard pressed to aim the camera at a subject without moving the entire platform -- their wrist -- to check framing, while at the same time dealing with a host of other concerns, from video stability to ergonomics.
To address the litany of anticipated issues, Apple proposes a system in which a wide-angle lens is fitted to a sensor built into a watch band. Alternatively, multiple cameras might simultaneously capture image data that is seamlessly stitched together to create one continuous image with a field of view that is much wider than any single-lens camera can provide.
The resulting wide-angle image reduces the need for precise aiming. Instead of framing subject matter beforehand, the final photo or video can be cropped after the fact to home in on a target or point of interest. This process can be conducted algorithmically or manually by the user.
Apple notes the technique not only works well for initial image acquisition -- users need only point the lens in a target's general direction -- but also adds a new level of convenience not usually available to portable camera users. Crop-after-capture can assist in recording spur of the moment events, for example.
Illustration of teleconference call technique.
Beyond basic photography, the patent delves into teleconferencing. Here, too, Apple provides thoughtful context to the proposed technology.
In some embodiments, the watch processes a captured image and applies facial recognition software to recognize a user's face. The device dynamically outputs a cropped image centered around the identified visage, making continuous on-the-fly adjustments to keep the face in frame even if the watch is in motion.
Perhaps most interesting is a contingency for rectifying unflattering "up nose" angles. If a user were to employ Watch in a FaceTime call, and do so without holding the device directly in front of their face as promised by Apple's auto-cropping tech, the resulting image would at times be captured from a low vantage point.
To negate "up nose" shots, Apple proposes outputting a representation of a user's face that appears to be taken from a straight-on perspective.
As noted in the patent, Watch's onboard processor might generate an angle-corrected image or representation of a user's face by compiling stored facial data, perhaps information gathered during biometric registration. Motion data captured by the camera and processed by the watch is then mapped onto the computer generated representation, which mimics a user's facial expressions and movements in real time.
Alternatively, source motion data can be used to inform the movements of a non-human avatar like Apple's Animoji and Memoji.
Illustration of automated cropping (top) and "up nose" angle correction.
Whether Apple intends to bring its Apple Watch camera band technology remains unknown. The company was in 2015 rumored to integrate a FaceTime camera into a future Apple Watch model, though the rumblings failed to bear fruit.
Apple's camera watch band patent was first filed for in September 2016 and credits Seung Wook Kim and Megan A. McClain as its inventors.
Source: USPTO
According to a patent granted to Apple by the U.S. Patent and Trademark Office on Tuesday, the tech giant is looking to broaden the feature set of its wearable by incorporating a novel camera system capable of automatically cropping in on subject matter, tracking objects like a user's face and generating angle-adjusted avatars for FaceTime calls.
Apple's U.S. Patent No. 10,129,503 for an "Image-capturing watch" details a hardware and software solution that makes a camera-toting Apple Watch not only feasible, but useful.
As noted in the property's language, such designs would allow users to be less reliant on their iPhones, perhaps ditching the handset altogether in certain situations. With a camera-enabled Watch, users would be able to leave bulky handheld equipment behind when playing sports, working out or performing other strenuous activities.
A workable smartwatch camera solution is difficult to execute. Simply graphing a traditional handheld image capture system, like an iPhone camera, onto Apple Watch would be a user experience nightmare. Owners would be hard pressed to aim the camera at a subject without moving the entire platform -- their wrist -- to check framing, while at the same time dealing with a host of other concerns, from video stability to ergonomics.
To address the litany of anticipated issues, Apple proposes a system in which a wide-angle lens is fitted to a sensor built into a watch band. Alternatively, multiple cameras might simultaneously capture image data that is seamlessly stitched together to create one continuous image with a field of view that is much wider than any single-lens camera can provide.
The resulting wide-angle image reduces the need for precise aiming. Instead of framing subject matter beforehand, the final photo or video can be cropped after the fact to home in on a target or point of interest. This process can be conducted algorithmically or manually by the user.
Apple notes the technique not only works well for initial image acquisition -- users need only point the lens in a target's general direction -- but also adds a new level of convenience not usually available to portable camera users. Crop-after-capture can assist in recording spur of the moment events, for example.
Illustration of teleconference call technique.
Beyond basic photography, the patent delves into teleconferencing. Here, too, Apple provides thoughtful context to the proposed technology.
In some embodiments, the watch processes a captured image and applies facial recognition software to recognize a user's face. The device dynamically outputs a cropped image centered around the identified visage, making continuous on-the-fly adjustments to keep the face in frame even if the watch is in motion.
Perhaps most interesting is a contingency for rectifying unflattering "up nose" angles. If a user were to employ Watch in a FaceTime call, and do so without holding the device directly in front of their face as promised by Apple's auto-cropping tech, the resulting image would at times be captured from a low vantage point.
To negate "up nose" shots, Apple proposes outputting a representation of a user's face that appears to be taken from a straight-on perspective.
As noted in the patent, Watch's onboard processor might generate an angle-corrected image or representation of a user's face by compiling stored facial data, perhaps information gathered during biometric registration. Motion data captured by the camera and processed by the watch is then mapped onto the computer generated representation, which mimics a user's facial expressions and movements in real time.
Alternatively, source motion data can be used to inform the movements of a non-human avatar like Apple's Animoji and Memoji.
Illustration of automated cropping (top) and "up nose" angle correction.
Whether Apple intends to bring its Apple Watch camera band technology remains unknown. The company was in 2015 rumored to integrate a FaceTime camera into a future Apple Watch model, though the rumblings failed to bear fruit.
Apple's camera watch band patent was first filed for in September 2016 and credits Seung Wook Kim and Megan A. McClain as its inventors.
Comments
Taking the logic one step further this solution would only require a fresh view of the face at the beginning of the call, and the rest could be done by manipulating the face based on the voice only. (Ditching the background, to “focus” on the face/user, would be a requirement for this to look realistic.)
In the late 90s I met with a German-American startup in Santa Monica that basically did this; but due to the hardware requirements their work, at that time, of course required a powerful desktop just to animate a computer generated character live.
That said, regardless of any software fix for nostril view, FaceTime calls from your watch. if the watch fakes your moving image, that would be impressively scary.
I still get the willies from my wife’s Animoji. It looks exactly like Mrs Entropy in her twenties. A long time ago. Sigh.
208236
Off duty, you want a camera on your phone, add the Cam-Band. Going to work, swap it out with a 'work compliant' band.
This could be another Apple patent that doesn't make it to a product. Maybe a prototype gets made for fun around the lab, or maybe Apple just doesn't want somebody to build a better mousetrap.
I used my original iPad for years when I went into locations which did not allow cameras, and was one of the few able to use an electronic device. Everyone else had their cell phones held at the door.
Using the bands create unique problems for Apple as rumored during the development of the watch, so it remains to be seen whether they turn to them to augment the features of the watch. Most customers are likely going to want the camera in the watch regardless of the bands they wear.
Mild-mannered Bruce Banner becomes The Incredible Hulk!