Apple's camera-toting Apple Watch band employs facial recognition tech for clear FaceTime ...

Posted:
in Apple Watch edited November 13
Apple is investigating methods of incorporating camera hardware into its Apple Watch product lineup, focusing specifically on systems that avoid the inherent pitfalls of embedding image capturing equipment in a device not well-suited to the task.

Apple Watch Camera
Source: USPTO


According to a patent granted to Apple by the U.S. Patent and Trademark Office on Tuesday, the tech giant is looking to broaden the feature set of its wearable by incorporating a novel camera system capable of automatically cropping in on subject matter, tracking objects like a user's face and generating angle-adjusted avatars for FaceTime calls.

Apple's U.S. Patent No. 10,129,503 for an "Image-capturing watch" details a hardware and software solution that makes a camera-toting Apple Watch not only feasible, but useful.

As noted in the property's language, such designs would allow users to be less reliant on their iPhones, perhaps ditching the handset altogether in certain situations. With a camera-enabled Watch, users would be able to leave bulky handheld equipment behind when playing sports, working out or performing other strenuous activities.

A workable smartwatch camera solution is difficult to execute. Simply graphing a traditional handheld image capture system, like an iPhone camera, onto Apple Watch would be a user experience nightmare. Owners would be hard pressed to aim the camera at a subject without moving the entire platform -- their wrist -- to check framing, while at the same time dealing with a host of other concerns, from video stability to ergonomics.

To address the litany of anticipated issues, Apple proposes a system in which a wide-angle lens is fitted to a sensor built into a watch band. Alternatively, multiple cameras might simultaneously capture image data that is seamlessly stitched together to create one continuous image with a field of view that is much wider than any single-lens camera can provide.

The resulting wide-angle image reduces the need for precise aiming. Instead of framing subject matter beforehand, the final photo or video can be cropped after the fact to home in on a target or point of interest. This process can be conducted algorithmically or manually by the user.

Apple notes the technique not only works well for initial image acquisition -- users need only point the lens in a target's general direction -- but also adds a new level of convenience not usually available to portable camera users. Crop-after-capture can assist in recording spur of the moment events, for example.

Apple Watch Camera
Illustration of teleconference call technique.


Beyond basic photography, the patent delves into teleconferencing. Here, too, Apple provides thoughtful context to the proposed technology.

In some embodiments, the watch processes a captured image and applies facial recognition software to recognize a user's face. The device dynamically outputs a cropped image centered around the identified visage, making continuous on-the-fly adjustments to keep the face in frame even if the watch is in motion.

Perhaps most interesting is a contingency for rectifying unflattering "up nose" angles. If a user were to employ Watch in a FaceTime call, and do so without holding the device directly in front of their face as promised by Apple's auto-cropping tech, the resulting image would at times be captured from a low vantage point.

To negate "up nose" shots, Apple proposes outputting a representation of a user's face that appears to be taken from a straight-on perspective.

As noted in the patent, Watch's onboard processor might generate an angle-corrected image or representation of a user's face by compiling stored facial data, perhaps information gathered during biometric registration. Motion data captured by the camera and processed by the watch is then mapped onto the computer generated representation, which mimics a user's facial expressions and movements in real time.

Alternatively, source motion data can be used to inform the movements of a non-human avatar like Apple's Animoji and Memoji.

Apple Watch Camera
Illustration of automated cropping (top) and "up nose" angle correction.


Whether Apple intends to bring its Apple Watch camera band technology remains unknown. The company was in 2015 rumored to integrate a FaceTime camera into a future Apple Watch model, though the rumblings failed to bear fruit.

Apple's camera watch band patent was first filed for in September 2016 and credits Seung Wook Kim and Megan A. McClain as its inventors.

Comments

  • Reply 1 of 10
    radarthekatradarthekat Posts: 2,710moderator
    Great example of working backward from a desired result/use case.  This is why I trust Apple with the details and trust them to make the right compromises when required.  
    macguiSoundJudgmentracerhomie3d_2StrangeDaysphilboogierepressthisjony0
  • Reply 2 of 10
    In theory... they basically have all that is needed from their Face ID-tech. Essentially using a 3D model of the face to intelligently apply a manipulated livestream on top of it.

    Taking the logic one step further this solution would only require a fresh view of the face at the beginning of the call, and the rest could be done by manipulating the face based on the voice only. (Ditching the background, to “focus” on the face/user, would be a requirement for this to look realistic.)

    In the late 90s I met with a German-American startup in Santa Monica that basically did this; but due to the hardware requirements their work, at that time, of course required a powerful desktop just to animate a computer generated character live.
    edited November 13 caladanianrepressthis
  • Reply 3 of 10
    entropysentropys Posts: 1,374member
    Apple is more likely to think through the use case instead of just chucking a “feature” in.  

    That said, regardless of any software fix for nostril view, FaceTime calls from your watch. if the watch fakes your moving image, that would be impressively scary.

    I still get the willies from my wife’s Animoji. It looks exactly like Mrs Entropy in her twenties.  A long time ago. Sigh.
    edited November 13 radarthekatjdgazGeorgeBMacAppleExposedrepressthis
  • Reply 4 of 10
    entropys said:
    Apple is more likely to think through the use case instead of just chucking a “feature” in.  

    That said, regardless of any software fix,  there would also be the wonderful view up the nostrils likely from all too many FaceTime calls from your watch. It would make the Dell XPS camera seem almost circumspect.
    Feel free to read the article before replying to it the next time...
    From the article: Perhaps most interesting is a contingency for rectifying unflattering "up nose" angles. If a user were to employ Watch in a FaceTime call, and do so without holding the device directly in front of their face as promised by Apple's auto-cropping tech, the resulting image would at times be captured from a low vantage point. 

    StrangeDaysMacPro
  • Reply 5 of 10
    I like the idea of the camera in the Watch band and not the Watch. Some people work in facilities that don't allow cellphones with cameras unless the cameras have been destroyed, i.e. drilled out. And how wants that done to their slick phone.
    208236
    Off duty, you want a camera on your phone, add the Cam-Band.  Going to work, swap it out with a 'work compliant' band.

    This could be another Apple patent that doesn't make it to a product. Maybe a prototype gets made for fun around the lab, or maybe Apple just doesn't want somebody to build a better mousetrap.
    caladanian
  • Reply 6 of 10
    mac_128mac_128 Posts: 3,280member
    macgui said:
    I like the idea of the camera in the Watch band and not the Watch. Some people work in facilities that don't allow cellphones with cameras unless the cameras have been destroyed, i.e. drilled out. And how wants that done to their slick phone.
    208236
    Off duty, you want a camera on your phone, add the Cam-Band.  Going to work, swap it out with a 'work compliant' band.

    This could be another Apple patent that doesn't make it to a product. Maybe a prototype gets made for fun around the lab, or maybe Apple just doesn't want somebody to build a better mousetrap.
    Yes but that's likely a very small percentage of Apple's user base, and as you say, it's already a problem with even having a cell phone in some locations. 

    I used my original iPad for years when I went into locations which did not allow cameras, and was one of the few able to use an electronic device. Everyone else had their cell phones held at the door.

    Using the bands create unique problems for Apple as rumored during the development of the watch, so it remains to be seen whether they turn to them to augment the features of the watch. Most customers are likely going to want the camera in the watch regardless of the bands they wear. 


  • Reply 7 of 10
    eightzeroeightzero Posts: 2,199member
    macgui said:
    I like the idea of the camera in the Watch band and not the Watch. Some people work in facilities that don't allow cellphones with cameras unless the cameras have been destroyed, i.e. drilled out. And how wants that done to their slick phone.
    208236
    Off duty, you want a camera on your phone, add the Cam-Band.  Going to work, swap it out with a 'work compliant' band.

    This could be another Apple patent that doesn't make it to a product. Maybe a prototype gets made for fun around the lab, or maybe Apple just doesn't want somebody to build a better mousetrap.
    I've often wanted a snap and shoot on my wrist. I take far more snapshots (awww...cute dog. click. By the time I've fumbled to my phone, the dog woke up.)
  • Reply 8 of 10
    eightzero said:
    macgui said:
    I like the idea of the camera in the Watch band and not the Watch. Some people work in facilities that don't allow cellphones with cameras unless the cameras have been destroyed, i.e. drilled out. And how wants that done to their slick phone.
    208236
    Off duty, you want a camera on your phone, add the Cam-Band.  Going to work, swap it out with a 'work compliant' band.

    This could be another Apple patent that doesn't make it to a product. Maybe a prototype gets made for fun around the lab, or maybe Apple just doesn't want somebody to build a better mousetrap.
    I've often wanted a snap and shoot on my wrist. I take far more snapshots (awww...cute dog. click. By the time I've fumbled to my phone, the dog woke up.)
    Are you willing to wear these instead?


  • Reply 9 of 10
    radarthekatradarthekat Posts: 2,710moderator
    It just hit me that this article uses the correct term; facial recognition, and that points out the difference.  Facial recognition is the ability to recognized various aspects of human faces, such as eyes, nose, mouth, smiling, frowning, etc. and would be used here to acquire and track a person’s face (to crop in on it) and facial expressions (to replicate these on a pre-captured straight-on image or onto an Avatar).  The important difference between facial recognition and face recognition is that facial recognition is not concerned with who’s face it is, while face recognition is all about determining whether a face belongs to a specific person.  They are different recognition tasks/use cases, and that’s the reason FaceID shouldn’t be described as facial recognition.  It’s not.  It’s face recognition.  
  • Reply 10 of 10

    Apple Watch Camera
     

    Mild-mannered Bruce Banner becomes The Incredible Hulk!
    radarthekat
Sign In or Register to comment.