Apple awarded pair of 3D user interface patents related to computer vision

Posted:
in General Discussion
A pair of newly awarded patents could hint at new ways of interacting with Apple devices, potentially using 3D motion and space to interact with a Mac or iOS device.




Apple was awarded two new patents this week detailing 3D input with advanced camera systems. They were discovered on Tuesday by AppleInsider.

The first patent, No. 9,417,706, covers a hand motion observed by a camera, forcing a user interface to transition from one state to another. The second, patent No. 9,417,763, addresses a mobile device's determination of its location in space, using all sensors to determine relative motion and identifying a frame of reference, such as a user's face.

No. 9,417,706 requires only a camera connected to a computing device. The patent references a situation where a specific motion must be done by a user to enable an action. While a desktop computer with attached webcam is shown in the patent documentation, the technology granted is not limited to desktop computing.

An example given in the patent document is of a user sitting, and raising an arm over a shoulder, and then back down. In the patent document's example, a "push" or a "wave" at the computer could initiate a monitoring action in the operating system, which could to tell a device to start watching for future gestures. Another example illustrated by the patent is a user controlling items on the computer screen in 3D.




A specific application mentioned in the first patent is an unlock gesture, similar to the iOS "swipe to unlock," but performed in the air rather than on a touch-sensitive surface.

The second patent, number 9,417,763 has augmented reality implications. Should the patent ever be implemented, it suggests a device that combines yaw, pitch, and roll information with user facial monitoring. A device with technology suggested by the patent can not only keep track of a device's location relative to the user, but also one that can monitor the user's eye position, which the filing says will allow for "more realistic virtual 3D depictions of the objects on the device's display."

A practical use of patent No. 9,417,763 is actuation of high-powered GPU routines, which would be disabled to preserve the battery when the user is either not looking at the device or cannot see the screen of the device.

Patent No. 9,417,706 was applied for in May 2014, and awarded in May 2015. Patent No. 9,417,763 was filed in December of 2014 and awarded in April 2015. Both were published on Tuesday.

Neither patent appears to be currently in use by any Apple product. Apple and suspected shell companies file for hundreds of patents per year, and many never see use.

Comments

  • Reply 1 of 12
    dysamoriadysamoria Posts: 2,140member
    "Granted", not "awarded". Macrumors gets it right.

    Also, like most computer tech patents, these just look like prior art...
  • Reply 2 of 12
    sockrolidsockrolid Posts: 2,788member
    All good ideas.  But maybe the first implementation of 3d gesture recognition should be in the car.
    BMW already has 3D gesture recognition in the new 7-Series as part of the "iDrive" system.
    The 7-Series infotainment system also includes a touch-screen like Apple CarPlay.

    (Off-topic: Apple has a history of releasing and iterating foundation technologies to prepare for future products.
    For example, they released iTunes in 2001, which paved the way for the iPod, iTunes Store, and iPhone and App Store.
    I would argue that Swift, with its heavy focus on eliminating entire categories of common programming errors,
    is laying the foundation for car control software that needs to be vastly more reliable than typical Objective-C apps.
    I think it's possible that Apple's top management decided that Obj-C was not reliable enough for automotive control, so
    one of the milestones on the "Apple Car" project might have been "better, more reliable programming language."
    Just a thought.)
    oneof52lolliver
  • Reply 3 of 12
    appexappex Posts: 687member
    Just about time for true 3D Apple displays, including a brand new 4K Thunderbolt display.
  • Reply 4 of 12
    mcdavemcdave Posts: 1,140member
    Input for an AR UI?
  • Reply 5 of 12
    nolamacguynolamacguy Posts: 4,758member
    dysamoria said:

    Also, like most computer tech patents, these just look like prior art...
    just so i can weigh your opinion on prior art and these patents appropriately -- are you a patent attorney? are you an engineer? have you reviewed the patents in question to see how their implementations may vary from the other patents you're referring to? 
    edited August 2016 badmonk
  • Reply 6 of 12
    flaneurflaneur Posts: 4,509member
    sockrolid said:
    All good ideas.  But maybe the first implementation of 3d gesture recognition should be in the car.
    BMW already has 3D gesture recognition in the new 7-Series as part of the "iDrive" system.
    The 7-Series infotainment system also includes a touch-screen like Apple CarPlay.

    (Off-topic: Apple has a history of releasing and iterating foundation technologies to prepare for future products.
    For example, they released iTunes in 2001, which paved the way for the iPod, iTunes Store, and iPhone and App Store.
    I would argue that Swift, with its heavy focus on eliminating entire categories of common programming errors,
    is laying the foundation for car control software that needs to be vastly more reliable than typical Objective-C apps.
    I think it's possible that Apple's top management decided that Obj-C was not reliable enough for automotive control, so
    one of the milestones on the "Apple Car" project might have been "better, more reliable programming language."
    Just a thought.)
    Interesting POV.
    patchythepirate
  • Reply 7 of 12
    flaneurflaneur Posts: 4,509member

    appex said:
    Just about time for true 3D Apple displays, including a brand new 4K Thunderbolt display.
    Are you suggesting autostereoscopic, polarized with glasses, or . . . ? 
  • Reply 8 of 12
    appexappex Posts: 687member
    Whatever to watch 3D SBS movies on Mac with passive LCD glasses. As can be done on smart TV or on Windows PC for years now.
  • Reply 9 of 12
    noivadnoivad Posts: 186member
    appex said:
    Whatever to watch 3D SBS movies on Mac with passive LCD glasses. As can be done on smart TV or on Windows PC for years now.
    Unfortunately, this ability is a niche application among PCs. I’ve only ever seen these at trade shows costing thousands of dollars. Apple could bring it to the masses for much less like they did, Mice, Optical Drives, Laptop design, virtual keyboards, tablets, etc. All of these existed before Apple started offering them but they popularized them. So, forget PC/Mac debates: propagation of novel interfaces has been Apple’s bread & butter for decades, and everyone—even non-Mac users—benefit.
    patchythepirate
  • Reply 10 of 12
    noivadnoivad Posts: 186member
    dysamoria said:
    "Granted", not "awarded". Macrumors gets it right.

    Also, like most computer tech patents, these just look like prior art...
    While this could be true—IANAL. It doesn’t matter with the current “first to patent” standard in place. Apple has to at least try to protect its future interfaces from trolls by patenting the hell out of everything they think up. Everyone I’ve heard from in the patent game (people working with the EFF, patent lawyers writings, etc) thinks the system is broken to various degrees & in need of reform. Everyone has their own ideas on how to fix it, but honestly, with legal issues, this could take decades from whenever they decide to act. So, patenting is a modicum of protection—not from other companies copying the idea, but from lawsuits.
    edited August 2016 patchythepirate
  • Reply 11 of 12
    For me, true 3D means no eyeglasses. If Nintendo can do it on their DS models, Apple can do it on a much larger scale. It may take a few more years but it will happen.
    edited August 2016
  • Reply 12 of 12

    Obviously I'm not smart enough to see the difference between this motion detection using gestures and cameras, and Kinect's motion detection using gestures and cameras.

Sign In or Register to comment.