Apple awarded PrimeSense patent for 3D virtual keyboard that would allow typing on air

Posted:
in General Discussion edited February 2015
While the reason behind Apple's acquisition of PrimeSense remains unknown, the Cupertino company was awarded a PrimeSense patent on Tuesday describing 3D camera system allowing for an entirely new type of virtual keyboard.




The U.S. Patent and Trademark office awarded Apple with U.S. Patent 8,959,013, entitled "Virtual Keyboard for a Non-Tactile Three Dimensional User Interface." First discovered by AppleInsider, the newly granted invention describes a virtual keyboard that would allow users to type on air.

The system would use a 3D capturing device, such as an array of cameras, that would detect a user's hand movements. A user would be able to type and control a virtual keyboard by moving their fingers or hands, and the system would determine which words and keystrokes a user intends to type.

In the patent, Apple notes that its system could capture 3D information by scanning an "interactive area" located in front of a display screen. In this sense, users could interact with objects on a Mac or HDTV by using their hands in front of them, without physically touching a keyboard, remote, or touchscreen.

If the concept sounds familiar, it's because PrimeSense is the same company that powered the technology behind the first generation of Microsoft's Xbox Kinect.

However, even in its second generation hardware, Kinect does not allow the kind of precise gesture input described in the latest Apple patent, which was originally filed by PrimeSense in 2011. Apple's invention would feature cameras accurate enough to detect subtle fingertip movements for a virtual keyboard or otherwise.

PrimeSense
PrimeSense's Carmine short-range sensor.


To help determine which keys a user intends to press, Apple's system would use a "language model" similar to its QuickType suggestions in iOS 8. This would provide a "best guess of the user's intended input," the filing reads, which would allow users to enter more quickly.

Apple's awarded patent is credited to PrimeSense employees Micha Galor, Ofir Or, Shai Litvak, and Erez Sali.

Apple's first awarded patent via the PrimeSense acquisition came last December, and it described a projection-based 3D mapping solution for advanced gesture input.

PrimeSense was founded in 2005, gaining notoriety in 2010 when Microsoft licensed its infrared motion-tracking, depth-sensing chip for the Kinect accessory for Xbox 360. Apple eventually bought Primesense in 2013 at a price tag said to be between $345 million and $360 million.

Comments

  • Reply 1 of 20
    As if typing on glass wasn't tactile-less enough...here comes air typing¡
  • Reply 2 of 20
    No, really. It could add new meaning to MacBook Air, iPad Air.
  • Reply 3 of 20

    No thanks.

  • Reply 4 of 20
    Can it in work in a hybrid hydrogen electric car dashboard?......had to ask :)
  • Reply 5 of 20
    wizard69wizard69 Posts: 13,377member
    Sometimes tech goes in the wrong direction. I still think Apple needs to put more focus on voice recognition and processing right on the device. Siri has certianly improved of late but it isn't good enough and the trip to the server is a joke. Seriously why go to a server when you want to set an option on the device.
  • Reply 6 of 20
    mjtomlinmjtomlin Posts: 2,446member

    Lame application... but if they've actually been able to capture this much detail in real-time, then they're really coming along with this technology from PrimeSense.

  • Reply 7 of 20
    Wow, clearly designed for the windows mindset.
  • Reply 8 of 20
    Here we go again with another installment of generic images used for patent announcements. Love that c. 1995 tower with clunky webcam attachment. Can't seem to locate any AOL install floppies on this gentleman's desk, though. Looks like he's using some type of assistive accessibility keyboard for Windows '95, from what I can make out on the screen.

    Thankful we haven't seen any close-ups of that creepy, weird hand recently.
  • Reply 9 of 20
    Many of you are wining like little [email protected]#hes. I am sure if Apple releases something remotely close to this it will be well thought out. Forward thinking is needed...not saying every idea works but keep an open mind.
  • Reply 10 of 20
    Frankly, this is the kind of junk you'd find on SkyMall pages...
    http://m.brookstone.com/laser-projection-virtual-keyboard
  • Reply 11 of 20
    boredumbboredumb Posts: 1,418member

    Gosh, I sure hope Blackberry doesn't have any patents on air...

  • Reply 12 of 20
    fallenjtfallenjt Posts: 4,034member
    Quote:
    Originally Posted by Suddenly Newton View Post



    Frankly, this is the kind of junk you'd find on SkyMall pages...

    http://m.brookstone.com/laser-projection-virtual-keyboard

    No is not. The SkyMall crap is light based keyboard reflected on a surface and your fingers have to strike the right keys. This is camera based system which calibrates the positions of your hands/fingers with proximity. It doesn't matter where you move your hand within the camera zone, it still works just like the way Kinect captures your image in Xbox game. That means you can put a dummy keyboard right in front of you and type on it and it should work like you actually connect that keyboard to the system..LOL.

  • Reply 13 of 20

    Gonna help with configuration of my

     

     

    air guitar. 

  • Reply 14 of 20
    quinneyquinney Posts: 2,528member
    quadra 610 wrote: »
    Gonna help with configuration of my


    air guitar. 

    you beat me to it. I was thinking a MIDI air drums controller.
  • Reply 15 of 20
    MarvinMarvin Posts: 14,548moderator
    You don't have to type in the air like this:


    [VIDEO]


    With the sensor positioned correctly, it should be able to track your hands in a resting position so you can tap out letters on your knee on a sofa without a keyboard into an Apple TV interface or iPad without an add-on keyboard.
  • Reply 16 of 20
    MacProMacPro Posts: 19,382member
    Marvin wrote: »
    You don't have to type in the air like this:


    [VIDEO]


    With the sensor positioned correctly, it should be able to track your hands in a resting position so you can tap out letters on your knee on a sofa without a keyboard into an Apple TV interface or iPad without an add-on keyboard.

    Exactly. I bet a lot of the folks mocking this enjoyed Minority Report and are failing to make the connection. :D

    1000
  • Reply 17 of 20
    fallenjtfallenjt Posts: 4,034member
    Quote:

    Originally Posted by Marvin View Post



    You don't have to type in the air like this:









    With the sensor positioned correctly, it should be able to track your hands in a resting position so you can tap out letters on your knee on a sofa without a keyboard into an Apple TV interface or iPad without an add-on keyboard.

    Now, that's fking cool way to type. I hope it works for other languages too.

  • Reply 18 of 20
    quadra 610 wrote: »
    Gonna help with configuration of my


    air guitar. 


    Wait! Are you that "Quadro?" The 1993 World Air Guitar champion "Quadro?"
  • Reply 19 of 20



    PrimeSense was founded in 2005, gaining notoriety in 2010 when....

     

    Gained "notoriety"? How so?

     

    This is the second time I am seeing the exact same sentence on articles around PrimeSense. The last one being: http://forums.appleinsider.com/t/183798/apples-first-patent-reassignment-from-primesense-buy-hints-at-3d-mapping-on-apple-tv-iphone ;

  • Reply 20 of 20
    MarvinMarvin Posts: 14,548moderator
    Exactly. I bet a lot of the folks mocking this enjoyed Minority Report and are failing to make the connection. :D

    Yep although I wouldn't say the Minority Report setup is the way to go. The qwerty keyboard has been given a level of importance when it comes to language input but it has a limited design:

    http://www.businessinsider.com/chinese-keyboards-2011-9?op=1
    http://www.businessinsider.com/apple-keyboard-2011-7

    People have complained about the software keyboard in iOS but that's mainly the tactile part. From a functionality point of view, software inputs are much more flexible. We have to map functionality onto key combinations. Why do we use command-s to save files rather than tap an icon that represents saving a file? Why do we hit b to enable a brush in Photoshop and j for the heal brush? We simply don't have contextual representations to do it directly.

    The keyboard really needs to be a thing of the past. We only have it because of typewriters. Letters don't need to be arranged the way they are because children aren't taught on typewriters any more, they are taught the alphabet and keyboards present a letter layout that is nothing like it.

    When it comes to keyboard-less devices like phones and tablets, gesture input can provide the higher functionality of the software keyboard without touching the display. Microsoft has focused on sticking qwerty keyboards to the tablets but it should really be a gesture sensor or touch pad. The pad can be narrow because keys can be arranged in a single line. It would take character input for Asian users.

    The precision of these sensors is high enough that it will be as reliable as pressing a key. They can detect 0.01mm distances. They can also anticipate your next gesture so they can type ahead before you enter a character. They have no moving parts so would be reliable and feel consistent across devices.
Sign In or Register to comment.