'Apple Glass' could use a virtual keyboard projected onto any surface

in General Discussion
Apple is working on multiple methods to use a keyboard with a pair of augmented reality glasses or other head-mounted device, including virtual keyboards projected onto surfaces.

Credit: LP Information
Credit: LP Information

The Cupertino tech giant is thought to be developing at least one head-mounted device. It's likely to be an Apple AR headset or AR glasses like the so-called "Apple Glass," but there are also indications that a virtual reality device is also in the works.

In a patent application published Oct. 15 and titled "Keyboard Operation with Head-mounted Device," Apple details several different ways that a user can use a keyboard with a pair of AR glasses or a similar product.

The patent gives some background on what the head-mounted device would entail. It could, for example, be a headset that sports lenses in front of the eyes and is equipped with several cameras. The display lenses, which could vision-correcting optical properties, could exclude a physical environment or overlay information on top of it.

When needing to type text, users could use this head-mounted device with either an actual physical keyboard or a "representation thereof through the display." Text that is typed on the keyboard could appear in the user's field of vision through the display lenses.

Most interestingly, however, are the alternative methods for text input that Apple explores here. They could include a virtual keyboard projected onto a touch-sensitive surface, or a keyboard projected onto any flat surface like a desk or tabletop.

Along with a tracking device, an AR headset could allow for use of any surface as a virtual keyboard. Credit: Apple
Along with a tracking device, an AR headset could allow for use of any surface as a virtual keyboard. Credit: Apple

"A head-mounted device can provide a virtual keyboard based on the arrangement of the user's hands with respect to a surface," the patent reads, adding that the device could detect where the keyboard should be by the position of the hands on the surface.

In any of these cases, Apple notes that a tracking device placed in close proximity to a user's hands could submit text to the AR device by using hand-tracking systems, inertial measurement units, or force detection sensors.

"It will be understood that combinations of the above can be provided such that operation of the keyboard is detected based on multiple detections," the patent reads.

Furthermore, these tracking sensors could allow a keyboard to function without needing to rely on signals. In other words, the typing would be detected by the headset, which could allow for keyboards with virtually no electronics within them, or virtual keyboards projected onto surfaces.

For projected keyboards, Apple notes that a user's hands could be made transparent to facilitate viewing the actual keys when typing. When it comes to editing text, hand gestures or other inputs could be used to select text and make changes.

An illustration of the hand transparency effect, which could allow users to better see the keyboard. Credit: Apple
An illustration of the hand transparency effect, which could allow users to better see the keyboard. Credit: Apple

Additionally, the patent suggests that users could also use an Apple Pencil or another stylus to draw or annotate on flat surfaces.

The patent lists Paul X. Wang, Keith K. Hendren, and Alex J. Lehmann as its inventors. Wang has worked on Apple patents relating to glass keyboards; Hendren has worked on MacBook keyboard technology; and Lehmann has been named as an inventor on bendable MacBook patents.

Apple files numerous patent applications weekly, and there's no guarantee that the technology described in them will ever make it to market. Similarly, patent applications don't give any indication when such features could arrive.


  • Reply 1 of 5
    This would be quite amazing, actually. 
  • Reply 2 of 5
    shaminoshamino Posts: 488member
    Been there, done that.  Projection keyboards have been around for decades.  I can't remember anybody actually using (or even wanting to use) one.  The one time I tried (using a demonstrator model in a store), I hated it - it was slow and inaccurate, and my fingers started hurting after typing on a solid desk surface.

  • Reply 3 of 5
    Projection keyboards have some good aspects but myself, and no one I know, who used one ever wanted to HAVE to use it. Then again, no one's tried it on a mass scale and done it well in real world situations. At least in my experience. It could be nice in fluid situations when you are not on a show floor or some convention presentation booth just 'trying the technology'. It could be an acceptable stop gap for quick use. And people can't complain about key bounce! 
  • Reply 4 of 5
    Nice but it could use haptic gloves to go with it. That way you could feel the keys before you pressed them.
  • Reply 5 of 5
    Not interested. Too many errors and not very user friendly.
Sign In or Register to comment.