Apple's virtual keyboard tech could lead to backside input for tablets
A newly-published Apple patent application reveals the company is investigating a system that helps users efficiently employ the touch type method, or typing on a keyboard without looking at one's hands, by dynamically remapping a keyboard's keys. The fiing also mentions that a backside sensor array can be embedded in a tablet to create a secondary input surface.
Apple's proposed "Configurable input device" offers a solution for both the well-versed power user and those who peck at keys with two fingers by augmenting the keyboard, as well as other input devices, with sensors. The discrete input members, or sensors, feed a processor data regarding the relative positioning of a user's digits, which can then be overlaid onto a virtual on-screen representation of the device.
The patent describes "discrete input members" as being any sensing device, from capacitive touch and pressure sensors to cameras and physical keys. Also mentioned is the inclusion of haptic feedback devices for further system enhancement.
Source: USPTO
From the patent application's background:
Keyboard with sensors, including haptic feedback capability.
For example, the virtual buttons can be dynamically assigned as the "home row" on a keyboard (a, s, d, f, etc. for a QWERTY keyboard) when a user's fingers are at rest. If the fingers move or become repositioned during typing, a user can reassign the keys by periodically holding a "resting position," which will dynamically change the keyboard's mapping.
Alternatively, the keyboard can merely display the virtual keys without dynamically remapping them, thus enabling an on-screen reference for touch typists. Yet another method describes user-configurable virtual keys that can serve a multitude of functions.
In this configuration, the technology can be advantageous for a tablet like the iPad, which uses its screen as a main input device. For example, the virtual representation of a user's fingers may be preferred when the device's entire screen is being used while viewing a movie or playing a video game.
As with the keyboard and laptop embodiments, a user's fingers are tracked and visually presented on-screen.
While the description is short, it alludes to possible future tech in Apple's iPad line that may offer new ways of interacting with the device. The study of similar backside input systems is well underway at companies like Microsoft, and a form of backside multitouch tech was recently implemented in a consumer device with Sony's Playstation Vita portable game console.
As is the case with many Apple patent applications, the technology may or may not make its way to consumers, however the invention could prove beneficial as devices become increasingly reliant on touchscreen input.
Touch Typing
Perhaps one of the most difficult, and arguably essential, skills to master when learning how to effectively use a computer or any machine that requires the use of input devices, is touch typing. Many schools offer classes specifically tailored for to teach the eyes-free method, however even trained users can sometimes stumble when switching between a mouse or trackpad and a keyboard.Apple's proposed "Configurable input device" offers a solution for both the well-versed power user and those who peck at keys with two fingers by augmenting the keyboard, as well as other input devices, with sensors. The discrete input members, or sensors, feed a processor data regarding the relative positioning of a user's digits, which can then be overlaid onto a virtual on-screen representation of the device.
The patent describes "discrete input members" as being any sensing device, from capacitive touch and pressure sensors to cameras and physical keys. Also mentioned is the inclusion of haptic feedback devices for further system enhancement.
Source: USPTO
From the patent application's background:
Going further, one embodiment calls for a keyboard or input device's buttons to be programmable, with configurable virtual buttons that can be remapped based on the relative location of a user's hand. Built-in sensors can determine the position of hands and fingers in relation to the input surface, enabling the virtual buttons to be configured accordingly.Often users look down at the keyboard when they type. This is not ergonomic and may lead to neck and muscle strain. Additionally, it is generally not advantageous for accurate and/or fast typing. Specifically, when users are focused on the keyboard, they cannot tell what has been input and typically revisit the input data to edit it. Moreover, when someone accustomed to looking at the keyboard tries to view the display instead of the keyboard, errors and frustration may occur as they cannot tell where their hands are relative to the keys they want to use.
Keyboard with sensors, including haptic feedback capability.
For example, the virtual buttons can be dynamically assigned as the "home row" on a keyboard (a, s, d, f, etc. for a QWERTY keyboard) when a user's fingers are at rest. If the fingers move or become repositioned during typing, a user can reassign the keys by periodically holding a "resting position," which will dynamically change the keyboard's mapping.
Alternatively, the keyboard can merely display the virtual keys without dynamically remapping them, thus enabling an on-screen reference for touch typists. Yet another method describes user-configurable virtual keys that can serve a multitude of functions.
Backside Tablet Input
Buried deep within the patent's illustrations is the brief mention of a "tablet computer" with a sensor array under the surface of its backside, opposite the display. The sensors can be positioned in a linear layout, a grid or "any other suitable pattern" to achieve desired functionality. In contrast to the keyboard embodiment, the tablet version presents icons or other graphics instead of a virtual overlay, offering a more intuitive user experience.In this configuration, the technology can be advantageous for a tablet like the iPad, which uses its screen as a main input device. For example, the virtual representation of a user's fingers may be preferred when the device's entire screen is being used while viewing a movie or playing a video game.
As with the keyboard and laptop embodiments, a user's fingers are tracked and visually presented on-screen.
While the description is short, it alludes to possible future tech in Apple's iPad line that may offer new ways of interacting with the device. The study of similar backside input systems is well underway at companies like Microsoft, and a form of backside multitouch tech was recently implemented in a consumer device with Sony's Playstation Vita portable game console.
As is the case with many Apple patent applications, the technology may or may not make its way to consumers, however the invention could prove beneficial as devices become increasingly reliant on touchscreen input.
Comments
Any invention in virtual or physical space that aids in typing accurately must be a good thing... until at least, the keyboard becomes obsolete!
Sorry AI: You get neglected tonight!
Backside touchscreen reminds me a lot of
Do not know why no devices using this tech have been released yet (PS vita has only a limited implementation of this technology)
tablets with keyboards?
Quote:
Originally Posted by mausz
Backside touchscreen reminds me a lot of
Do not know why no devices using this tech have been released yet (PS vita has only a limited implementation of this technology)
The underside of the PS-Vita has a touch interface too, i dont like they way its done on the Vita tho.
Quote:
Originally Posted by GTR
And in related news ...(cough)...Plants vs Zombies 1.5 (proper HD with some extra stuff) just released!
Sorry AI: You get neglected tonight!
Dynamically remapping the keyboard using haptic feedback is a huge asset for touch typing. My pops never learned touch in school and I don't think he would relate to anything new about hand placement/key location even though he's not really fast/accurate consistently. Not sure how Plants and Zombies relates to haptic technology,or maybe that's a jk.
... Ahhh.. I was gunna say - i struggle to type with my fingers; apple want me to type with my backside now???
Is there an echo, or did you not read the whole post?
Quote:
Originally Posted by mausz
Backside touchscreen reminds me a lot of
Do not know why no devices using this tech have been released yet (PS vita has only a limited implementation of this technology)
Quote:
Originally Posted by Zozman
The underside of the PS-Vita has a touch interface too, i dont like they way its done on the Vita tho.
Is there an echo, or did you not read the whole post?
If the computer senses that you've shifted your hands to a different position (let's say your left pinkie is on the "s" instead of the "a" and then adjusts and then you intuitively realize your hands are in the wrong position and then you switch back to proper home position and it has to adjust again, I could see this being more annoying than helpful because you know it's not going to get it right a lot of the time. However, making intelligent guesses about what you really wanted to type could be useful for auto correction on devices that use virtual keyboards, where the error rates are much higher due to hitting an adjacent key to the one you really wanted.
Haptic keyboards are a big positive IMO, not just for feedback, but for feeling the key position in the first place (assuming that's technically feasible). I use remote control software on my iPhone, but the reality is that the physical remote is still better because I don't have to look down to use it. I can feel where the proper key is located.
I will make sure to use it with caution.
It's good to see they are thinking about virtual input. I don't think the camera/sensor on top of the laptop/iMac display would work too well but certainly on the base - it depends on the sensor of course. This sort of thing will take a lot of experimentation to get it just right but Apple staff have the experience to pull this off. Their iOS input gestures were all done amazingly well. There are so many things they can use this tech for.
If they have an on-screen keyboard, it would be best behaving like the iPad one though. An overlay just gets in the way. They don't really have to have a keyboard at all but just a strip of letters showing in a coverflow-like display. It would accelerate the more left/right your finger goes. In an app like TextEdit, it can print the next letter your finger is hovering over before you tap to confirm.
[IMG ALT=""]http://forums.appleinsider.com/content/type/61/id/11711/width/500/height/1000[/IMG]
This sort of thing might not be good for games but it would in some cases. The action strip can be customised so RTS games can have a row of custom items/commands. Photoshop,Final Cut etc would have a tools pallette that could change to letters when required. Something like shift or tab can be a gesture. Multi-touch would make it trickier but the left hand can be used for actions instead of selections e.g caps, deletion, punctuation, change character set and so on.
The automatic remapping of the keyboard based on the user's placement of their fingers, it genius.
I love how this company continues to innovate!
Quote:
Originally Posted by techno
The automatic remapping of the keyboard based on the user's placement of their fingers, it genius.
I love how this company continues to innovate!
It's all user-centric. the tech has to be/feel dead-easy and totally natural. Even if it does exist now, until Apple can make it *that*, it's unfit to carry the Apple logo.
Frick's sake, not the "touchscreen on the back" rumor again. This should have died in the first article thread about it over two years ago, not even continued to the iPad launch where people were actually disappointed it didn't happen…
Originally Posted by aBeliefSystem
Whatever next, tablets with keyboards?
Keyboards with tablets.
Quote:
Originally Posted by mausz
Backside touchscreen reminds me a lot of
Do not know why no devices using this tech have been released yet (PS vita has only a limited implementation of this technology)
The Motorola FlipSide phone had this. It was... an interesting experiment. I didn't like it very much.
Quote:
Originally Posted by techno
The automatic remapping of the keyboard based on the user's placement of their fingers, it genius.
I love how this company continues to innovate!
Sure, and this genius can't add a Dvorak layout as a virtual keyboard for 2007, 2008, 2009, 2010, 2011, 2012 - for 6 years.