Evidence of Apple's tablet-like input interface reappears

[AppleInsider]

In a new patent application revealed this week, Apple has again disclosed plans for a multi-touch surface that could accommodate two full hands and distinguish between palms and individual fingers for typing, gestures and more.

The application, filed by Morrison and Foerster LLP in Los Angeles in June 2009 on behalf of Apple, expands on information first revealed by AppleInsider in early 2008. The massive document details a hand-based system that would allow "unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device."

The document notes that input with a stylus, mouse, keyboard and voice recognition are all options that work well in specific circumstances, but are not dynamic enough to address the many needs of users. But, it states, many of those needs can be met with touch-sensitive technology that can recognize a number of hand configurations.

"Many attempts have been made to embed pointing devices in a keyboard so the hands do not have to leave typing position to access the pointing device... The limited movement range and resolution of these devices, leads to poorer pointing speed and accuracy than a mouse, and they add mechanical complexity to keyboard construction," the application reads. "Thus there exists a need in the art for pointing methods with higher resolution, larger movement range, and more degrees of freedom yet which are easily accessible from typing hand positions."



The described system in the patent application would individually detect all ten fingers and separate palms on a person's hand, giving the ability to type, write, draw and interact with a device large enough to support multiple hands. Examples of the touchscreen capabilities include resting of hands, measuring when a hand or fingers touches and leaves the surface, interpreting taps from one finger as mouse button clicks, but disregarding a tap from two fingers, and more. Activities done with multiple fingers are referred to as "chords."



Such a system could ergonomically adapt to individual hand sizes, eliminate the need for a stylus and mouse, would require minimal typing force. The application notes that there are other patents for touchscreen devices that negate the need for a keyboard or mouse, but states that Apple's method is unique because it addresses both needs.

Typing is a large part of the lengthy application. The document goes into great detail about how a multi-touch interface could distinguish what keys a set of hands intend to type on the surface. It discusses pressure on the sides or center of individual fingers and palms, and how to interpret those various signals.



Key points of the invention, as described in the application, include:

Integrating and distinguishing different types of input, such as typing, multiple degree-of-freedom manipulation, and handwriting, via different hand configurations that are easy to use and recognize.

Includes an electronic system which minimizes the number of sensing electrodes to allow easier understanding of a variety of hand configurations.

Provide a multi-touch surface that is contoured to be comfortable and ergonomic under extended use.

Provide tactile key or hand position feedback without interfering with a hand resting on the smooth touch-sensitive surface.

Provide images of "flesh proximity" to a variety of sensors that can distinguish hand configuration.

Understand when the user wants cursor motion, and ignore commands when deceleration by the user is detected.

Understand the movement of two or more hands to allow manipulation of two-dimensional electronic documents, like rotation and scaling of photos.


The application is partially credited to Wayne Westerman of Fingerworks, a company absorbed by Apple several years ago as part of its quest to deliver iPhone and a new generation of input devices.

Though long-rumored, Apple has not yet released any details about its anticipated tablet. Sources have told AppleInsider that the device will not appear until early 2010. Since co-founder Steve Jobs returned to the company after a leave due to health reasons, the project has reportedly been the CEO's number one focus.

Apple has been at work on the project for many years, and the hardware has seen numerous internal iterations, only to be reset a number of times. The current device is believed to have a 10-inch screen, 3G connectivity, and sport a custom-made chip from P.A. Semi.

[ View this article at AppleInsider.com ]

[estolinski]

Can't wait to see this thing.

[zindako]

I feel sorry for the netbook market already, watch out Microsoft, you're about to find yourself hobbling behind trying to get your very own tablet device once apple drops this baby, and fail horribly at its inception and interface.

[JohnnyKrz]

To me, this almost sounds more like a potential new input device for desktops which would replace the keyboard, mouse and pretty much any other type of input device.

[Bobo Decosta]

That the tablet is Jobs number on focus actually scares me. I hope this is just a rumor because otherwise it means Apple can't live without Jobs. Releasing the tablet without the help of Jobs would actually have proved that Apple can live without Jobs. Just hoping that Jobs realized that and it's just an egotrip of him.

[jwyatt]

Here's hoping that the complexity of this UI reflects the notion that the tablet will run the full Mac OS and not the just iphone OS. While whatever the tablet ends up being I'm sure will be awesome I'm hoping for a device that is capable of running iwork, pages etc. as well as iphone apps. Rumors keeping going back and fourth on this topic...exciting.

[Roc Ingersol]

Originally Posted by JohnnyKrz

To me, this almost sounds more like a potential new input device for desktops which would replace the keyboard, mouse and pretty much any other type of input device.

I don't see those as alternative devices.

If the 10" tablet exists, it could easily function as a USB monitor and/or input device when connected.

[Roc Ingersol]

Originally Posted by jwyatt

Here's hoping that the complexity of this UI reflects the notion that the tablet will run the full Mac OS and not the just iphone OS.

There is 0 chance that Apple (of all companies) will roll out a device that has no keyboard or mouse, running software that was designed and built for the keyboard and mouse. Desktop UIs barely scale down gracefully to 10" screens to begin with. But trying to use touch to navigate UI elements that expect mouse precision is right out.

I'm not even convinced that Apple would greenlight an ultra-mobile device running software that wasn't specifically designed for that exact device (With the appropriate priorities on screen, UI, battery usage, available processing power, etc) let alone release something with less horsepower than a macbook of several years ago with the express purpose of running desktop software.

[jwyatt]

Originally Posted by Roc Ingersol

There is 0 chance that Apple (of all companies) will roll out a device that has no keyboard or mouse, running software that was designed and built for the keyboard and mouse. Desktop UIs barely scale down gracefully to 10" screens to begin with. But trying to use touch to navigate UI elements that expect mouse precision is right out.

I'm not even convinced that Apple would greenlight an ultra-mobile device running software that wasn't specifically designed for that exact device (With the appropriate priorities on screen, UI, battery usage, available processing power, etc) let alone release something with less horsepower than a macbook of several years ago with the express purpose of running desktop software.

I'm not sure I entirely agree with that assessment, but I'll leave it at that. Guess we'll see next year.

[GQB]

Originally Posted by JohnnyKrz

To me, this almost sounds more like a potential new input device for desktops which would replace the keyboard, mouse and pretty much any other type of input device.

Hmmm... kinda like a 'touch table', huh?

[sflocal]

"Minority Report" here we come!

[melgross]

From what we've seen literally over the decades, tablets are one of the hardest computer forms to do.

Other than the iPhone/Touch and those that have followed, there have been three waves of tablets. None have been successful. The current MS OS based tablets have been at about 1% of purchases for years, with no indication that sales will pick up.

If Apple does come out with this, they'll have to overcome a lot of resistance.

One difference now is that most computer input for consumer use is no longer alpha-numeric, but mouse based.

Multitouch can work very well for that, and this multitouch "keyboard" would serve well for the little input required for filling out forms etc.

But a standard keyboard is a bit over 10.5" in length, excluding the numeric pad. A 10.6" diag screen will be too short, so the keyboard would be a bit cramped. They'll have to figure out a way around that.

[shadow]

Originally Posted by jwyatt

Here's hoping that the complexity of this UI reflects the notion that the tablet will run the full Mac OS and not the just iphone OS. While whatever the tablet ends up being I'm sure will be awesome I'm hoping for a device that is capable of running iwork, pages etc. as well as iphone apps. Rumors keeping going back and fourth on this topic...exciting.

The idea that the tablet should run either iPhone OS or desktop OS is utterly wrong to begin with. iPhone and Mac share the same Foundation with different UI layer - UIKit and AppKit respectively. Each is designed to fit the specifics of the device including screen size, input methods etc. Could you imagine iPhone-style full screen table view on a tablet? Or desktop-sized pop-up menu on a touch device? Both don't make any sense. A tablet might be able to run iPhone app in a separate window, but it should be capable to do more.

Originally Posted by Roc Ingersol

There is 0 chance that Apple (of all companies) will roll out a device that has no keyboard or mouse, running software that was designed and built for the keyboard and mouse. Desktop UIs barely scale down gracefully to 10" screens to begin with. But trying to use touch to navigate UI elements that expect mouse precision is right out.

I'm not even convinced that Apple would greenlight an ultra-mobile device running software that wasn't specifically designed for that exact device (With the appropriate priorities on screen, UI, battery usage, available processing power, etc) let alone release something with less horsepower than a macbook of several years ago with the express purpose of running desktop software.

That's more to the point.

[8CoreWhore]

Intriguing!

[shadow]

Originally Posted by melgross

But a standard keyboard is a bit over 10.5" in length, excluding the numeric pad. A 10.6" diag screen will be too short, so the keyboard would be a bit cramped. They'll have to figure out a way around that.

They will do what they did on the iPhone: context sensitive keyboards - for URLs, mail, numeric or general purpose. The last one will have the punctuation away from the base layout.

[digitalclips]

Originally Posted by melgross

From what we've seen literally over the decades, tablets are one of the hardest computer forms to do.

Other than the iPhone/Touch and those that have followed, there have been three waves of tablets. None have been successful. The current MS OS based tablets have been at about 1% of purchases for years, with no indication that sales will pick up.

If Apple does come out with this, they'll have to overcome a lot of resistance.

One difference now is that most computer input for consumer use is no longer alpha-numeric, but mouse based.

Multitouch can work very well for that, and this multitouch "keyboard" would serve well for the little input required for filling out forms etc.

But a standard keyboard is a bit over 10.5" in length, excluding the numeric pad. A 10.6" diag screen will be too short, so the keyboard would be a bit cramped. They'll have to figure out a way around that.

I am not too worried. Every innovation Apple ever comes out with is met with skepticism however most end up changing the World and leave competitors scrambling trying to copy them. As to over coming resistance, remember they have an almost lunatic following (I am one) and the iTunes ecosystem to feed off.

I also suspect SJ is going to do with the publishing industry what he did with the music industry and have most popular magazines and newspapers available from iTunes shortly after launch.

And one thing more ... whatever that is

[anantksundaram]

This is probably a really dumb question, But I'll ask anyway. Is there any finiteness to the life of a capacitance-based input device. In other words, could, say, a keyboard based on that be expected to last for many, many years?

[melgross]

Originally Posted by shadow

The idea that the tablet should run either iPhone OS or desktop OS is utterly wrong to begin with. iPhone and Mac share the same Foundation with different UI layer - UIKit and AppKit respectively. Each is designed to fit the specifics of the device including screen size, input methods etc. Could you imagine iPhone-style full screen table view on a tablet? Or desktop-sized pop-up menu on a touch device? Both don't make any sense. A tablet might be able to run iPhone app in a separate window, but it should be capable to do more.



That's more to the point.

I'n not sure either is correct. I don't see a tablet with a click keyboard. I don't agree that the iPhone OS which IS based on OS X, is the same thing.

It's running on a different processor, and as you said, it doesn't have the GUI. That makes it different.

It can't run OS X programs because of this.

Will iPhone programs run on a big screen? Well, they would have to if Apple is going to base this on the iPhone OS, and expects to have 100 thousand reasons for people buying it ready at launch.

Apple could have multitasking on this, with multiple programs showing up at once.

Many programs would likely work well expanded to a larger size, and others wouldn't take too much work to allow it.

The problem is the processor.

[~ufo~]

Now this, to me at least, is really interesting.

Consider having an iMac like desktop.

with a touch tablet flat in front of it, kind of like the touch wacoms, but in the form of a portable tablet.

I, as an iPhone used, have found myself touching my iMac's screen quite a few times in vein to click on something. Now, as nice as it would be to have a 24" multitouch screen on my iMac, we can hopefully all agree that it is not the most ergonomic of input interfaces.

if you were to have your home desktop iMac type machine, be it touch screen or not, and get rid of the keyboard and mouse altogether replacing it by a 10" touch smart touch screen now that would be a whole new approach to computing.

After you are done working at home, you just take the tablet with you, which is an autonomous device on it's own, as the apple tablet is proposed to be.

I can truly see a lot of merit in this.
It is both the desktop and laptop redefined.

I would have no problem paying $600-800 for it either.

Any takers?

[melgross]

Originally Posted by shadow

They will do what they did on the iPhone: context sensitive keyboards - for URLs, mail, numeric or general purpose. The last one will have the punctuation away from the base layout.

That's fine, but doesn't cover typing out a long letter.

They'll have to decide what these things are going to do.

[Jeff K-C]

I suspect that we are about to be introduced to the next revolutionary Apple product.

It seems likely to me that once more, this is going to be a device beyond what anybody anticipated. Gaming; telephone; magazine, book and newspaper reader; internet; email; iphone and/or OSX-like applications; GPS/mapping device; two handed input have all been mentioned. If this is as much a pet project of SJ's as has been reported, I don't pretend to see how any or all of the above may come together, but I'm thinkin' we may see a jaw dropper.

I just keep thinking back to all of the predictions about the iphone, and how very short they fell from what was actually released.

[melgross]

Originally Posted by digitalclips

I am not too worried. Every innovation Apple ever comes out with is met with skepticism however most end up changing the World and leave competitors scrambling trying to copy them. As to over coming resistance, remember they have an almost lunatic following (I am one) and the iTunes ecosystem to feed off.

I also suspect SJ is going to do with the publishing industry what he did with the music industry and have most popular magazines and newspapers available from iTunes shortly after launch.

And one thing more ... whatever that is

I expect they will figure it out in some way. But predicting what the public will buy isn't easy.

With the iPod, it was seen that the concept was becoming popular.

With the iPhone, it was known the concept was popular.

With the tablet, the concept has been seen to be a failure.

That's a very different area to break into. They have to turn a very unpopular concept into a popular one. Can they do that?

Remember the Cube. Great idea, beautiful device, but a failure for several reasons.

Can Apple really make a tablet costing between $800 and $1,000, and make a good case for it?

We see that the price estimates keep moving up. As the screen gets larger, so does the price. And so does the clumsiness.

If, as some have written, they can replace the MB with a tablet, that might work, but this doesn't look to be that.

[anonymouse]

Originally Posted by JohnnyKrz

To me, this almost sounds more like a potential new input device for desktops which would replace the keyboard, mouse and pretty much any other type of input device.

I agree, especially the part about it being ergonomically shaped.

[Cubert]

It would seem like there would have to be some sort of tactile feedback so that you wouldn't have to look at the keys to avoid accidentally hitting two at once.

Otherwise, we will have a lot of pissed off high school typing teachers.
("Achtung!!! Eyes on the screen!")

[melgross]

Originally Posted by anantksundaram

This is probably a really dumb question, But I'll ask anyway. Is there any finiteness to the life of a capacitance-based input device. In other words, could, say, a keyboard based on that be expected to last for many, many years?

Essentially, they have unlimited life, esp if coated with glass, as done in Apple's products, and some others.

[Cubert]

Originally Posted by jwyatt

Here's hoping that the complexity of this UI reflects the notion that the tablet will run the full Mac OS and not the just iphone OS. While whatever the tablet ends up being I'm sure will be awesome I'm hoping for a device that is capable of running iwork, pages etc. as well as iphone apps. Rumors keeping going back and fourth on this topic...exciting.

My guess is that it will be closer to the iPhone OS than OS X. I also think it will be able to run apps for both OS'es. You could probably install Photoshop on it but it may not be pretty trying to run it. Kinda like installing Photoshop on an old iBook - nothing is preventing you from doing it (assuming Universal binary, of course) but it wouldn't be pretty.

[Cubert]

"Provide images of "flesh proximity" to a variety of sensors that can distinguish hand configuration."

Just what in the world does this mean?

[Jeff K-C]

Originally Posted by Cubert

It would seem like there would have to be some sort of tactile feedback so that you wouldn't have to look at the keys to avoid accidentally hitting two at once.

Unless this is a different concept. What if there are no keys to hit, in the traditional sense? What struck me from the diagram, and what the input device would be sensing, is that wherever you place your hands on the surface could be "home", defining your keyboard position. Then the input could "sense" what keys you are hitting, based upon movement relative to your "home" position. I'm sure there could still be a visual keyboard in the traditional sense if desired, However, I at times type jibberish because I mis-place my hands on the keyboard. In this situation, the device would look for my fingers, rather than my fingers looking for the keys.

[melgross]

Originally Posted by Cubert

My guess is that it will be closer to the iPhone OS than OS X. I also think it will be able to run apps for both OS'es. You could probably install Photoshop on it but it may not be pretty trying to run it. Kinda like installing Photoshop on an old iBook - nothing is preventing you from doing it (assuming Universal binary, of course) but it wouldn't be pretty.

I think that a lot of people here aren't thinking this through.

This is a DIFFERENT PROCESSOR folks!

Everyone who could directly run their 68xxxx programs on their new PPC machines without using Apple's emulator (or third party ones), raise your hands.

Everyone who could run their PPC program on their 68xxxx machines, raise your hands.

Everyone who could run their PPC programs on their Intel Macs without Rosetta, raise your hands.

Everyone who could run their Intel programs on their PPC machines, raise your hands.

Hmm. No hands up.

No x86 chip in this, no OS X programs.

[probably]

Originally Posted by melgross

With the iPod, it was seen that the concept was becoming popular.

With the iPhone, it was known the concept was popular.

With the tablet, the concept has been seen to be a failure.

That's a very different area to break into. They have to turn a very unpopular concept into a popular one. Can they do that?

Remember the Cube. Great idea, beautiful device, but a failure for several reasons.

The cube is entirely unrelated to fabricating a new way to compute. It didn't offer anything new: just a really fun design and a high price tag.

Also I didn't see anyone mention that lesser-known detail regarding how the iPhone keyboard dynamically resizes the letters invisibly. That Fingerworks-derived detail of the interface is what every other mobile's software keyboard lacks -- is why the rest of the no-hardware-button devices don't feel right. It's why iPhone users can type at reasonable speeds now. See the "Dynamic Key Resizing" section of this writeup: http://ignorethecode.net/blog/2009/0...e-and-android/

This patent looks like a ridiculously complex evolution of all that statistical-assistance in correcting the fact that you're typing on a piece of glass.

[melgross]

Originally Posted by probably

The cube is entirely unrelated to fabricating a new way to compute. It didn't offer anything new: just a really fun design and a high price tag.

And that's why the lesson is important. Since people didn't see it as a radical way of computing, they could be comfortable knowing they wouldn't have to change their methods, or buy new programs.

This might be radical. Radical isn't good because it's radical. It must offer people something that they've wanted, or, now that they see it, something that they now want.

And it looks as though it will also be expensive. If it doesn't replace something else, then it's an additional expense. Do people want large additional expenses? Now?

And if they have to pay $30 a month for 3G service, in addition to their monthly phone bill, will they want to do that? And if they don't, does that kill much of the usefulness of the device?

[JupiterOne]

Originally Posted by Cubert

"Provide images of "flesh proximity" to a variety of sensors that can distinguish hand configuration."

Just what in the world does this mean?

Bow chicka bow bow.....

[dagaz]

Apparently when SJ was first shown the 'tablet' he said something along the lines of "looks cool, but what I would I actually use it for?" Even after all this time I have yet to hear anything that would make me want to get a tablet device. I love my MacBook, I love my iPhone, what would I use something in between for? Reading newspapers?

[Rob55]

Originally Posted by ~ufo~

Now this, to me at least, is really interesting.

Consider having an iMac like desktop.

with a touch tablet flat in front of it, kind of like the touch wacoms, but in the form of a portable tablet.

I, as an iPhone used, have found myself touching my iMac's screen quite a few times in vein to click on something. Now, as nice as it would be to have a 24" multitouch screen on my iMac, we can hopefully all agree that it is not the most ergonomic of input interfaces.

if you were to have your home desktop iMac type machine, be it touch screen or not, and get rid of the keyboard and mouse altogether replacing it by a 10" touch smart touch screen now that would be a whole new approach to computing.

After you are done working at home, you just take the tablet with you, which is an autonomous device on it's own, as the apple tablet is proposed to be.

I can truly see a lot of merit in this.
It is both the desktop and laptop redefined.

I would have no problem paying $600-800 for it either.

Any takers?

This is precisely what I envisioned reading the article. I would be all over some kind of multi-touch keyboard/mouse replacement and tablet type of device. The trick will be to make typing as smooth and efficient as using a conventional keyboard where you can actually feel the keys. I imagine it'll have to incorporate some kind of predictive software similar to (but much better than) what is used on the iPhone. I love the touch-pad on my MBP so much, I've been wishing someone would make something like it for the iMac/MacPro. Wacom just came out with their multi-touch device, but I'm really hoping for one made by Apple.

[jazzguru]

Keyboards are already evolving in this direction. It's the next natural step. Just take a look at the ASUS EEE Keyboard.

[Ireland]

If you are a good company Apple will absorb you.

[Ireland]

Originally Posted by JohnnyKrz

To me, this almost sounds more like a potential new input device for desktops which would replace the keyboard, mouse and pretty much any other type of input device.

I fail to see it being more efficient than a keyboard and mouse, though if it was I wouldn't complain.

[applebook]

Originally Posted by zindako

I feel sorry for the netbook market already, watch out Microsoft, you're about to find yourself hobbling behind trying to get your very own tablet device once apple drops this baby, and fail horribly at its inception and interface.

Premium (overpriced) netbooks do have to worry, but the likes of the Acer Aspire and other $300 netbooks needn't fret too much since Apple's tablet will cost substantially more.

[jazzguru]

Originally Posted by applebook

Premium (overpriced) netbooks do have to worry, but the likes of the Acer Aspire and other $300 netbooks needn't fret too much since Apple's tablet will cost substantially more.

I don't understand the whole "premium netbook" thing.

It seems netbooks are moving away from the small 8.9" screens to 10" - 12" screens.

I absolutely love my ASUS EEE PC 900HA with the 8.9" screen. I got it for $230 refurbished and I don't think I'd be as pleased with it if it were any larger than it is.

I think you're right, Apple's tablet will fill in the gap between the low-end netbooks and the full-fledged notebooks. Why buy a 10" netbook for $500-$600 when you can have an Apple tablet for approx. the same price?

As with the iPhone, I think it is going to be a game changer.

[CurtisEMayle]

Originally Posted by melgross

...

But a standard keyboard is a bit over 10.5" in length, excluding the numeric pad. A 10.6" diag screen will be too short, so the keyboard would be a bit cramped. They'll have to figure out a way around that.

Agreed. So what about an iTube? The 10" is the height of the "tube" screen when rolled up.