Apple details next-gen multi-touch techniques for tablet Macs

[AppleInsider]

In a revised company filing discovered by AppleInsider, iPhone maker Apple Inc. illustrates a number of techniques that would pave the way for tablet Macs that display a near full-sized multi-touch keyboard and run an undiluted version of the Mac OS X operating system.

Much of the 52-page filing describes methods for accurately detecting and deciphering a plurality of simultaneous contacts on a touch screen, which sets the foundation for future tablet-based products that users can manipulate using not only more than one finger, but more than one hand.

The illustrations in the April 15, 2008 document, which builds on filings dating back two years, depict a tablet Mac with a full windowing interface, by which interface windows can be enlarged, shrunk, moved, and cycled, rather than being locked or set in place as they are with the company's iPhone.

Several examples from the filing, which lists Apple industrial design chief Jonathan Ive among its inventors, are provided below. Some examples not listed but included in the filing include cycling through pages by vertical and horizontal finger swipes, and simultaneous zoom-and-rotate maneuvers.



Apple explains that on a tablet Mac, the buttons of a control box may be smaller than a users finger and located close together. Therefore it may be difficult for the user to make a selection directly without possibly pressing an undesirable button. To solve this problem, at least a portion of the control box would be enlarged, including the buttons therein when the user places their thumb over the control box. Once the control box has reached its enlarged state, the user can then accurately select one of the enlarged buttons. The control box would then reduce to its initial size after the button is selected or after a predetermined time period in which no selection was made or when the user moves their finger away from the control box.



In the above example, Apple explains that scrolling through data items in iTunes on a full-screen tablet would function similar to scrolling through a list of contacts on the iPhone. When the user slides their finger or fingers over the touch screen, vertical scrolling, which moves media items up or down through the window, is implemented. The direction of scrolling may follow the same direction as finger movement, or it may go in the reverse direction, the company said. In one particular embodiment, a single finger is used for selecting the media items from the list, and two fingers are used to scroll through the list.



Apple also describes a method for simulating a full-sized keyboard on a multi-touch screen, where the presence of a first object over a first key and a second object over a second key are detected at the same time.

For instance, the first key may be a shift key and the second key may be a symbol key (e.g., letters, numbers). In this manner, Apple said the keyboard acts like a traditional keyboard in that the user is allowed to select multiple keys at the same time in order to change the symbol, i.e., lower/upper case. The keys may also correspond to the control key, alt key, escape key, function key, and the like.

The specific illustration above shows how a user would position their fingers over the multipoint touch screen over the keyboard to enter data into a word processing program. For example, the user may place one of their fingers on the Q key in order to produce a lower case "q" in the word processing program. When the user decides that a letter should be in upper case, the user places one finger on the shift key and another finger on the desired letter (as indicated by the arrows). In order to continue typing in lower case, the user simply removes their finger from the shift key and places their finger over the next desired letter as they would on a traditional keyboard.

"Although only two keys are described, it should be noted that two keys is not a limitation and that more than two keys may be actuated simultaneously to produce one or more control signals," Apple said. "For example, control-alt-delete functionality may be implemented or larger groups of characters can be typed at the same time."



Another next-generation multi-touch technique is illustrated through the display of a virtual scroll wheel. The scroll wheel may be displayed automatically as part of a program or it may be displayed when a particular gesture is performed, according to Apple. "By way of example, during the operation of a music program [such as iTunes], the virtual scroll wheel may appear on the GUI of the music program when two fingers are placed on the touch screen rather than one finger which is typically used for tracking in the music program," the company said. "In some cases, the virtual scroll wheel only appears when two fingers are placed on a predetermined area of the GUI."

At some point, the fingers may be locked to the scroll wheel. This can occur at set down, for example. When the fingers are rotated in a clockwise direction, the scroll wheel is rotated in the clockwise direction in accordance with the rotating fingers. When the fingers are rotated in a counterclockwise direction, the virtual scroll wheel is rotated in the counter clockwise direction in accordance with the rotating fingers. Alternatively, rotation of the virtual scroll wheel may also be rotated with linear motion of the fingers in a tangential manner.

"It should be noted that although a surface scroll wheel is shown, the principals thereof can be applied to more conventional scroll wheels which are virtually based," Apple added. "For example, scroll wheels, whose axis is parallel to the display screen and which appear to protrude through the display screen. In this particular implementation, however, linear motion of the fingers are used to rotate the virtual scroll wheel."

[ View this article at AppleInsider.com ]

[teckstud]

Ladies and gentlemen- the Etch i Sketch.

[JCG]

For it to be really useful for real applications they need to add in some form of stylus to give the user more exacting control and ease data entry via handwriting recognition. Finger painting is not exact enough for sketching and writing, thus the invention of the pen, pencil and paint brush.

[merdhead]

Yawn. Call me when they release a product. Patent filings don't mean much.

[orange whip]

I do agree that a stylus would be great for drawing and the like, but I'm sure that Apple will focus on the simple message of fingers only.

Bit like the persistence with the one mouse button message... frustrating for some but an effective marketing ploy.



...oh yeah..... there will be no optical drive in this device.... it's dead

[solipsism]

Anyone want to take a gander at what type of device this will most likely be used for?

Originally Posted by JCG

For it to be really useful for real applications they need to add in some form of stylus to give the user more exacting control and ease data entry via handwriting recognition. Finger painting is not exact enough for sketching and writing, thus the invention of the pen, pencil and paint brush.

You can already buy styluses designed for capacitance touchscreens.

• http://gizmodo.com/391639/brightonne...ragging-action

[satchmo]

Originally Posted by JCG

For it to be really useful for real applications they need to add in some form of stylus to give the user more exacting control and ease data entry via handwriting recognition. Finger painting is not exact enough for sketching and writing, thus the invention of the pen, pencil and paint brush.

I know there are 'pens' that are available for the iPhone...for those with fat fingers I suppose.
Maybe it'll be a screen that can accept both pen and finger touch. That would be ideal.

[noirdesir]

Originally Posted by orange whip

...oh yeah..... there will be no optical drive in this device.... it's dead

Like the optical drive in my MBP

[solipsism]

Originally Posted by orange whip

...oh yeah..... there will be no optical drive in this device.... it's dead

edit: The otipcal drive is dead, not the device for not having one. (catching... on... very... slowly... today)

[teckstud]

Originally Posted by JCG

For it to be really useful for real applications they need to add in some form of stylus to give the user more exacting control and ease data entry via handwriting recognition. Finger painting is not exact enough for sketching and writing, thus the invention of the pen, pencil and paint brush.

??Do you need a stylus for the iPhone?

[zunx]

HERE IT IS:

MacBook Touch tipped to be Apple’s secret development
http://www.slashgear.com/macbook-tou...nt-2212825.php

MacBook Touch mockups from a fan
http://www.slashgear.com/macbook-tou...fan-059379.php

Put full blown video out port, true Ethernet port and true Firewire ports with a FULL complete and native Mac OS X inside, and we will purchase thousands for our University. THE ULTIMATE presentation remote using native Keynote and native PowerPoint files (full blown ones with animations, transitions, video, etc).

Like the OQO model e2 (5 to 7 inch screen at most), but with Mac OS X inside and much better:

http://www.oqo.com

BECAUSE EVEN THE MACBOOK AIR IS TOO LARGE AND TOO HEAVY FOR US!!! WE WANT THE FULL THING IN OUR POCKET. IN OUR HAND.

Remember that even Apple has confirmed that such a SECRET product will be released by September 2008:

Apple’s secret product is ‘MacBook touch’
http://macdailynews.com/index.php/we...ok_touch/P100/

[solipsism]

Originally Posted by teckstud

??Do you need a stylus for the iPhone?

Depends on what you need the phone for. For instance, if I wanted to use the iPhone/Touch as a handheld application that required a user's signature to agree to something, a stylus would make that much easier to complete.

[guest]

It's about time Apple jumps in on the touch-interface computer. I'd wish also for pen/handwriting recognition as well as voice command, but hey. Maybe this is the major product transition Apple was talking about - ALL new products with some sort of touchscreen interface. Be it a iPod touch-like pad or full screen.

Just realized that all this would require a revamped OS, i.e. Snow Leopard. Thus, in a year or so.

[teckstud]

[brianus]

What's up with those ghastly claws? Is that what happens to you after a few years using multi-touch interfaces? I'll stick to my keyboard, thanks..

[teckstud]

Originally Posted by solipsism

Depends on what you need the phone for. For instance, if I wanted to use the iPhone/Touch as a handheld application that required a user's signature to agree to something, a stylus would make that much easier to complete.

Who needs a pen for signatures for a computer no less a phone? That's a secondary need if ever there was one.

[solipsism]

Originally Posted by zunx

HERE IT IS:

MacBook Touch tipped to be Apple’s secret development
[...]
BECAUSE EVEN THE POWERBOOK AIR IS TOO LARGE AND TOO HEAVY FOR US!!!

The first two pics are the same. It is sexy, but I think it's impractical. While the touchscreen is great for being versatile there the lack for tactile feedback is an issue for excessive usage. I also don't see much of a consumer market for that type of device. it would have to be very expensive, yet underpowered compared to a notebook and still not fit in a pocket.

There is also an issue with OS X. This could not be the "native" OS X that the other Macs use, not without a great deal of added frameworks added to the upgrade DVD. This version would be a hybrid between OS X Mac and OS X iPhone. Finally, Apple's aesthetics have moved into tapering edges which gives the illusion of svelteness while adding structural support. This device is more Sony than Apple. Don't get me wrong, I like it, I just don't see the consumer aspect of it or Apple producing anything like it.

[solipsism]

Originally Posted by teckstud

Who needs a pen for signatures for a computer no less a handheld? That's a secondary need if ever there was one.

Then are you probably the only one on these forums that has never used an electronic stylus scribe your signature.

[night9hawk]

Originally Posted by teckstud

Ladies and gentlemen- the Etch i Sketch.

That was my first thought as well. Judging from how often I inadvertently touch the screen on my iPhone-with unintended consequences-I don't think I'd be interested in using the touch screen on a laptop/tablet to do any serious typing. (Others I know have the same problem and accept it as part of the iPhone experience.)

[Gee4orce]

Originally Posted by brianus

What's up with those ghastly claws? Is that what happens to you after a few years using multi-touch interfaces? I'll stick to my keyboard, thanks..

I was thinking the same thing.

[lepton]

First, there will be a tablet and soon! January announcement, I'd say. Second, there will be touch screen displays that can tilt down to almost horizontal on a desk. Why do you think Apple has had no new displays for... ages?

Third, Apple should use 1 finger for point and click, two fingers for scroll horizontally, vertically, or diagonally, three fingers for control. For example, three fingers on screen to bring up a virtual scroll wheel.

Did the article say anything about something we are all interested in - cut and paste gestures? We have seen some proposals a while back but is there anything found here?

My suggestion: Touch and hold brings up a loupe as usual to set the insertion point. Double touch (finger down, then quickly up and down and hold) to bring a loupe that can set the selection area, using the insertion point as the anchor for the other end of the selection. The selection highlights as usual. Touch and hold on the selection for a menu pop up that allows Cut/Copy/Delete. For paste, that is on the keyboard! You touch and hold the existing DEL key. As with many other keys, this will pop up some alternates including Cut, Copy, Paste as appropriate. There you go!

[teckstud]

Originally Posted by solipsism

Then are you probably the only one on these forums that has never used an electronic stylus scribe your signature.

NO I just don't use it every day. WTF?

[aegisdesign]

Originally Posted by teckstud

??Do you need a stylus for the iPhone?

There's currently a whole group of applications that need a stylus that the iPhone doesn't run that are even more prevalent on a larger screened tablet computer.

On my current phone I use a stylus to sketch directions and MMS/Email it to others. It's not possible currently to do that on an iPhone. It's also a kind of digital fag packet for ideas and I end up with pages and pages of scribbles on my phone's notes app.

[teckstud]

Originally Posted by solipsism

Then are you probably the only one on these forums that has never used an electronic stylus scribe your signature.

Maybe you use one every day for signatures when you check people out on your "elitist" handheld device at the Apple store.

[KindredMac]

My hands hurt from looking at the diagrams.....
That person has Nosferatu hands, the thumbs were fricking long!!!!!

[bloggerblog]

This device might be more practical for Apple's upcoming solutions for hotels and cruise ships. Government agencies were asking for such a device since the beginning of this century but Jobs insisted that the technology was not available yet, especially the screen resolution.

[teckstud]

Originally Posted by aegisdesign

There's currently a whole group of applications that need a stylus that the iPhone doesn't run that are even more prevalent on a larger screened tablet computer.

On my current phone I use a stylus to sketch directions and MMS/Email it to others. It's not possible currently to do that on an iPhone. It's also a kind of digital fag packet for ideas and I end up with pages and pages of scribbles on my phone's notes app.

That I understand but I don't believe it would be included with the device as sold by Apple since these applications aren't Apple's and would also conflict with its advertising.
SOunds cool - who makes it?

[Right_said_fred]

Originally Posted by aegisdesign

There's currently a whole group of applications that need a stylus that the iPhone doesn't run that are even more prevalent on a larger screened tablet computer.

On my current phone I use a stylus to sketch directions and MMS/Email it to others. It's not possible currently to do that on an iPhone. It's also a kind of digital fag packet for ideas and I end up with pages and pages of scribbles on my phone's notes app.

I would guess that no stylus would be provided. For common applications, fingers with zooming would take care of things. For detailed cad work for example - a virtual touchpad would take care of the precision, just like the real touch pad on a macbook. Of course sometimes when you have lots to do, a mouse helps, but is never required

[solipsism]

Originally Posted by teckstud

NO I just don't use it every day. WTF?

Other people's needs aren't necessarily congruent with your own. Using this forum as an example I'd say that is true considerably more often that not. If you try to think outside your ow limited scope of the world you might be able to consider a few commercial uses where a stylus for a capacitance touchscreen might be useful.

[RAM_Doubler]

Originally Posted by brianus

What's up with those ghastly claws? Is that what happens to you after a few years using multi-touch interfaces? I'll stick to my keyboard, thanks..

Nice gadget, I think I'd.....ZOMG what the hell is wrong with my hands!!!

[vinea]

One problem with touch screen keyboards is the inability to go to the rest position easily without false positives.

Lack of tactile feedback and that a lot of folks naturally "touch" the keyboard without actual key activation in physical keyboards either to maintain postion, rest fingers or whatever. I know when I pause typing my hands naturally go to the rest position with every finger touching a key and I can do so without looking.

[oodlum]

Originally Posted by teckstud

Who needs a pen for signatures for a computer no less a phone? That's a secondary need if ever there was one.

Try looking beyond your own needs. Designers, photographers, artists... a lot of people would be all over this thing as a graphics tablet, hence the need for an (optional) stylus. The ability to draw directly on the screen would be one of it's biggest attractions. Finger-painting doesn't cut it.

Now that I think of it, it would be amazing to be able to use this as a peripheral to the desktop; a wireless second monitor for input.

Added:

Having said that, I agree that Apple would play down the use of a stylus in favour of their beloved multi-touch. It wouldn't be something that popped out of the tablet, Palm-style.

[Magic_Al]

Based on the illustrations, the real breakthrough is Apple's technology for preventing wrist strain: having no wrists.

[astrodude18]

I'd probably only use it though if it was like a regular macbook(the screen opened up to reveal trackpad and keyboard), but then on the back of the screen(for if you closed it) would become the tablet writing portion, that way it gets the tablet, note-taking style of the touch-interface, without sacrificing the functionality and quickness of a real keyboard. Because, honestly, no matter how good the 'simulated' keyboard is, I just don't think that there is any substitute for the real thing

[bloggerblog]

Originally Posted by vinea

... I know when I pause typing my hands naturally go to the rest position with every finger touching a key and I can do so without looking.

Since there are no keys to feel, this keyboard may not be suited for 10 finger typing, two to four fingers may be more suitable.

[Virgil-TB2]

Originally Posted by lepton

... Did the article say anything about something we are all interested in - cut and paste gestures? We have seen some proposals a while back but is there anything found here?...

Did anyone ever think that the main reason we don't have cut and paste for the iphone is that it's a document *reader* at best, and we are waiting for the larger format device to be able to readily *edit* documents? If this is true, then isn't it likely that cut and paste will be introduced with the new tablet (iBook?) and isn't it likely that it will operate exactly the same way as it did previously on the Newton, i.e. - with a stylus?

Anyone who thinks about it for more than a second or two, should also realise that without a stylus, a tablet device (even with multi-touch) is not of much use. The market for tablets is exactly that part of the current market that still needs to use a pen. Think doctors, shippers, writers, artists, etc.

Add that information to other patents Apple has put out that indicate a method of recognising a persons hand and when they are "making the pen gesture" with their fingers and you have some big hints as to the form factor and utility of a new tablet device if it ever arrives.

Think of a bigger iPhone where you move sheets of "paper" with your fingers, but write on them with a stylus, or simply by pretending that you are holding a stylus. It's possible that this arrangement will also require a particular type of touch screen with a higher input detection resolution thus making the current iphone incapable of being used in the same way at least initially.

It might help if when AppleInsider puts up one of these articles about a patent, that they take the time to at least do what I did here and link it to other patents and trends and make some kind of prediction as to what it all means. The information about the other patents that I quoted was obtained from this very site! How hard would it be for one of the bloggers to do this?

[Porchland]

I'm glad to see ET still gainfully employed by Apple's R&D department.

[bdkennedy1]

What is up with those nasty alien hands?

[guest]

Originally Posted by bloggerblog

Since there are no keys to feel, this keyboard may not be suited for 10 finger typing, two to four fingers may be more suitable.

What is this clinging to keyboards???
Typing on a keyboard was always a pain and only a crutch! A stylus with a good handwriting recognition will have humanity forget about the terrible torturous instrument called keyboard in a second!

[amac4me]

This patent filing is interesting and proves that Apple is actively working (a quite far along) on porting the Cocoa Touch framework of iPhone OS to Mac OS X. We should see a tablet within a year (hopefully!). Maybe as early as MacWorld.

Looks exciting.