or Connect
AppleInsider › Forums › Software › Mac Software › Apple developing configurable multi-touch gesture dictionary
New Posts  All Forums:Forum Nav:

Apple developing configurable multi-touch gesture dictionary

post #1 of 70
Thread Starter 
In an apparent bid to extend the use of its multi-touch technology to devices other than iPhone, Apple in a new patent filing discusses the concept of a software-based multi-touch gesture dictionary that would let users to assign custom multi-touch finger movements to computer actions.

"Many expect that the use of multi-finger, touch-sensitive user interfaces will become widely adopted for interacting with computers and other electronic devices, allowing computer input to become even more straightforward and intuitive," Apple wrote in the January 3, 2007 filing.

"Users of these multi-touch interfaces may make use of hand and finger gestures to interact with their computers in ways that a conventional mouse and keyboard cannot easily achieve. A multi-touch gesture can be as simple as using one or two fingers to trace out a particular trajectory or pattern, or as intricate as using all the fingers of both hands in a complex sequence of movements reminiscent of American Sign Language."

The Cupertino-based electronics maker further explains that each motion of the hands and fingers, whether complex or not, would convey a specific meaning or action that is acted upon by the computer or electronic device at the behest of the user:

"The number of multi-touch gestures can be quite large because of the wide range of possible motions by fingers and hands. It is conceivable that an entirely new gesture language might evolve that would allow users to convey complex meaning and commands to computers and electronic devices by moving their hands and fingers in particular patterns."

To manage the new language, Apple's patent proposal calls for a "dictionary of multi-touch gestures" that is interactively presented to a user of a computer system having a multi-touch user interface. In one embodiment, the company said the dictionary may take the form of a dedicated computer application that identifies a chord (e.g., a combination of fingers, thumbs, and/or other hand parts) presented to the multi-touch interface by the user and displays a dictionary entry for the identified chord.

A dictionary entry may include, for example, visual depictions of one or more motions that may be associated with the chord and meanings of the gestures including the identified chords and the various motions.

"The visual depictions may take the form of motion icons having a graphical depiction of the motion and a textual description of the meaning of the gesture," Apple wrote. "The visual depictions may also take the form of animations of the one or more motions. The application could also identify one or more motions of the chord by the user and provide visual and/or audible feedback to the user indicating the gesture formed and its meaning."



In another embodiment, Apple said a dictionary application can run in the background while other applications on the computer systems are used. In this scenario, if a user presents a chord associated with a gesture without a motion completing the gesture, the dictionary application can present a dictionary entry for the presented chords.

Apple explains: "As in other embodiments, the dictionary entry may include visual depictions of one or more motions and meanings of the gestures comprising the identified chord and the various motions. Also as in other embodiments, the visual depictions may take the form of motion icons or animations of the motions. A user guided by the dictionary entry may perform a motion completing a gesture, and the system may execute a meaning of the gesture and may also provide visual and/or audible feedback indicating the meaning of the gesture. "

In yet another approach to its dictionary concept, Apple discusses the concept of an interactive computer application that allows a user to assign meanings to multi-touch gestures is provided. The computer application may display a dictionary entry (like those described above, for example) and accept inputs from the user to assign a meaning to one or more of the gestures in the dictionary entry.



Apple further explained that the application could be used to assign meanings to gestures that do not have default meanings selected by a system designer or may be used to change the meanings of gestures that do have default meanings assigned by a system designer. The application, the company added, may also include program logic to selectively present only those motions that may be more easily performed in a form different from those motions that may be more difficult to perform. Alternatively, the more difficult motions may not be displayed at all.

Finally, Apple also explains that the gesture dictionary applications could be triggered by events other than presentation of a chord:

"These events may include hand parts hovering over a multi-touch surface, audible events (for example, voice commands), activation of one or more buttons on a device, or applying a force and/or touch to a force and/or touch sensitive portion of a device. These events may correspond to chords and invoke a dictionary entry corresponding to such a chord. Alternatively or additionally, these events may correspond to other groupings of gestures not based on chords, such as custom dictionary entries. In yet another variation, the event triggering a gesture dictionary application may not correspond to a gesture grouping at all. In these cases, a dictionary index may be invoked, allowing a user to select from a plurality of dictionary entries."

According to the filing -- credited to Apple employees John Elias, Wayne Westerman, and Myra Haggerty -- the multi-touch gesture dictionary concept could be deployed on a notebook computer, tablet computer, handheld computer, personal digital assistant, media player, mobile telephone, or other consumer electronics devices of similar nature.
post #2 of 70
There were some doubters out there, but it is obvious now: multitouch is here to stay and grow.

Self-programmable keyboards are on the way.

I would love to say... I told you so!

 

Your = the possessive of you, as in, "Your name is Tom, right?" or "What is your name?"

 

You're = a contraction of YOU + ARE as in, "You are right" --> "You're right."

 

 

Reply

 

Your = the possessive of you, as in, "Your name is Tom, right?" or "What is your name?"

 

You're = a contraction of YOU + ARE as in, "You are right" --> "You're right."

 

 

Reply
post #3 of 70
i thought they meant some electronic dictionary for English deficient people.
post #4 of 70
Quote:
Originally Posted by Bergermeister View Post

There were some doubters out there, but it is obvious now: multitouch is here to stay and grow.

Self-programmable keyboards are on the way.

I would love to say... I told you so!

I would love to say--"You told us so" (as did ohers like kolchak).

Bring on the multi-touch ultralights, "regular" and tablets.

Fujitsu, Sony, Toshiba, Panasonic, Asus, HP...., beware--the Mac MTULs with OS X are coming. As Japan resident and poster ElectricMonk keeps saying, if AAPL makes these 1-1.5 kg units, Japan goes from wasteland to boomtown. And, for those who don't venture outside the Apple mothership, ULs are not a niche market--just take a stroll around Wall Street or go to a large science or business conference-- ULs on parade.
post #5 of 70
We have had a few good discussions on the topic. One is here:

http://forums.appleinsider.com/showt...ght=multitouch

 

Your = the possessive of you, as in, "Your name is Tom, right?" or "What is your name?"

 

You're = a contraction of YOU + ARE as in, "You are right" --> "You're right."

 

 

Reply

 

Your = the possessive of you, as in, "Your name is Tom, right?" or "What is your name?"

 

You're = a contraction of YOU + ARE as in, "You are right" --> "You're right."

 

 

Reply
post #6 of 70
Quote:
Originally Posted by AppleInsider View Post

I
According to the filing -- credited to Apple employees John Elias, Wayne Westerman, and Myra Haggerty -- the multi-touch gesture dictionary concept could be deployed on a notebook computer, tablet computer, handheld computer, personal digital assistant, media player, mobile telephone, or other consumer electronics devices of similar nature.

Multi-touch Ultraportable or iMac anyone?
MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
post #7 of 70
Cool. Visual "macros".
post #8 of 70
Quote:
Originally Posted by AppleInsider View Post

the multi-touch gesture dictionary concept could be deployed on a notebook computer, tablet computer, handheld computer, personal digital assistant, media player, mobile telephone, or other consumer electronics devices of similar nature.

I know they're just covering their arses with this patent filing, but I'm betting on multi-touch on iPods this fall, and on Macs next year (or 2009 at the latest)...

Or are we looking at a revival of the Newton??

Apple would blow everyone away if next Tuesday's iMac update implemented multi-touch - but I think it's still too early for that.
post #9 of 70
Here comes the iTablet G5 with moulditouch on Tuesday 8/7/2007.
Bring it on. LOL
post #10 of 70
Quote:
Originally Posted by MsNly View Post

Multi-touch Ultraportable or iMac anyone?

I'll take two of each!
Journalism is publishing what someone doesn't want us to know; the rest is propaganda.
-Horacio Verbitsky (el perro), journalist (b. 1942)
Reply
Journalism is publishing what someone doesn't want us to know; the rest is propaganda.
-Horacio Verbitsky (el perro), journalist (b. 1942)
Reply
post #11 of 70
If they add method to select a piece of text (or maybe this exists already on the iPhone), then they have a way now to implement copy/paste on it.
post #12 of 70
Quote:
Originally Posted by Bergermeister View Post

There were some doubters out there, but it is obvious now: multitouch is here to stay and grow.

Self-programmable keyboards are on the way.

I would love to say... I told you so!

So would I.
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #13 of 70
Quote:
Originally Posted by Lafe View Post

I'll take two of each!

I'd prefer a minitower but I guess I could settle for a multi-touch iMac (possibly upgradeable? )

So much wishing so little time
MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
post #14 of 70
Hate to rain on a the parade here but what about Mentor Graphics use of Strokes? Seems to me they (Apple) might be stepping on patents here.

Many years ago, Mentor Graphics started using "strokes" (or mouse gestures) to define functions instead of mouse clicks in their EDA schematic capture and layout tools. It's very effective and saves a ton of time. However, instead of a finger, you use your mouse.
post #15 of 70
Quote:
Originally Posted by noirdesir View Post

If they add method to select a piece of text (or maybe this exists already on the iPhone), then they have a way now to implement copy/paste on it.

I take it you didn't look at the images?

Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #16 of 70
Adding copy, cut and paste to the iPhone is not that difficult for Apple. They already have the magnifying-glass for positioning, you position at the starting point, add a second finger, move the magnifying-glass to the end point, lift your fingers and a copy/cut/paste/delete button could appear - perfect.
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #17 of 70
Quote:
Originally Posted by Ireland View Post

I take it you didn't look at the images?

I did take a look, in fact these images were my inspiration, particularly the cut, copy, and paste gestures. I simply could not find a gesture for the selecting action, which has to precede any cut or copy action.
post #18 of 70
Quote:
Originally Posted by noirdesir View Post

If they add method to select a piece of text (or maybe this exists already on the iPhone), then they have a way now to implement copy/paste on it.



Quote:
Originally Posted by Ireland View Post

I take it you didn't look at the images?

MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
post #19 of 70
Quote:
Originally Posted by kkerst View Post

Many years ago, Mentor Graphics started using "strokes" (or mouse gestures) to define functions instead of mouse clicks in their EDA schematic capture and layout tools. It's very effective and saves a ton of time. However, instead of a finger, you use your mouse.

That would qualify as different, however, there could be overlapping patent issues.

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #20 of 70
Quote:
Originally Posted by SpamSandwich View Post

That would qualify as different, however, there could be overlapping patent issues.

Apple seems to be going much further though. Mentor's concept was just for specific usage. Apple is using this as a general user input device interface. Not what Mentor envisioned. Also using an older patented idea for a newer purpose can be justified. It depends on the new restrictions about that use just handed down from the Supreme Court.
post #21 of 70
Let's all remember that the iMac is kind of Apple's "flagship" Mac. It was the first to get the Intel processors and has been first for many other things. I'm thinking it's VERY possible that we're going to get an iMac Tuesday with some kind of multitouch technology built in. What it would be used for, I don't know, but it could be cool.
Living life in glorious 4G HD (with a 2GB data cap).
Reply
Living life in glorious 4G HD (with a 2GB data cap).
Reply
post #22 of 70
Quote:
Originally Posted by EagerDragon View Post

Here comes the iTablet G5 with moulditouch on Tuesday 8/7/2007.
Bring it on. LOL

G5??? What is THAT!??

--

I dont think multitouch would be fit to enter an iMac as it is... (of course, we don't know its "new" shape just yet)

im betting on a multitouch peripheral. i.e: multitouch keyboard?
it just seems more logical to keep your hands on a keyboardy position, not so much "up in the monitor".

But we'll find out sooner or later. Maybe they'll prove me wrong.
post #23 of 70
This would be truly useful only if the gesture library could be user-configurable; I have several gestures I have practiced my entire life that I would like to use to, for example, delete spam or discard a pop-up window.
post #24 of 70
Quote:
Originally Posted by noirdesir

If they add method to select a piece of text (or maybe this exists already on the iPhone), then they have a way now to implement copy/paste on it.

Quote:
Originally Posted by Ireland View Post

I take it you didn't look at the images?

I read that differently. I thought that was just a sarcastic remark about the need to copy and paste on iPhone.

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #25 of 70
Quote:
Originally Posted by CosmoNut View Post

Let's all remember that the iMac is kind of Apple's "flagship" Mac. It was the first to get the Intel processors and has been first for many other things. I'm thinking it's VERY possible that we're going to get an iMac Tuesday with some kind of multitouch technology built in. What it would be used for, I don't know, but it could be cool.

Nobody is getting their greasy mitts on my screen that's for sure. Interestingly the patent doesn't mention monitors and desktop computers, just portable devices, and that's where it belongs.
post #26 of 70
I think this smells like a Tuesday event. Have you seen the video of the guy who invented multitouch? Impressive. This will be good for artists too.
post #27 of 70
Quote:
Originally Posted by aegisdesign View Post

Nobody is getting their greasy mitts on my screen that's for sure. Interestingly the patent doesn't mention monitors and desktop computers, just portable devices, and that's where it belongs.

I don't know, I think a software configurable keyboard could be useful. If it were multitouch, the letters on the keyboard could be reconfigurable through software so that if you were typing in Spanish for example, you would have the characters that you need such as accents, tildes and upside down question marks, etc. They would just automatically appear in a context sensitive way.

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #28 of 70
Just to correct myself, the patent does mention desktop computers explicitly even though AppleInsider omitted that.
post #29 of 70
Quote:
Originally Posted by drjjones View Post

I think this smells like a Tuesday event. Have you seen the video of the guy who invented multitouch? Impressive. This will be good for artists too.

Jeff Han didn't invent it either.
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #30 of 70
Quote:
Originally Posted by mstone View Post

I don't know, I think a software configurable keyboard could be useful. If it were multitouch, the letters on the keyboard could be reconfigurable through software so that if you were typing in Spanish for example, you would have the characters that you need such as accents, tildes and upside down question marks, etc. They would just automatically appear in a context sensitive way.

Nothing to do with this patent though which is about a multi-touch gesture based interface.
post #31 of 70
Quote:
Originally Posted by aegisdesign View Post

Just to correct myself, the patent does mention desktop computers explicitly even though AppleInsider omitted that.

Quote:
Originally Posted by AppleInsider View Post

In an apparent bid to extend the use of its multi-touch technology to devices other than iPhone, Apple in a new patent filing discusses the concept of a software-based multi-touch gesture dictionary that would let users to assign custom multi-touch finger movements to computer actions.

"Many expect that the use of multi-finger, touch-sensitive user interfaces will become widely adopted for interacting with computers and other electronic devices, allowing computer input to become even more straightforward and intuitive," Apple wrote in the January 3, 2007 filing.

"Users of these multi-touch interfaces may make use of hand and finger gestures to interact with their computers in ways that a conventional mouse and keyboard cannot easily achieve. A multi-touch gesture can be as simple as using one or two fingers to trace out a particular trajectory or pattern, or as intricate as using all the fingers of both hands in a complex sequence of movements reminiscent of American Sign Language."

The Cupertino-based electronics maker further explains that each motion of the hands and fingers, whether complex or not, would convey a specific meaning or action that is acted upon by the computer or electronic device at the behest of the user:

"The number of multi-touch gestures can be quite large because of the wide range of possible motions by fingers and hands. It is conceivable that an entirely new gesture language might evolve that would allow users to convey complex meaning and commands to computers and electronic devices by moving their hands and fingers in particular patterns."

Sure they never said the word "desktop" but does that really matter?
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #32 of 70
Quote:
Originally Posted by aegisdesign View Post

Nobody is getting their greasy mitts on my screen that's for sure. Interestingly the patent doesn't mention monitors and desktop computers, just portable devices, and that's where it belongs.

That's just inaccurate. The patent goes out its way to mention computers. You like many others here will be proved wrong over time. There's nothing to love about mice or keyboard. I would welcome multi-touch into my life and don't care for 'tactile' feedback. I think what you are missing is that while they mention computer they don't mention monitors because that's not the part that's getting the touch-screen, it's the keyboard that's getting the touch-screen. On the screens of tablets however, that's a different matter though. The tablet would be just like the iPhone with a pop-up keyboard when needed. Say goodbye to laptops I say, eventually.
Quote:
"Users of these multi-touch interfaces may make use of hand and finger gestures to interact with their computers in ways that a conventional mouse and keyboard cannot easily achieve."
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #33 of 70
Quote:
Originally Posted by aegisdesign View Post

Nothing to do with this patent though which is about a multi-touch gesture based interface.

Very well then you could use a gesture to switch it to another configuration. The point is that anything is possible once you remove the physical keys and go multitouch.

m

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #34 of 70
I remember all the articles about how mice would never catch on, and what a terrible idea it was.

"You have to take a hand off the keyboard", they said.

"You then have to look for a menu, find the command you need, and then press the button", they said.

"You then have to bring your hand back to the keyboard", they said.

"It's much easier to remember a two or three key command for these things so you don't waste time and effort moving your hand, and eyes", they said.

They were right. The mouse never caught on.

We're getting the same thing now with those opposed to multiTouch.

Greasy hands on the keyboard or mouse, same thing as on the monitor, except that the monitor is vastly easier to clean. And, not everyone has dirty hands most of the time.

Much easier making gestures on a screen than using a mouse or keyboard.

People will get used to it, and wonder how we did without, just like the mouse (and trackball, which I prefer).
post #35 of 70
There are things I like, and many things I don't like about multi-touch. One thing I don't like is the idea that my fingertips would be the components that would be wearing down, vs. the bottom of the mouse, or the nib of the wireless pen. Anyone consider this? Imagine the calluses from working on the multiMac(TM) 12 hours a day...

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #36 of 70
Quote:
Originally Posted by melgross View Post

Apple seems to be going much further though. Mentor's concept was just for specific usage. Apple is using this as a general user input device interface. Not what Mentor envisioned. Also using an older patented idea for a newer purpose can be justified. It depends on the new restrictions about that use just handed down from the Supreme Court.

And hopefully these patents get thrown out under the obviousness test. There isn't a gesture interface without a dictionary of gestures that translate gestures into computer action.

If Apple locked down gestures it would have a massive chilling effect as if they had managed to patent the GUI...or Xerox did. I'm all for IP but these patents are pretty obvious for the practioner.

Gee...I have a set of gestures...how do I train the user? How do I let the user associate a gesture to an action? I guess I need a diagram and show them how it works and let them build new gestures and gee...I should store these things somewhere...how about a database (dictionary) of gestures?

I doubt Apple will push these patents into the Peer Review program but rather hope to get a clueless examiner.

Vinea
post #37 of 70
Quote:
Originally Posted by SpamSandwich View Post

There are things I like, and many things I don't like about multi-touch. One thing I don't like is the idea that my fingertips would be the components that would be wearing down, vs. the bottom of the mouse, or the nib of the wireless pen. Anyone consider this? Imagine the calluses from working on the multiMac 12 hours a day...

post #38 of 70
Apple needs to drop multi-touch and invent Thought-Touch, you think and it moves!
MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
MacBook Pro 17" Glossy 2.93GHz, iPad 64GB, iPhone 4 16GB, and a lot of other assorted goodies.

If you're a troll and you have been slain. Don't be a Zombie.
Reply
post #39 of 70
Quote:
Originally Posted by melgross View Post


You heard it here first, folks... multiMac.

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #40 of 70
Quote:
Originally Posted by vinea View Post

And hopefully these patents get thrown out under the obviousness test. There isn't a gesture interface without a dictionary of gestures that translate gestures into computer action.

If Apple locked down gestures it would have a massive chilling effect as if they had managed to patent the GUI...or Xerox did. I'm all for IP but these patents are pretty obvious for the practioner.

Gee...I have a set of gestures...how do I train the user? How do I let the user associate a gesture to an action? I guess I need a diagram and show them how it works and let them build new gestures and gee...I should store these things somewhere...how about a database (dictionary) of gestures?

I doubt Apple will push these patents into the Peer Review program but rather hope to get a clueless examiner.

Vinea

We'll have to see. We're not going to get patents like this passed anymore if they aren't much stronger than they had to be before the new rules were handed down.

If they deserve the patent(s), then they should get them. If they don't, then they shouldn't. Neither you or I can claim that we know enough about the field to have the last word on it.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Mac Software
AppleInsider › Forums › Software › Mac Software › Apple developing configurable multi-touch gesture dictionary