or Connect
AppleInsider › Forums › Software › Mac OS X › New Apple filing details virtual input touch-screen interface
New Posts  All Forums:Forum Nav:

New Apple filing details virtual input touch-screen interface

post #1 of 32
Thread Starter 
Apple Computer in its research and development labs is experimenting with a variety of touch-screen technologies, recent patent filings have revealed.

The latest filing surfaced late last week, describing a virtual input device placement on a touch-screen user interface. Based on the visual diagrams associated with the filing, Apple appears to be working on several methods of displaying a virtual keyboard as part of the Mac OS X interface, which would allow users to type and input data without the use of a physical keyboard.

Virtual keyboard

In the filing, made Sept. 16, 2005, Apple said the technology includes "an application display, associated with an application executing on the computer, and a virtual input device display for a user to provide input to the application executing on the computer via the touch screen."

"In response to a virtual input device initiation event, initial characteristics of the virtual input device display are determined," the filing goes on to read. "Based on characteristics of the application display and the characteristics of the virtual input device display, initial characteristics of a composite display image are determined including the application display and the virtual input device display. The composite image is caused to be displayed on the touch screen."

Illustrations included with the patent filing reveal several different approaches Apple could use to integrate the virtual keyboard into the Mac OS. The first shows the keyboard merging upward from the base of the display to accommodate the bottom half of the touch-screen display, overlaying the Mac OS and application interfaces, which remain unchanged.

Similarly, a second and third approach describes example touch screen displays where the spatial aspect of the application display is modified in accommodation of the virtual keyboard. In one example, the entire Mac OS interface is shifted upwards, cropping the top half of the the Mac OS interface to allow room for the keyboard. Another example skews the Mac OS interface to fit entirely above the virtual keyboard without cropping.

Yet another approach portrayed in the illustrations is a partial pop-up virtual keyboard, which floats above the Mac OS interface in balloon-help window.

The patent filing is credited to three Apple software engineers, Imran Chaudhri, Greg Christie and Bas Ording.

Virtual click-wheel

Earlier this month, another touch-screen Apple patent filing surfaced, which describes "Gestures for touch sensitive input devices." Unlike the most recent filing, visual diagrams associated with this patent request feature illustrations of a hand using a touch-screen click-wheel controller on a PDA-sized device.

The patent states that "the invention relates, in one embodiment, to a computer implemented method for processing touch inputs... the method includes reading data from a touch sensitive device having a multipoint capability. The method also includes identifying at least one multipoint gesture based on the data from the touch sensitive device."

Some design aspects covered in the patent include a touch-screen device's ability to rotate, zoom, pan, page turn, and make scroll wheel, inertia, and floating control sequence actions, among others.

A detailed description of the invention stated that "the invention generally pertains to gestures and methods of implementing gestures with touch sensitive devices." Examples of touch sensitive devices include touch screens and touch pads, Apple said in the filing.
post #2 of 32
A virtual keyboard.. hmmm. How 'bout for use on a laptop that can swivel into a tablet? Though I don't know about combining the two.. and I'm not interested in one.. I have to say.. this is quite interesting. Time will tell.
post #3 of 32
Quote:
Originally posted by AppleInsider

Virtual click-wheel

Earlier this month, another touch-screen Apple patent filing surfaced, which describes "Gestures for touch sensitive input devices." Unlike the most recent filing, visual diagrams associated with this patent request feature illustrations of a hand using a touch-screen click-wheel controller on a PDA-sized device.


Now we know how the Video iPod will work. One big screen occupying the whole face of the unit. No external buttons or click-wheel, bring on the virtual click-wheel.
post #4 of 32
Well, this no longer sounds like an interface for a video iPod.
post #5 of 32
Interesting. It looks like Apple is very serious about making either a new portable user-interface device or a tablet mac.
post #6 of 32
You would think that by now, with all the years of development, and products that use touch screens, there wouldn't be anything worthwhile left to patent in this area.

I still don't see Apple competing for the 1% of users in the PC market that use tablets. This an area that hasn't shown any growth.
post #7 of 32
Quote:
Originally posted by AppleInsider
Illustrations included with the patent filing reveal several different approaches Apple could use to integrate the virtual keyboard into the Mac OS. The first shows the keyboard merging upward from the base of the display to accommodate the bottom half of the touch-screen display, overlaying the Mac OS and application interfaces, which remain unchanged.

Similarly, a second and third approach describes example touch screen displays where the spatial aspect of the application display is modified in accommodation of the virtual keyboard. In one example, the entire Mac OS interface is shifted upwards, cropping the top half of the the Mac OS interface to allow room for the keyboard. Another example skews the Mac OS interface to fit entirely above the virtual keyboard without cropping.

How can Apple patent a keyboard at the bottom of the screen shifting the rest of the interface upward? Isn't this the sort of thing that Palm has done for years?
Proud member of AppleInsider since before the World Wide Web existed.
Reply
Proud member of AppleInsider since before the World Wide Web existed.
Reply
post #8 of 32
Quote:
Originally posted by melgross
I still don't see Apple competing for the 1% of users in the PC market that use tablets. This an area that hasn't shown any growth.

Maybe they're hoping to create a market rather than compete within one...
post #9 of 32
Quote:
Originally posted by CharlesS
How can Apple patent a keyboard at the bottom of the screen shifting the rest of the interface upward? Isn't this the sort of thing that Palm has done for years?

Well the Palm screen keyboard is for hunting/pecking with a pen... pointer... stylus! Stylus. Jeez I need more sleep.

Anyway, a "real" keyboard would --or should-- allow multi-finger input at good typing speeds.

And this is the perfect thread to show people this again:
http://www.youtube.com/watch?v=zp-y3ZNaCqs
MWSF '07: Steve Jobs hates my wallet and my mobile carrier.
Reply
MWSF '07: Steve Jobs hates my wallet and my mobile carrier.
Reply
post #10 of 32
Quote:
Originally posted by psychboy
Maybe they're hoping to create a market rather than compete within one...

There's been a lot of work done on tablets, going back as far as GRID. All of these developments were to establish a tablet market. All of them failed.

The tablet OS from MS is now in its 3rd generation, supposedly, much improved over its earlier incarnations. Those who love them won't give them up, but the vast majority of people who try them, can't stand them. Several manufacturers have stopped making them.

Unless this is going to be some kind of grown up version of the Newton, but with some version of X on it, I just can't see it. Tablets are 13" to 15.4" devices weighing at least 3 pounds (at least all of the ones I've read about).

If Apple comes out with some device with a 10" screen weighing a pound, or so, having a long battery life, I might see it having a market. I would buy something like thatif it didn't cost too much.

I'm not sure that it would be a tablet in the way we think of one though. It would be a separate category.
post #11 of 32
Quote:
Originally posted by sCreeD
Well the Palm screen keyboard is for hunting/pecking with a pen... pointer... stylus! Stylus. Jeez I need more sleep.

Anyway, a "real" keyboard would --or should-- allow multi-finger input at good typing speeds.

And this is the perfect thread to show people this again:
http://www.youtube.com/watch?v=zp-y3ZNaCqs

The Palms and the Windows PDA and phone screens are pretty small. There isn't much choice there.

But, I find Graffiti to work very well. I only go to the keyboards when I forget some symbol I rarely use.

Larger screens shouldn't have that problem.
post #12 of 32
Quote:
Originally posted by melgross
You would think that by now, with all the years of development, and products that use touch screens, there wouldn't be anything worthwhile left to patent in this area.

I still don't see Apple competing for the 1% of users in the PC market that use tablets. This an area that hasn't shown any growth.

Thats because no one has done it right yet.
I feel Apple could change this. If this announcement is the vPod with touch screen and it turns out to be pretty dam good user interface then users would embrace a tablet with touch screen interface. IMHO

G5 2GHZ Power Mac, iPod Shuffle (1st Gen),iPod Nano (2nd Gen),iPod (5th Gen), Apple TV, Apple TV 2G x2, iPad 2,iPhone 4S, rMBP 15" 2.6

Reply

G5 2GHZ Power Mac, iPod Shuffle (1st Gen),iPod Nano (2nd Gen),iPod (5th Gen), Apple TV, Apple TV 2G x2, iPad 2,iPhone 4S, rMBP 15" 2.6

Reply
post #13 of 32
Quote:
Originally posted by jimbo123
Thats because no one has done it right yet.
I feel Apple could change this. If this announcement is the vPod with touch screen and it turns out to be pretty dam good user interface then users would embrace a tablet with touch screen interface. IMHO

That remains to be seen.
It's the size and weight of the tablets that are holding them back. They have full size keyboards as well.

If Apple came out with the small light weight model I mentioned, it might be different. Perhaps with a 1.8" HD, instead of the 2.5" models. FLASH is still too expensive to replace HD's in computers.

No keyboard?

$1,000 (less, please) bucks with a low voltage Core Single (I don't remember what they call that) chip.
post #14 of 32
I would love to see a tablet mac.... Forget about a swivel monitor. Make it look much like a small iMac (or better yet, a solid PowerBook/MacBook Pro), with either no real keyboard or a (bluetooth) keyboard that slides out from the back and stores within the unit. I know a lot of people will disagree, but I think a tablet with no hinges would be the ulitmate portable computer. It would just need either a hard, raised touch surface to protect the monitor or (against the no hinge idea) have just a thin alloy notebook style cover that flips around and snaps to the back side when in use. Better yet, and this is really dreaming, some sort of retractable screen cover that opens and closes at the touch of a button (but watch your fingers). While they're at it.. make a dock interface for it in my office, car and house with software capable of controlling it all! I'll settle for a simple tablet, for now.
post #15 of 32
or,

....dare I say it.

Rolo's TV with a new kind of interface and remote.


just a thought.
post #16 of 32
Once you have wireless AirTunes (video or no) it becomes hard to control your music when not at the computer. A remote touch-sensitive pad like this would allow you to control your computer's iTunes library without physically being near it.

Sure, I want WiFi in my iPod for wireless playback (to AirTunes) and wireless syncing, but a touchscreen remote device like this would be for all iTunes users, those with iPods or those without.

If a full media center system is connected to your TV you can use a Front Row interface. But if you want something like an iPod to carry from room to room to control your music and maybe do a few basic web apps (email, surfin', widgets) this tablet may be for you!
Download BARTsmart BART Widget, the best BART schedule widget for Mac OS X's Dashboard.
Reply
Download BARTsmart BART Widget, the best BART schedule widget for Mac OS X's Dashboard.
Reply
post #17 of 32
Virtual (on-screen) keyboard has already been done (see "On-screen Keyboard" in the links below):

http://www2.smarttech.com/st/en-US/P...oard+Tools.htm
http://www2.smarttech.com/st/en-US/P...oard+Tools.htm

So has multitouch for that matter:

http://www.smarttech.com/dvit/index.asp

It's intended for a different market than Apple would appear to be targetting, but I can't see how their patents will hold up in a technology segment that's had people working on it (and filing patents) for the past 20+ years.

I'm still interested to see what they do with the technology though, independent of all the legal wrangling...
 
Reply
 
Reply
post #18 of 32
Quote:
Originally posted by melgross
You would think that by now, with all the years of development, and products that use touch screens, there wouldn't be anything worthwhile left to patent in this area.

There aren't any serious commercial products out there that multi-point touch input. I don't think I've seen anything outside of a research lab or very limited production out of a research group that actually works.
Quote:
I still don't see Apple competing for the 1% of users in the PC market that use tablets. This an area that hasn't shown any growth.

This would be a very different type of tablet. We are supposed to be buying some tablets soon, and I really hope Apple get this out before our team spends money on Windows Tablet PC technology.
post #19 of 32
Quote:
Originally posted by auxio
Virtual (on-screen) keyboard has already been done (see "On-screen Keyboard" in the links below):

http://www2.smarttech.com/st/en-US/P...oard+Tools.htm
http://www2.smarttech.com/st/en-US/P...oard+Tools.htm

So has multitouch for that matter:

http://www.smarttech.com/dvit/index.asp

It's intended for a different market than Apple would appear to be targetting, but I can't see how their patents will hold up in a technology segment that's had people working on it (and filing patents) for the past 20+ years.

I'm still interested to see what they do with the technology though, independent of all the legal wrangling...

We have a SMART Board with DViT. Despite their claims, I have never been able to do any kind of multi-point input on it. If you know of any way to get it to support that, let me know. I'm sure the hardware can handle it, but I've never seen a version of their software that could. Maybe their latest Windows software version does (haven't checked lately, ours is on a Mac ), but it'd be news to me. I've even written them a couple of times about getting developers info for multi-point input and gotten no response.

It is also very easy to block the DViT cameras as part of normal interactions such that the (single point) input gets rather screwed up. I'd have to imagine it'd be even worse trying to do multi-point. Since they're using cameras, I suspect a lot of this is software, and their driver software just isn't up to snuff yet (a lot of computer vision type things). Then again, it may just be an OS issue: what do you do with two or more input points if you have them anyhow, mouse emulation goes out the window?

BTW, I seriously doubt SMART Technologies was the first company out there to do a virtual keyboard.
post #20 of 32
Quote:
Originally posted by melgross
You would think that by now, with all the years of development, and products that use touch screens, there wouldn't be anything worthwhile left to patent in this area.

I still don't see Apple competing for the 1% of users in the PC market that use tablets. This an area that hasn't shown any growth.

I don't either... that's why I'm guessing it's some kind of media device or an interface for the iPhone.

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #21 of 32
Quote:
Originally posted by AppleInsider
[B]Apple Computer in its research and development labs is experimenting with a variety of touch-screen technologies, recent patent filings have revealed.

The latest filing surfaced late last week, describing a virtual input device placement on a touch-screen user interface. Based on the visual diagrams associated with the filing, Apple appears to be working on several methods of displaying a virtual keyboard as part of the Mac OS X interface, which would allow users to type and input data without the use of a physical keyboard.


...and just imagine the scratches on that baby!

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #22 of 32
Quote:
We have a SMART Board with DViT. Despite their claims, I have never been able to do any kind of multi-point input on it.

That's because no operating system natively supports multiple contact points. So it's limited to performing "multi-contact" gestures. Read the DViT description closely and you'll see it (right mouse click is one contact point beside the other) -- the hardware does support it. The only way to truly get multiple contacts with it (eg. two people drawing at the same time) is with a custom written application. Hopefully that will change if Apple gets into the game and puts support at the OS level for this.

Quote:
I've even written them a couple of times about getting developers info for multi-point input and gotten no response.

Try here:
http://www.smarttech.com/resourcecenter/sdk/index.asp

Quote:
Then again, it may just be an OS issue: what do you do with two or more input points if you have them anyhow, mouse emulation goes out the window?

Basically, yes. Think about rearchitecting the OS to handle multiple window focus and you get some idea of the technical challenges. However, if it were limited to tablet style input and not worrying about multiple mouse pointers, then things get a bit simpler (how I imagine that Apple would do it).

Quote:
BTW, I seriously doubt SMART Technologies was the first company out there to do a virtual keyboard.

Absolutely. I was just giving a concrete example of an existing virtual keyboard. If you know of others, feel free to post them too.
 
Reply
 
Reply
post #23 of 32
post #24 of 32
Quote:
Originally posted by elwood56
This is interesting.
http://www.youtube.com/watch?v=iVI6xw9Zph8

Yeah, it's one heck of a demo, it was noted on Slashdot a week or two ago... I wonder what kind of computing power they have to back that demo up? Must be massive. 8)

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #25 of 32
Quote:
Originally posted by melgross
But, I find Graffiti to work very well.

I still can't reliably write a 'k' with *(&@#$'ing Graffiti 2.
post #26 of 32
Quote:
Originally posted by Xool
If a full media center system is connected to your TV you can use a Front Row interface.

No, thanks, unless it's wireless connectivity between the media center and TV, etc.

Quote:
But if you want something like an iPod to carry from room to room to control your music and maybe do a few basic web apps (email, surfin', widgets) this tablet may be for you! [/B]

Yes, please. With an input mechanism efficient enough for composing forum posts.
post #27 of 32
Quote:
Originally posted by sjk
I still can't reliably write a 'k' with *(&@#$'ing Graffiti 2.

Yeah, that was tough, until I got the hang of starting from the top right, moving down slightly to the left, making a small circle clockwise, and then continuing downwards towards the right after the circle crossed the upper line. Then it was easy. But it took several days.
post #28 of 32
Quote:
If Apple comes out with some device with a 10" screen weighing a pound, or so, having a long battery life, I might see it having a market.

In that case, Apple would be late to the party. Fujitsu already has one.

And about patents, it's not how original your idea is, it's how good your lawyer is!
post #29 of 32
Quote:
Originally posted by skatman
In that case, Apple would be late to the party. Fujitsu already has one.

And about patents, it's not how original your idea is, it's how good your lawyer is!

Apple was late with the iPod and an internet music store as well.
post #30 of 32
Quote:
Originally posted by melgross
Apple was late with the iPod and an internet music store as well.

Absolutely not true.
post #31 of 32
Quote:
Originally posted by skatman
Absolutely not true.

So, Apple was one of the first to come out with a music player, and one of the first to come out with an internet music store?

Has history changed that much?
post #32 of 32
Quote:
Originally posted by melgross
So, Apple was one of the first to come out with a music player, and one of the first to come out with an internet music store?

Has history changed that much?

I remember when the iPod was first announced...lots of folks were underwhelmed because it seemed so unoriginal to them. But since then, people seem to think that Apple invented the mp3 player.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Mac OS X
AppleInsider › Forums › Software › Mac OS X › New Apple filing details virtual input touch-screen interface