New media patent filing puts Apple closer to touchscreen Macs

[AppleInsider]

A new patent filing by Apple illustrates the potential of future multi-touch Macs could allow a whole new suite of creative software, and even a whole desktop operating system, controlled simply through on-screen gestures.

Filed just over a year ago as the first iPhone was about to go on sale, the patent application published on Thursday explains that the multi-touch input found on Apple's newer trackpads -- particularly the MacBook Air and MacBook Pro pads that launched several months after the filing -- are limited in what they can achieve and in many cases go against the user's own instinct for what to do in a given event.

With that in mind, the patent's sole inventor, Greg Christie, suggests that many creative programs could be changed to take advantage of a full multi-touch interface in what appears to mirror the same music, photo, and video editing tools it already produces for Macs.

A music editor could have a live mixing feature that lets the user "scratch" two on-screen turntables just by spinning fingers and pushing faders, not unlike a real DJ; sound effects could play at the same time to provide audio as well as visual feedback, Apple says.

In turn, a photo editing program could overlay on-screen sliders and other tools for adjusting image quality or rotating photos on the fly. A video editing tool could alternately let users manipulate the timeline, pan the image, and change volumes on individual channels by grabbing on-screen components with one or more fingers.

Many apps could use an iPod-like scroll wheel to navigate content, Apple says.





Despite the skew towards media apps, however, the filing also makes clear that Apple is devising methods to control its entire Mac operating system through multi-touch rather than a mouse and keyboard.

On a basic level, finger input could be used to select icons in the Dock or to tap individual menu items. Apple's described methods, however, would replace the two-finger trackpad scrolling of today with controls on the screen proper: an interface could either recognize two fingers on the screen itself and pan along with the user, or else turn to an on-screen directional pad that permits four-way scrolling with single finger presses.

The Mac maker's inventor has also developed a unique way of registering a right-click equivalent on an icon or list item. Placing two fingers on an item and then tapping the right finger would open a context-sensitive menu that would allow copying, shortcut creation, and other functions frequently off-limits to touch displays.





As always, filings such as this are never guarantees that Apple will produce a shipping device based on what it has discovered; the document also implies that what was learned with the new invention could be applicable to a phone or portable media player instead of the computer-grade interface it appears to describe.

Nevertheless, the patent filing adds to the growing catalog of technology that indicates Apple's desire to expand beyond handhelds with simple interfaces, including research the company has performed on more complex multi-touch surfaces. It also comes as Apple is still believed to be developing a tablet-like device in between the iPhone and full Macs that would require more advanced multi-touch controls -- albeit one which Apple has delayed and without a definite release window in sight.

[ View this article at AppleInsider.com ]

[ Digg this story ]

[Crtaylor]

I cannot wait for the touchscreen Macs to come out. I STILL WANT STYLUS SUPPORT THOUGH. If they are going to make it so that you will not need a mouse or a keyboard, some details are just so minute on a screen that fingers are not small enough to get to them.

[dagamer34]

We know it's coming, it's only a matter of when. 10.6 maybe? Or perhaps 10.7 (end of 2010 likely). We shall see...

[mcarling]

I don't think this will replace traditional Macs anytime soon. Rather, I think this is the basis for a tablet that will be new part of the product line.

[Ireland]

10.6 and a tablet, s'obvious.

[quinney]

H P is selling a touchscreen computer with Vista. It will be interesting to
see how it sells and is accepted.http://www.provantage.com/hp-compaq-...a~7HEWX038.htm

[Ireland]

Originally Posted by quinney

H P is selling a touchscreen computer with Vista. It will be interesting to
see how it sells and is accepted.http://www.provantage.com/hp-compaq-...a~7HEWX038.htm

That thing has been out over a year, it's a no show, really.

[Walter Slocombe]

Originally Posted by quinney

H P is selling a touchscreen computer with Vista. It will be interesting to
see how it sells and is accepted.http://www.provantage.com/hp-compaq-...a~7HEWX038.htm

it WAS interesting to see how badly it sucked on the M$ demo.

interesting that this patent info is out just at the right time to be sunk by iPhone news



http://www.engadget.com/2008/07/10/v...-in-two-years/

cant believe you missed that Brendan

[SpamSandwich]

The other reason I dislike this type of interface... it could be a very difficult, if not impossible interface to work with for a large number of people. There are plenty of folks without hands and arms, even more these days, and this seems to be a computer interface developed with blinders on to the needs of the disabled.

[McHuman]

It was confirmed in Wired's article that the iPhone was a distraction from the tablet team that had already been in progress for 1-2 years. I'm sure they are back at work now for Jan 2009 or Jan 2010 release.

[SpamSandwich]

If there ever is a full screen multi-touch/multifunction computer from Apple, I would hope they'd implement a true 'control surface', with something akin to a deformable surface to simulate key edges that would appear and disappear as needed.

[melgross]

Originally Posted by SpamSandwich

The other reason I dislike this type of interface... it could be a very difficult, if not impossible interface to work with for a large number of people. There are plenty of folks without hands and arms, even more these days, and this seems to be a computer interface developed with blinders on to the needs of the disabled.

You might as well say that about keyboards, mice, trackballs, game controllers, etc.

There are spoken interfaces to help these people.

When a major interface is developed, it must be developed for the 95+ percent of people who can use, and benefit, from them.

Then accommodations must be made for the rest.

[melgross]

Originally Posted by SpamSandwich

If there ever is a full screen multi-touch/multifunction computer from Apple, I would hope they'd implement a true 'control surface', with something akin to a deformable surface to simulate key edges that would appear and disappear as needed.

I don't know of any current technology that would allow that on a viewscreen. There are ways to do that on a non viewing surface though. There are materials that can allow raised keys on the surface with some small electrical input. but that's for certain opaque materials. If they can get that to work on a top surface material for a display, it would be of far more use than for just a keyboard.

[iVlad]

Ohh I wish OS 11 was closer. We're 4 OS away.

Would really be a new step. Mac OS 11touch. I guess technology moving much faster than Apple expected. Or maybe not. I think 10.7 might be touch. I don't think 10.5 or 10.6 can handle all the touching. Leopards might get angry. Maybe "Lion" will let us touch.

100% that Apple will do the touch-screen right, just like iPhone. Have anyone noticed how Apple is slooooowllyy introducing touch into our lives. Macbook Air, iPhone, iPods. Its a very smart way to teach its users how to operate Apple machines.

On the other hand I feel bad for Windows users, because they will just jump in into touch world in Windows 7. Im sure most will not even use it, because its such a big step.

Im just fascinated with Apple's plans.
Thank you for reading. =) lol

[monstrosity]

I have had a touch screen mixing app exactly like that on the cards for three years, I dont understand what the patent here is for, I have not read in detail, but if its as simple as touch screen mixing my prior art goes back a long way.

Sorry apple.

[Marvin]

Originally Posted by Ireland

That thing has been out over a year, it's a no show, really.

What about the one I posted here:

http://forums.appleinsider.com/showthread.php?t=88689

There's a video demo of it in action. It's much more attractive than the other touch HP model.

I wonder if Apple will be using the same capacitive touch on the desktop - the IR version HP use still seems to have issues with responsiveness. It's not too bad though and certainly better than the Microsoft demo not too long ago on the laptop.

[Parsec]

I don't buy the touchscreen interface. The technology has been out for decades (not multi touch of course) and never took off.

Just too impractical, a lot of people don't want to be interacting with a computer quite so much especially in an office environment, and there's the fingerprint problem.

My idea (patent pending lol) would be for a separate device to replace the mouse that is an OLED screen and an input device. It could mirror what is on the main display (or do specialised things) and allow multi-touch gestures.

[themoonisdown09]

Originally Posted by monstrosity

I have had a touch screen mixing app exactly like that on the cards for three years, I dont understand what the patent here is for, I have not read in detail, but if its as simple as touch screen mixing my prior art goes back a long way.

Sorry apple.

If no one has every sent in a patent for this, then that's why Apple did this.

[melgross]

Originally Posted by Parsec

I don't buy the touchscreen interface. The technology has been out for decades (not multi touch of course) and never took off.

Just too impractical, a lot of people don't want to be interacting with a computer quite so much especially in an office environment, and there's the fingerprint problem.

My idea (patent pending lol) would be for a separate device to replace the mouse that is an OLED screen and an input device. It could mirror what is on the main display (or do specialised things) and allow multi-touch gestures.

Your idea is nothing new. Wacom has had a similar device out for years. It's called the Cintiq.

It can be controlled with their stylus, but not a finger, so that;s a difference.

But, the principal is the same. Look at the prices, and the weights.

I just bought the new, small version. It's 9" x 12", and weighs 4 pounds (approx.). It's still $999.

If it used multi-touch, it would match Apple's concept, but would be at least as expensive, likely more so, because multi-touch screens cost more, and just as heavy.

So, no, not a good idea for most people.

The idea of making the main screen multi-touch is the only practical way, because it will only be an incremental price increase over the regular screen, which wouldn't amount to more than a few percent of the cost of the computer.

But a Cintiq-like device would double the price if it were anyway near being full size.

[melgross]

Originally Posted by themoonisdown09

If no one has every sent in a patent for this, then that's why Apple did this.

He ASSUMES his app is exactly like this. Most likely, it isn't.

Besides, it's not what the end result is that is patentable, it's HOW you achieve that end result.

There can be several ways to do the same thing, and they can all receive patents.

[BB Sting]

Originally Posted by iVlad

Ohh I wish OS 11 was closer. We're 4 OS away.

Would really be a new step. Mac OS 11touch. I guess technology moving much faster than Apple expected. Or maybe not. I think 10.7 might be touch. I don't think 10.5 or 10.6 can handle all the touching. Leopards might get angry. Maybe "Lion" will let us touch.

100% that Apple will do the touch-screen right, just like iPhone. Have anyone noticed how Apple is slooooowllyy introducing touch into our lives. Macbook Air, iPhone, iPods. Its a very smart way to teach its users how to operate Apple machines.

On the other hand I feel bad for Windows users, because they will just jump in into touch world in Windows 7. Im sure most will not even use it, because its such a big step.

Im just fascinated with Apple's plans.
Thank you for reading. =) lol

Apple likely won't release an os called OS 11. Steve Jobs has been quoted as saying that OS X could last as long as 20 yrs. Please forgive me, I can't give you the exact quote(I saw it in a youtube video) but Apple will probably release OS X.11 in place of OS 11.

[melgross]

Originally Posted by BB Sting

Apple likely won't release an os called OS 11. Steve Jobs has been quoted as saying that OS X could last as long as 20 yrs. Please forgive me, I can't give you the exact quote(I saw it in a youtube video) but Apple will probably release OS X.11 in place of OS 11.

I thought he said 15.

[DaveyJJ]

I personally have thought that the idea of a touchscreen laptop is a no-brainer that is, without doubt, going to come to pass sooner or later. That is, a laptop with two screens, like the one pictured above, but much like an Apple Air in dimensions etc.

The iPhone, touch and new slim/flat aluminum Apple keyboards with flat keys are simply training devices for Apple to both train computer users to expect it, know how to deal with it when it comes out, and gather intelligence and feedback as they close in on some final design solutions to this idea.

People here know the possibilities ... obviously. A standard keyboard when needed on the laptop's bottom screen. Customizable perhaps based on application so that Garage Bands "keyboard" might include other items. The ability to touch and slide components in your keyboard around to accommodate each user's preferences easily, i.e., moving the traditional touch pad from the centre to the right or left side, splitting the keyboard and moving the touchpad up between the two pieces, the ability to save keyboard configurations based on application, sliding letters around into more useable combinations than the standard QWERTY the way you move iPhone app icons around now, etc etc etc. And of course, all of Apple's patented multi-touch gestures available for use and smart typing features ala the touch and iPhone.

All of these are huge advantages to a screen-based keyboard. I agree though, as an artist and designer, I'd like a small stylus tucked away somewhere too for precise sketching, for example, or at least support for an Apple stylus ($29, sold separately).

And finally, I agree that there are some disadvantages as people get used to "no tactile feedback" especially. But mark my words, Apple's training us right now and watching us and will move in this direction sooner than later ... I think it's inevitable.

[melgross]

Originally Posted by DaveyJJ

I personally have thought that the idea of a touchscreen laptop is a no-brainer that is, without doubt, going to come to pass sooner or later. That is, a laptop with two screens, like the one pictured above, but much like an Apple Air in dimensions etc.

The iPhone, touch and new slim/flat aluminum Apple keyboards with flat keys are simply training devices for Apple to both train computer users to expect it, know how to deal with it when it comes out, and gather intelligence and feedback as they close in on some final design solutions to this idea.

People here know the possibilities ... obviously. A standard keyboard when needed on the laptop's bottom screen. Customizable perhaps based on application so that Garage Bands "keyboard" might include other items. The ability to touch and slide components in your keyboard around to accommodate each user's preferences easily, i.e., moving the traditional touch pad from the centre to the right or left side, splitting the keyboard and moving the touchpad up between the two pieces, the ability to save keyboard configurations based on application, sliding letters around into more useable combinations than the standard QWERTY the way you move iPhone app icons around now, etc etc etc. And of course, all of Apple's patented multi-touch gestures available for use and smart typing features ala the touch and iPhone.

All of these are huge advantages to a screen-based keyboard. I agree though, as an artist and designer, I'd like a small stylus tucked away somewhere too for precise sketching, for example, or at least support for an Apple stylus ($29, sold separately).

And finally, I agree that there are some disadvantages as people get used to "no tactile feedback" especially. But mark my words, Apple's training us right now and watching us and will move in this direction sooner than later ... I think it's inevitable.

What's happening is that every company with an OS, and some without, are looking for the "Next Big Thing" to replace mouse (trackball) and keyboard input.

But, it's really difficult to do that. For decades, it was said that voice input would be the next step. But, not only has that proven to be much more difficult than was thought, it's also clumsy for anything more than private use.

Can you imagine an office with those cubbyholes of workers, each one speaking out every command all day long?

Or doing work on your computer in some public space?

I can't.

So, as that seems to fade into the sunset, except for some specialized purposes, something else must take its place.

So far, a touch screen looks like the most likely result. Possibly, some form of multitouch will work.

But even this isn't perfect.

It's also not secret. Everyone's working on their own version.

[Olternaut]

Lets get one thing straight. There is no such thing as a tablet computer until Apple invents it. Whatever the market has out there right now that claims to be a tablet is crap plain and simple. Once again Apple is going to have to define the market for this category of computer device. No, not re-define it.....but to define it from scratch.

[murphyweb]

Originally Posted by Olternaut

Lets get one thing straight. There is no such thing as a tablet computer until Apple invents it. Whatever the market has out there right now that claims to be a tablet is crap plain and simple. Once again Apple is going to have to define the market for this category of computer device. No, not re-define it.....but to define it from scratch.

Well said.

[ap780]

Originally Posted by murphyweb

Well said.

I don't think thatbapplw will do touch screen macbooks but many macbookpro but not unroll 2010 I'm guessing august 16 2010

[ap780]

Originally Posted by murphyweb

Well said.

I don't think thatbapplw will do touch screen macbooks but many macbookpro but not unroll 2010 I'm guessing august 16 2010

[Rickibird]

Having worked in the past for an airline developing controls for glass cockpits for use on touchscreen, I've been thinking of a touch screen keyboard for a few years now. Glad to see some others here are thinking that way too.

Imagine the flexibility of standard keyboard, piano keyboard, drawing tablet, mouse-pad, all kinds of widgets and applications in any configuration, for improving your interfacing with your Mac.

I can't see that a touchscreen monitor is anywhere near as good as they're slower to work with and don't really promise as much flexibility as a "touch keyboard" could. A touch screen would be a gimmick, a touch keyboard would be revolutionary.

Yep . . . I'm looking forward to the "iMac Touch".

[Marvin]

The trouble with capacitive touch is it won't have pressure sensitivity so it won't be as good as a Wacom for drawing - you also can't use average pens. If it uses pressure, it won't control like an iphone. Perhaps they'd need to develop a best-of-both solution.

I guess if they somehow measure the the area of contact a finger has, that might be able to simulate pressure but it's not ideal.

[melgross]

Originally Posted by Marvin

The trouble with capacitive touch is it won't have pressure sensitivity so it won't be as good as a Wacom for drawing - you also can't use average pens. If it uses pressure, it won't control like an iphone. Perhaps they'd need to develop a best-of-both solution.

I guess if they somehow measure the the area of contact a finger has, that might be able to simulate pressure but it's not ideal.

This isn't a major engineering obstacle. It's a matter of cost. I've red discussions about the requirements that would be needed, and how to overcome them.

If there's a demand, someone will manage it. Perhaps Wacom themselves. I just bought the 9 x 12 Cinteq, and it works great. If it had multitouch as well, perhaps with a switch to change modes, that would be great!

[Olternaut]

Originally Posted by Rickibird

Having worked in the past for an airline developing controls for glass cockpits for use on touchscreen, I've been thinking of a touch screen keyboard for a few years now. Glad to see some others here are thinking that way too.

Imagine the flexibility of standard keyboard, piano keyboard, drawing tablet, mouse-pad, all kinds of widgets and applications in any configuration, for improving your interfacing with your Mac.

I can't see that a touchscreen monitor is anywhere near as good as they're slower to work with and don't really promise as much flexibility as a "touch keyboard" could. A touch screen would be a gimmick, a touch keyboard would be revolutionary.

Yep . . . I'm looking forward to the "iMac Touch".

With one of Apple's patents (I forget which one) it details how the very pixels on the lcd can be made to shift position up and down. That suggest the possibility of a morphing screen. Now imagine your product when turned off or blank is totally flat like any touch surface. Then, when in keyboard mode not only does it display keyboard keys but the pixels underneath them physically raise up so you can actually feel them. And as you press down on them it depresses with a feedback similar or perhaps even surperior to that of a regular keyboard.
Now consider if custom buttons, shapes, clickwheels, whatever you can imagine, could appear and dissapear at your will! That opens the door to almost unlimited possibilities for both a tablet and the iphones.....the ipods even.

[xc3ll]

With all the fervor about dual lcd screens, people forget to think about two simple obstacles:

Fingernails and watches.

Try typing on a table or any other hard, flat surface for over 10 minutes. Your index and middle fingers will hurt like hell after two minutes. There's no way anybody is gonna be doing any significant typing on a hard surface (that's why keys depress inwards) Now you may say that it just takes time to adjust to, and that may be true, but anyone walking into an apple store to try it out isn't gonna take the time to adjust to it. They're gonna try it and say, "ouch, that f***ing hurts" and move on. (as a side note, anyone who has been trained to play the piano correctly would never play it on a tablet cuz it would also hurt like hell--if you wanna try it for yourself, raise your wrists to be parallel with your table and try airpianoing(word?) anything.)

Now, not everyone wears a watch, but a lot of business people do. Can you imagine the abuse the screen would take from watches, cuff links, dirt on hands, and just dust being wiped across the screen? (Have you ever ran your finger across a dirty car's paint? Well I myself wouldn't but doing so leaves some noticeable scratches in the clear coat.)

Now, if apple invents a soft, gel like screen that's impervious to scratches and is clear as day, then please, by all means, bring it out. Until that day comes, no thanks.

[zerfman]

Originally Posted by xc3ll

With all the fervor about dual lcd screens, people forget to think about two simple obstacles:

Fingernails and watches.

Try typing on a table or any other hard, flat surface for over 10 minutes. Your index and middle fingers will hurt like hell after two minutes. There's no way anybody is gonna be doing any significant typing on a hard surface (that's why keys depress inwards) Now you may say that it just takes time to adjust to, and that may be true, but anyone walking into an apple store to try it out isn't gonna take the time to adjust to it. They're gonna try it and say, "ouch, that f***ing hurts" and move on. (as a side note, anyone who has been trained to play the piano correctly would never play it on a tablet cuz it would also hurt like hell--if you wanna try it for yourself, raise your wrists to be parallel with your table and try airpianoing(word?) anything.)

Now, not everyone wears a watch, but a lot of business people do. Can you imagine the abuse the screen would take from watches, cuff links, dirt on hands, and just dust being wiped across the screen? (Have you ever ran your finger across a dirty car's paint? Well I myself wouldn't but doing so leaves some noticeable scratches in the clear coat.)

Now, if apple invents a soft, gel like screen that's impervious to scratches and is clear as day, then please, by all means, bring it out. Until that day comes, no thanks.

I disagree, think your just tapping your fingers to hard. i mean even with regular keyboards you are in essence jsut typing on a hard surface. when you press down does your keyboard get all squishy?? plus the touch screen is going to be like solid metal or anything.

[xc3ll]

Originally Posted by zerfman

I disagree, think your just tapping your fingers to hard. i mean even with regular keyboards you are in essence jsut typing on a hard surface. when you press down does your keyboard get all squishy?? plus the touch screen is going to be like solid metal or anything.

Its going to be glass, like the iPhone. Glass Plane is about the same feel as solid metal. (If you don't trust me, put on a blindfold, and punch a really thick glass pane and then punch a piece of steel. Tell me if you can tell the difference. If it's any consolation, I sure as hell can't. I never did check though. You might wanna double check that. j/k)

Clarification: The reason why I say you have to punch a really thick glass plane is so that it doesn't break. Because if it does break, you'll probably get all sorts of cuts, and it won't hurt nearly as much!

[melgross]

Originally Posted by xc3ll

With all the fervor about dual lcd screens, people forget to think about two simple obstacles:

Fingernails and watches.

Try typing on a table or any other hard, flat surface for over 10 minutes. Your index and middle fingers will hurt like hell after two minutes. There's no way anybody is gonna be doing any significant typing on a hard surface (that's why keys depress inwards) Now you may say that it just takes time to adjust to, and that may be true, but anyone walking into an apple store to try it out isn't gonna take the time to adjust to it. They're gonna try it and say, "ouch, that f***ing hurts" and move on. (as a side note, anyone who has been trained to play the piano correctly would never play it on a tablet cuz it would also hurt like hell--if you wanna try it for yourself, raise your wrists to be parallel with your table and try airpianoing(word?) anything.)

Now, not everyone wears a watch, but a lot of business people do. Can you imagine the abuse the screen would take from watches, cuff links, dirt on hands, and just dust being wiped across the screen? (Have you ever ran your finger across a dirty car's paint? Well I myself wouldn't but doing so leaves some noticeable scratches in the clear coat.)

Now, if apple invents a soft, gel like screen that's impervious to scratches and is clear as day, then please, by all means, bring it out. Until that day comes, no thanks.

A soft screen would scratch far more than a hard screen. a glass surface would be least scratchworthy.

Evaporated diamond on plastic is being done. It allows for some flexibility, with mucho hardness, and scratch resistance.

My moms old typewriter, an ancient Underwood, took almost a pound of pressure to get a key to impress on the paper. I still have that, and have measured it.

Yet, my mon could type at 90 words a minute, very fast for the day.

It's what we are used to, and now, we are used to terrible computer keyboards.

[xc3ll]

Originally Posted by melgross

A soft screen would scratch far more than a hard screen. a glass surface would be least scratchworthy.

Evaporated diamond on plastic is being done. It allows for some flexibility, with mucho hardness, and scratch resistance.

My moms old typewriter, an ancient Underwood, took almost a pound of pressure to get a key to impress on the paper. I still have that, and have measured it.

Yet, my mon could type at 90 words a minute, very fast for the day.

It's what we are used to, and now, we are used to terrible computer keyboards.

Yes, a glass surface would be most scratch resistant, but is unpleasant on the fingertips. Evaporated diamond on plastic sounds interesting however, I'll have to do some research on that.

As for your moms typewriter, those things were much harder to push down on than today's keyboards, but they're still much softer since they give.

On a side note though, I miss the days of good, solid keyboards. I might have to buy one of those Das Keyboards pretty soon or a old IBM Model 40. (btw, I hate Apple's new keyboard... ug)

[melgross]

Originally Posted by xc3ll

Yes, a glass surface would be most scratch resistant, but is unpleasant on the fingertips. Evaporated diamond on plastic sounds interesting however, I'll have to do some research on that.

As for your moms typewriter, those things were much harder to push down on than today's keyboards, but they're still much softer since they give.

On a side note though, I miss the days of good, solid keyboards. I might have to buy one of those Das Keyboards pretty soon or a old IBM Model 40. (btw, I hate Apple's new keyboard... ug)

They give all right. The keys must be depressed over one entire inch for them to work. It's a very strange feeling, as your fingers are moving for such a long distance. Very unpleasant, and difficult. Extremely tiring. And the jamming!