Apple files for external audio blending, wide touchpad patents

[AppleInsider]

Apple Inc. is looking to patent a software technique that would give iPod listeners access to critical external audio sounds -- such as airline announcements -- without having to remove their earbuds or pause their music. Separately, the company has also filed for continuation patent on a wide notebook touchpad that incorporates advanced touch control features.

iPod/iPhone external audio blending

The first filing, titled "presentation of audible media in accommodation with external sound," notes that external sounds -- such as traffic and street noise -- is typically reduced or blocked when a user is listening to audio content on their iPod or iPhone.

However, Apple claims that such external sound can be presented to users via the same interface, channel, or audio-output device used to present music or video sound tracks through a blending technique.

"This allows the user to use the headphones to listen, for example, to an announcement being made by the pilot, converse with flight attendants or other passengers without having to take the headphones off," the company wrote in the filing. "In addition, the media player can automatically resume the presentation of the media upon request (e.g., by pressing the pause again). It will be appreciated that external sound and digitally stored audible media (e.g., music files) can be combined (or blended) and presented to the user via the headphones in accordance with user input or various other programmable criteria."

Apple added that the media player can also store external sound as digital data and present it to the user with various play-back functions (e.g., pause, forward, backward, skip) that digital media players normally provide in connection with presentation of digital media. As such, users can configure the media player to effectively store the external sound while media is being presented to them. The user can later listen to the stored external sound when it is convenient. In addition, the use can conveniently go over the stored external sound with play-back functionality similar to that provided by digital media players.



The February 7, 2006 filing is credited to Apple employees Michael Lee and John Arthur.

Wide touchpad on a portable computer

In its second filing, which is presented as a continuation of an earlier patent on the same subject, Apple describes a notebook computer where the touchpad extends substantially into the palm rests areas of the base assembly.

"The wide touchpad may be a cursor control device having the capabilities of conventional computer mouse devices, such as the ability to point, drag, tap, and double tap objects on a graphical user interface, as well as more general purposes such as scrolling, panning, zooming, and rotating images on display screen," the filing states. "The wide touchpad extends into the areas on the surface of the base assembly that are normally reserved for palm rest areas (e.g., flat areas on the surface of the base assembly that support a user's palms and/or wrists while typing)."

Apple also says "the wide touchpad can filter multiple contact patches in order to accept a particular contact patch in one area of the touchpad while rejecting a second contact patch elsewhere on the wide touchpad. In one embodiment, a sensor is disposed between the keyboard and touchpad. The sensor defines a planar sensing region extending upwards from the top surface of the base assembly. The sensor detects a user's hand that may be resting on the base assembly with a palm portion making contact with a portion of the wide touchpad and the fingers extending toward keyboard."



When this detection is made, any contact made with a corresponding portion of the touchpad is rejected, having been interpreted as unintentional contact by the user, according to the filing. Alternatively, detection of fingers extending toward the keyboard may be evaluated as one of many factors used to decide whether and what significance to accord to contact with the touchpad.

"For example, other factors may include the profile of the contact with the touchpad, the level of keyboard activity at the time of contact, etc," Apple wrote. "In this way, the touchpad may effectively serve as a palm rest (e.g., the user may intentionally rest one or more palm or other part of a hand or arm on a portion of the touchpad, which is recognized as an unintentional input) in addition to a functional touchpad when an input is interpreted as being an intentional contact by the user."

The March 30, 2007 filing is credited to Apple interface designers Steve Hotelling, Chris Ligtenberg, Duncan Kerr, Bartley Andre, Joshua Strickon, Brian Huppi, Imran Chaudhri, Greg Christie, and Bas Ording.

[ View this article at AppleInsider.com ]

[GQB]

Originally Posted by AppleInsider

Apple added that the media player can also store external sound as digital data and present it to the user with various play-back functions (e.g., pause, forward, backward, skip) that digital media players normally provide in connection with presentation of digital media. As such, users can configure the media player to effectively store the external sound while media is being presented to them. The user can later listen to the stored external sound when it is convenient.

So headphone zombies will now be able to truly treat the real world as one big TIVO event.
What we say to Johnnie... "Clean up your room now!"
What Johnnie hears... "Clean up y.. [pause]"

[MsNly]

Originally Posted by GQB

So headphone zombies will now be able to truly treat the real world as one big TIVO event.
What we say to Johnnie... "Clean up your room now!"
What Johnnie hears... "Clean up y.. [pause]"

[blastfamy]

There is always somthing funny about the patent requests that Apple puts out. Apple - being the secret mongers they are (how we love them for it) - put out patenets for technologies that they never intend to implement.

Take their phone (have you heard of it?!); multi-touch was somthing that they would patenet for sure. Did we see anything go to the patenet office with the title 'implementation for multi-capacitor display for a handheld moble device' ? NO! I think the closest thing we saw was an implementation for an iPod to have a screen overlay that could accept multiple inputs. It looked like a soundboard that should be used with logic. The big difference is that instead of a mechanical slider or keyboard utilizing multi-touch, it was the finger that was intendid to give input. It was very coy indeed.

Apple could be doing anything with this. I saw nobody make any allusions to a phone interface from the mechanical overlay, and I doubt that anyone will come to any (correct) conclusions about this. Just know that whatever they're doing, it'l be cool when it's done!

[AnAppleADay]

Now this is interesting. Right before the Mac presentation of last August 7, I've been thinking that if Apple would extend MultiTouch into desktop and laptop computing, they would not do it using a touch screen, but something that would lay next to your keyboard. This patent does indeed show a way to do that.
But I also think that the new iMac might give some clues. Doesn't the combination of the new flat aluminum keyboard and the already existing white mouse look a bit awkward? It's just not totally Apple-like. This mouse is going to be replaced soon as well. By something aluminum and flat. With a scratch-resistant glass surface, just like the iMac itself. A MultiTouch pad! It would have the same form factor as the new keyboards, and could be placed neatly next to it, or even before it.
This pad could have a miniature representation of the screen for easy orientation. But it wouldn't be convenient to have to look at both the screen and the pad all the time, so the system would also need to give a visual clue on the real screen, like a mouse pointer, when the pad senses the approach of one or more fingers. Actually touching the pad then would mean a click, and allow for drag, resize, zoom, select etc. Once we got used to this idea, and more and more applications make full use of this, the touch thing could become more and more sophisticated, adding pressure sensitivity for example. Many creative, artistic possibilities here, along the lines of Jeff Han.

Considering all the patents in this area, Apple is clearly aware that touch would allow for a much more intuitive way to communicate with a computer. As an extra, if they have a very desirable and well-patented technology, that would give copycats less chance...

So this is what I think: together with Leopard, Apple will introduce a separate MultiTouch pad that is to replace the mouse, and allows for far more powerful input.

[murphyweb]

So whenever all the loony's come out with the wish for multi-touch iMac's and notebooks i laugh and laugh and laugh in their faces. It is the most stupid idea in the world and if anyone with half a brain actually thought about it for a minute they would realise that it is a non starter (and i know many sensible people on these forums also think this too) nobody wants to spend all day with their arms in the air touching an iMac screen, it will never work. he notebook screen is no better as it will very quickly become greasy and dirty to the point you would need to clean it every 5 mins. So everyone please stop now, their will never be a multi-touch screen for a computer from Apple.


BUT.. This patent application does in fact throw up a potential great application for multi-touch on a notebook. I am sure someone with good photoshop skills can mock one up in minutes but take a MBP, keep the screen as it is but replace the lower section (i.e. the whole keyoard/trackpad area) with one giant multi-touch surface. So your movements on the MT Surface are replicated on the screen. The keyboard would then become just like the iPhones keyboard, only popping up when you need to write something. I actually really like that idea.

I am pretty sure that this is the new MBP to be launched at Macworld next year!

[Bergermeister]

As one of those who would fall into your definition of a loony, I would ask if you have truly followed the discussions we have been having about multi-touch. Few suggested that monitor screen itself be the touch panel; my suggestion for the subnotebook was for a replacement for the keyboard. I can see great benefits in this in a compact device. I can also see great benefits of a USB device for a desktop (or a large interface on a notebook) that would allow similar input (recall Fingerworks?). There is great potential, but it does have to be implemented correctly.

The biggest potential I see is user-configurable workspaces. With the function dictionary Apple is working on, it could be possible to create your own workspaces that function the way you do, not the way the computer wants you to.

BTW, I live in a very humid location that also has lots of dust and volcanic ash, so I already have to clean my mousepad every five minutes and my keyboard every day.

[Kolchak]

Originally Posted by Bergermeister

I can see great benefits in this in a compact device. I can also see great benefits of a USB device for a desktop (or a large interface on a notebook) that would allow similar input (recall Fingerworks?). There is great potential, but it does have to be implemented correctly.

Define "correctly." I see nothing incorrect about my iGesture pad. Unless you want an integrated display, which would mean it doesn't really use gestures. That's nothing more than a touchscreen with MultiTouch as a separate function. A good pointing device doesn't require you to look at it to use (e.g. mouse, trackball, trackpad, joystick, joypad, Wii nunchuks, keyboard, etc.)

BTW, the "incorrect" iGesture pads have not been seen on eBay in quite a few months. Not even one. And the last ones that were sold went for several times their original retail price.

The biggest potential I see is user-configurable workspaces. With the function dictionary Apple is working on, it could be possible to create your own workspaces that function the way you do, not the way the computer wants you to.

Again, poor example. The Fingerworks iGesture pads were and are fully programmable via a Java application they included, then the command sets were stored in flash ROM in the pads so you can use it on any computer you plug it into without the need for a driver. Any of the supported gestures could output any of the supported USB commands, including pointing, clicking, double clicking, dragging, zooming and keystrokes. Combine it with USB Overdrive and its application-specific command sets and one can do pretty much everything you're asking for.

[appletweak]

Interesting replies by all, I think you may be on to something here:

• Everyone has been clamoring for a sub-notebook MacBook Pro
• Multi-touch can be used on surfaces other than glass...
• Gesture-based navigation can be useful on surfaces where there is enough room to make a gesture
• Apple filed for a larger surface gesture patent, a multi-touch library patent, and light-up buttons (shown in patent on i-pod like device) when touched, and a bottom loading dvd drive
• Already have 15" unit with lighter, more power-efficient LED displays

What does this mean? Forces are coming together to bring you the thinnest, most power-efficient sub-notebook on the market:

• 12" screen, I-Pod-like thickness, .40" or less?
• Bottom loading DVD-Combo Drive
• Touch-sensitive illuminated keyboard, flush with base (no mechanical keys)
• Full-width illuminated multi-touch pad
• Flash-based drive, 32GB?

And, just to prove it I made some pictures. You heard it hear first, folks.

Daytime:


Illuminated:

[Bergermeister]

Originally Posted by Kolchak

Define "correctly." I see nothing incorrect about my iGesture pad. Unless you want an integrated display, which would mean it doesn't really use gestures. That's nothing more than a touchscreen with MultiTouch as a separate function. A good pointing device doesn't require you to look at it to use (e.g. mouse, trackball, trackpad, joystick, joypad, Wii nunchuks, keyboard, etc.)

BTW, the "incorrect" iGesture pads have not been seen on eBay in quite a few months. Not even one. And the last ones that were sold went for several times their original retail price.


Again, poor example. The Fingerworks iGesture pads were and are fully programmable via a Java application they included, then the command sets were stored in flash ROM in the pads so you can use it on any computer you plug it into without the need for a driver. Any of the supported gestures could output any of the supported USB commands, including pointing, clicking, double clicking, dragging, zooming and keystrokes. Combine it with USB Overdrive and its application-specific command sets and one can do pretty much everything you're asking for.

You might want to reread my post: I in no way ever suggested that the Gesture Pads were "incorrect"; I was talking about the potential Apple implementation, which has not yet arrived. As you correctly point out, though, the Gesture Pads are no longer available, despite being the perfect device in your mind as well as that of the many very happy owners (why did they close down; bought out, perhaps?). I was discussing future implementation, and yes, I see a version that is actually a screen and replaces the keyboard altogether, though not in all situations. My "poor example" also did not deny the possibility of onboard memory; I simply related the device to another on-going effort by Apple to develop a gesture dictionary, and said it could plug in via USB (which the Gesture Pads could); we do not know for certain if Apple do not plan to combine gestures with a multitouch interface, or go one way of the other. A built in dictionary of gesture would be great, and there would likely be an option for the user to define his/her own gestures (he might be left-handed, or he might have only three fingers, and thus the built-in gesture might not work for him). There are many, many possibilities, but I also think therefore there are many ways to do it wrong; for example, how are they going to turn off the wide screen pad in the example above when the user is typing on the keyboard? Would not his palms be resting on and therefore interacting with the touch interface?

[Bergermeister]

Appletweak: Nice work. Would you mind exploring in am image the concept of a completely multi-touch based ultra-sub-notebook? It would not have a keyboard, IMO.

Before anyone gets all worked up, this is a concept device. I would like to see what someone with great artistic abilities than myself could do with the idea. I remember some people doubted the practicality of a finger-touch panel in a phone, but then Apple released the iPhone.

[Electric Monk]

Originally Posted by Bergermeister

As you correctly point out, though, the Gesture Pads are no longer available, despite being the perfect device in your mind as well as that of the many very happy owners (why did they [FingerWorks -Ed.] close down; bought out, perhaps?).

Apple bought them.

Before anyone gets all worked up, this is a concept device. I would like to see what someone with great artistic abilities than myself could do with the idea. I remember some people doubted the practicality of a finger-touch panel in a phone, but then Apple released the iPhone.

I, at least, am still with you on that. I would think that haptic (vibration) feedback would be needed though, because typing would be much more common then the iPhone. Likewise a hardware 'summon keyboard' button along with a couple bumps for the home keys would be needed for touch typists/usability.

I think that the subnotebook is so restricted in space that using an iPhone like screen (in place of the mechanical keyboard) with virtual keyboard is required. Larger products, of course, just use a regular touchpad scaled bigger and better.

FIngerworks themselves made a keyboard with built in gesture pad, and I imagine Apple could do the same. Likewise making the touchpad bigger on the larger notebookes (the palmrest problem could be simple: a large enough contact and it switches that part of the pad off)

[Kolchak]

Originally Posted by Bergermeister

A built in dictionary of gesture would be great, and there would likely be an option for the user to define his/her own gestures (he might be left-handed, or he might have only three fingers, and thus the built-in gesture might not work for him). There are many, many possibilities, but I also think therefore there are many ways to do it wrong; for example, how are they going to turn off the wide screen pad in the example above when the user is typing on the keyboard? Would not his palms be resting on and therefore interacting with the touch interface?

Perhaps you didn't read me right. All gestures are programmable. All of them. So your three-fingered fellow could remap gestures however he wants them on an iGesture pad, although obviously he'll lose some functionality since he can't use four-finger gestures. And from the Fingerworks FAQ:

I'm left-handed. Will I be able to point and gesture with my left hand?

iGesture products auto-detect which hand you are using and work the same with either hand. On TouchStream Keyboards, you can swap hand mappings with a special configuration sequence.

As for the palms, that's not a problem. Even with iGesture tablets, you can rest your fingers or palm heels on the pad. It's not that sensitive that the cursor is shuddering as you type. You are positing a bunch of hypotheticals which hands-on experience with any iGesture tablet would tell you are not issues at all.

[onlooker]

This is an expansion on the fingerworks patents that Apple bought? I think. I'd like to see it for desktops also.