or Connect
AppleInsider › Forums › Mobile › iPhone › Apple exploring improved, contextual voice commands for iPhone
New Posts  All Forums:Forum Nav:

Apple exploring improved, contextual voice commands for iPhone

post #1 of 36
Thread Starter 
Apple could improve voice commands on the iPhone by making them contextualized, asking users to select, by hand, an application before dictating commands aloud.

The new voice control method is described in a patent application from Apple made public this week by the U.S. Patent and Trademark Office. Discovered by AppleInsider, the patent entitled "Contextual Voice Commands" aims to make voice control of a device more reliable and efficient.

Relying on a combination of quick physical inputs as well as voice commands, Apple's proposed system would allow for contextual input on an application-by-application basis. This would narrow the voice command possibilities in any given application, as opposed to a system-wide method that has a large number of options.

"By using contextual voice commands, a user can execute desired operations faster than by navigating through a set of nested menu items," the filing reads. "Also, contextual voice commands can be used to teach the device to accurately predict the intent of the user from a single voice command. Further, contextual voice commands can be used to vary the manner in which the user provides the voice input based on the context of the device being used."

By having the user select an application and narrow the potential voice commands, a device like an iPhone would be less likely to be "confused." Also, for a user who may be driving a car, requiring them to simply select the e-mail application is much simpler and safer than actually typing out a note.

"Contextual voice commands can be more precise than conventional voice commands that merely control the device as a whole," the application states. "For example, different context specific voice commands can be implemented based on user activity or application in use."



Using this method, Apple would also be able to allow third-party applications to rely on voice commands as well. The application notes that an application programming interface (API) related to a contextual voice command module could be provided to developers, allowing their application to be controlled by voice.

In addition to an iPhone or another portable device with a touchscreen, the patent filing also notes that contextual voice commands could also be used with a Mac or another traditional computer. In this method, users would select a program on screen with a mouse cursor.



Apple's method could also provide visual or audible cues to the user, notifying them what application has been selected or providing a list of potential voice commands. For example, operations such as "compose, reply, forward, read, save, copy, paste, search" and more could be displayed on the screen in an e-mail application, or read aloud to the user.

The application, released this week by the U.S. Patent and Trademark Office, was first filed in June of 2009. The proposed invention is credited to Marcel Van Os, Gregory Novick, and Scott Herz.



The application is particularly noteworthy because earlier this year, Apple acquired Siri, the maker of a personal assistant application for the iPhone. The free Siri application, available on the App Store, relies on voice commands to accomplish tasks from multiple sources of information, such as making reservations at a restaurant, or buying tickets to a movie.

With Siri, voice queries are provided in "natural" English, as a person would use in a conversation. Questions such as "What's happening this weekend around here?" would provide local events, and a follow-up query of "How about in San Francisco?" would change the location.
post #2 of 36
It's been a year.. I wonder if we'll see Siri integrated into the next iOS version.. Apple bought them for a ($100M) reason..
post #3 of 36
This is one of only a couple areas I think iOS is lacking.
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #4 of 36
Quote:
Originally Posted by MarkJones View Post

The Nexus S has this feature and it's coming out in a few weeks, but I think it's not as refined as Apple's approach though.

Actually, voice commands are available in Froyo (2.2) - http://www.google.com/mobile/voice-actions/ . There were a few new ones added in 2.3 (such as setting alarms).
post #5 of 36
To post an apple fanboy style comment....


Copying android. Why do apple have to copy google? Kinda funny how when I mentioned voice actions as a feature of android lacking on iPhone the response I got was "voice commands are crap anyway"

Shape of things to come I guess: apple playing catch up to google.
post #6 of 36
Quote:
Originally Posted by tjw View Post

To post an apple fanboy style comment....


Copying android. Why do apple have to copy google? Kinda funny how when I mentioned voice actions as a feature of android lacking on iPhone the response I got was "voice commands are crap anyway"

Shape of things to come I guess: apple playing catch up to google.

So far, all voice commands I’ve used on phones have been pretty bad. iOS has some okay aspects, but it too is severely lacking so when someone states that a feature is “crap” that is not the same as saying it will always be “crap” and no one should ever try to make it better.
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #7 of 36
Quote:
Originally Posted by tjw View Post

To post an apple fanboy style comment....


Copying android. Why do apple have to copy google? Kinda funny how when I mentioned voice actions as a feature of android lacking on iPhone the response I got was "voice commands are crap anyway"

Shape of things to come I guess: apple playing catch up to google.

Mac OS X has had voice commands since the early 1990s.
post #8 of 36
Quote:
Originally Posted by Johnny Mozzarella View Post

Mac OS X has had voice commands since the early 1990s.

yes, a different type of voice commands that are interpreted with a shitty piece of local software and a dictionary.

Google are the first people to make voice commands that are actually useful by sending recordings to be processed in the cloud by cutting edge NLP algorithms, something that is no where near possible on a local device.
post #9 of 36
Google discovered the same thing, I think, trying to voice index their videos. If you try to do General speech recog, it is sh*t. But a limited vocab, a limited domain, today's tech is up to it. They have found they can index news broadcasts alone quite efficiently. In the case of voice dialing there is a limited number of people in your address book so the computer can do it.

Maybe in future voice recog will be the crunchies, but for now we must use it where we reliably can, and no further.
post #10 of 36
Quote:
Originally Posted by tjw View Post

Google are the first people to make voice commands that are actually useful by sending recordings to be processed in the cloud by cutting edge NLP algorithms, something that is no where near possible on a local device.

How is that a “voice command”?
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #11 of 36
Quote:
Originally Posted by solipsism View Post

How is that a voice command?

Watch this video for a demo - http://www.youtube.com/watch?v=gGbYVvU0Z5s

The phone records your audio, sends it to Google for processing, and receives the processed data. The phone then takes the appropriate action, such as sending an SMS, emailing, calling, opening a map, etc. It's pretty slick. I use this feature all the time on my Android phone.
post #12 of 36
Quote:
Originally Posted by derekmorr View Post

I use this feature all the time on my Android phone.

Same here. Very handy when driving. Could never send an SMS before while driving. Now it's one long press and speak.

It'll be interesting to see how Apple implements voice commands. I'm excited to see how they do it. And hopefully, they'll push the tech forward.
post #13 of 36
Quote:
Originally Posted by solipsism View Post

How is that a voice command?

How is where the command is processed relevant? Why does it matter if the processing is done on the device or on the cloud? FWIW, I've found Google's implementation much better than anything else I've tried (including the iPhone) and I can only surmise that this is because of the power of the cloud. Kudos to Apple though if they can pull off the same capability on-device.
post #14 of 36
Quote:
Originally Posted by Jetz View Post

It'll be interesting to see how Apple implements voice commands. I'm excited to see how they do it. And hopefully, they'll push the tech forward.

Apple has voice control in iOS - http://www.apple.com/iphone/features/voice-control.html

Voice Control supports fewer commands, but many more languages. Sadly, Voice Actions on Android only support U.S. English.
post #15 of 36
I've tried doing some of the more obvious voice interactions with my iphone - sending an SMS while driving, composing an email, etc. It's tedious at best and still involves a LOT of application switching and typing; not something I recommend while driving.

I have yet to try any of the pay services that allow me to do these, but the more I think about it (and the more I find myself itching to use my iPhone while driving...) the more I think it might be worth it.

I hope this is the "next killer ap" for the idevices and I hope it's soon.

What is everyone's experiences using the paid apps that do SMS and email via voice please?
post #16 of 36
Quote:
Originally Posted by tjw View Post

yes, a different type of voice commands that are interpreted with a shitty piece of local software and a dictionary.

Google are the first people to make voice commands that are actually useful by sending recordings to be processed in the cloud by cutting edge NLP algorithms, something that is no where near possible on a local device.

...Your cloud is not available. Clouding is nice as long as you have robust access, marginalized connectivity or interruptions in service immediately disable your lovely command controls. Google is set-up to provide fairly robust cloud services and we will see more capacity as they build out support for the Chrome OS, but ask any BB user what happens when the cloud is offline and you get the drift. Or you forget to pay your cell bill, or a hurricane takes out a coverage area - how much do you want to have to rely on cloud when there are so many vulnerabilities? How about someone deciding a DDOS is in the works because they're pissed about something Google has done? The network infrastructure serving public use is not as robust as you need for reliable voice control to depend on. Now if you setup a tiered structure where simple commands are locally resident and more complex controls are clouded, then you at least retain some level of serviceability. And no Google are NOT the first people to make voice commands that are actually useful - please leave the hyperbole for the actual Android/Google fora, not here.
If you are going to insist on being an ass, at least demonstrate the intelligence to be a smart one
Reply
If you are going to insist on being an ass, at least demonstrate the intelligence to be a smart one
Reply
post #17 of 36
Android has excellent voice reconition now. It is certianly ahead of iOS on this feature. I use the text and email feature all of the time...

Also, Being able to press the mic button on the phone's keypad anytime you want to enter text is a sweet feature....

I hope iOS does at least as good of a job on this....
post #18 of 36
Quote:
Originally Posted by sranger View Post

Android has excellent voice reconition now. It is certianly ahead of iOS on this feature. I use the text and email feature all of the time...


A monkey operating my phone would be better than the IOS voice commands. They have about a 5% success rate for me. It's so bad, I downloaded a tweak to remove them. Voice commands are the biggest failure I've seen apple make since allowing clones. Horrible, absolutely horrible.

Sheldon
post #19 of 36
Quote:
Originally Posted by tjw View Post

yes, a different type of voice commands that are interpreted with a shitty piece of local software and a dictionary.

Google are the first people to make voice commands that are actually useful by sending recordings to be processed in the cloud by cutting edge NLP algorithms, something that is no where near possible on a local device.

My point wasn't who is doing it better. That will continue to change every year for the foreseeable future.

My point was that Apple has been doing it since Andy Rubin was in diapers.
Apple has a long history of innovation.
Google has a short history of acquisition.
post #20 of 36
Quote:
Originally Posted by tjw View Post

...
Shape of things to come I guess: apple playing catch up to google.

Mac OS had speech recognition long before Google even existed.

Android is doomed because the Oracle lawsuit has merit and there is legal precedent. Clear violation of the Java license agreement by using a clone (Dalvik JVM) of the real JVM. If you're not 100% compliant, you're in violation. Period.

Microsoft was forced to pay Sun $20 million as the result of a similar Java breach of contract lawsuit. And if you're hoping that Larry Ellison will settle out of court and allow Google to continue to ship a non-compliant version of Java on Android, think again. The language of the Oracle suit requires all Android software to be "impounded and destroyed."

Maybe that's why Google is hedging its bets with Chrome OS. Because they know Android is doomed.

Sent from my iPhone Simulator

Reply

Sent from my iPhone Simulator

Reply
post #21 of 36
Quote:
Originally Posted by fecklesstechguy View Post

...Your cloud is not available. Clouding is nice as long as you have robust access, marginalized connectivity or interruptions in service immediately disable your lovely command controls. Google is set-up to provide fairly robust cloud services and we will see more capacity as they build out support for the Chrome OS, but ask any BB user what happens when the cloud is offline and you get the drift.

The most recent update to the Android Voice Search app (just released yesterday) had further improvements for EDGE and other high-latency, low-bandwidth connections.

Further, if the network is down, I suspect the user is going to be more upset about not being able to check email, news, weather, youtube, facebook, twitter, etc, rather than voice commands suddenly not working. At least I would be.
post #22 of 36
Quote:
Originally Posted by SockRolid View Post

Mac OS had speech recognition long before Google even existed.

The issue is not who had a feature first. Google has done a better job integrating voice input into Android than Apple has for iOS. On Android, any text box (except password fields) in any app can receive voice input. This currently isn't the case on iOS. It's possible that that will change in the future, but for now, Android has better integration.
post #23 of 36
I would simply like the existing command set to be expanded.

Additions I'd love and that should be simple:
"Redial" -- this would be awesome when dealing with the crappy AT&T network
"Call last caller" -- analogue to above, works on last incoming call
"Time" -- tell me the time
"Artist playing" -- tell me info on the song that's playing

All of these would prevent me having to reach into my bag/holster to get the phone to find out that information, and tie in with only Apple's apps (iPod/Phone).

If they really want to expand voice controls, they could allow voice to do things like:
"Open App _____ " -- launch a given app
"Text ____ " -- send SMS to a given contact (using mobile/iPhone number) - copy of Android functionality
...

I'd be happy with the first list.
post #24 of 36
Quote:
Originally Posted by SockRolid View Post

Maybe that's why Google is hedging its bets with Chrome OS. Because they know Android is doomed.

No, Android is the front against Apple.
Chrome (both browser and OS) is the front against Microsoft.

Both are disruptive plays against their biggest competitors.

The Android suit will go on for a decade or more, in the meanwhile the market will decide if Android is worth it, or not (I'm sure the carriers love it compared to even WP7). I think Android is far from doomed, but it may not be the world domination as previously forecasted from the Googleplex.
post #25 of 36
Well, one of them currently exists:
Ask your iPhone "What time is it?"

You should be pleasantly surprised.
Merry Christmas.
:d


Quote:
Originally Posted by r00fus View Post

I would simply like the existing command set to be expanded.

Additions I'd love and that should be simple:
"Redial" -- this would be awesome when dealing with the crappy AT&T network
"Call last caller" -- analogue to above, works on last incoming call
"Time" -- tell me the time
"Artist playing" -- tell me info on the song that's playing

All of these would prevent me having to reach into my bag/holster to get the phone to find out that information, and tie in with only Apple's apps (iPod/Phone).

If they really want to expand voice controls, they could allow voice to do things like:
"Open App _____ " -- launch a given app
"Text ____ " -- send SMS to a given contact (using mobile/iPhone number) - copy of Android functionality
...

I'd be happy with the first list.
post #26 of 36
Quote:
Originally Posted by tjw View Post

yes, a different type of voice commands that are interpreted with a shitty piece of local software and a dictionary.

Google are the first people to make voice commands that are actually useful by sending recordings to be processed in the cloud by cutting edge NLP algorithms, something that is no where near possible on a local device.

my wm5 was working with voice commands very well years before android even existed and without having to go to the cloud.

I'll be the first to admit the iOS commands are crap, and googles are pretty amazing by comparison. This comparison is completely inverted however once your network drops out, or you are abroad and haven't got data so readily available.

The purchase of Siri is for the AI only IFAIK and the speech recognition is supplied by Nuance, and i'm not sure they were brought as part of the deal. Even if they were .. this is now a cloud based solution which fails in low signal / foreign countries situations.

As far as I'm concerned they both have major shortcomings until you have googles accuracy using local resources.

this could help tho :
http://www.newscientist.com/article/...-language.html
post #27 of 36
Quote:
Originally Posted by t2af View Post

my wm5 was working with voice commands very well years before android even existed and without having to go to the cloud.

I'll be the first to admit the iOS commands are crap, and googles are pretty amazing by comparison. This comparison is completely inverted however once your network drops out, or you are abroad and haven't got data so readily available.

The purchase of Siri is for the AI only IFAIK and the speech recognition is supplied by Nuance, and i'm not sure they were brought as part of the deal. Even if they were .. this is now a cloud based solution which fails in low signal / foreign countries situations.

As far as I'm concerned they both have major shortcomings until you have googles accuracy using local resources.

this could help tho :
http://www.newscientist.com/article/...-language.html

Yes but this is an invalid argument since your smart phone is not 'smart' when you don't have a data connection anyway. You are not going to send an email or use navigation that requires data in those cases anyway, whether you have voice commands or not.
post #28 of 36
Quote:
Originally Posted by SockRolid View Post

Mac OS had speech recognition long before Google even existed.

Android is doomed because the Oracle lawsuit has merit and there is legal precedent. Clear violation of the Java license agreement by using a clone (Dalvik JVM) of the real JVM. If you're not 100% compliant, you're in violation. Period.

Microsoft was forced to pay Sun $20 million as the result of a similar Java breach of contract lawsuit. And if you're hoping that Larry Ellison will settle out of court and allow Google to continue to ship a non-compliant version of Java on Android, think again. The language of the Oracle suit requires all Android software to be "impounded and destroyed."

Maybe that's why Google is hedging its bets with Chrome OS. Because they know Android is doomed.

native (read crappy) voice actions compared with googles method are completely different. The later is so much better it is a new product. It's like saying the person who invented the film camera should be credited with inventing the digital camera. Face it google are light years ahead of apple on this. Apple don't even have the infrastructure to pull it off.

I have seen the details of the law suit but are you really stupid enough to think larry ellison, the famous competitive billionaire, would pay a huge amount of money in a law suit for no financial return - just to kill android.... You're pretty stupid if you believe this will kill android. It will end with a settlement of $millions for oracle to pocket. I don't think he is as deranged as some of you fan boys - i.e would happily pay a fortune just to close something down because you don't like it. His motivations are financial and a dislike of Eric Schmidt.
post #29 of 36
Quote:
Originally Posted by tjw View Post

Yes but this is an invalid argument since your smart phone is not 'smart' when you don't have a data connection anyway. You are not going to send an email or use navigation that requires data in those cases anyway, whether you have voice commands or not.

I disagree. I think t2af is right. Android's voice actions do have some lag. That's one issue that could be resolved by local processing. At the same time I doubt you could get Google's accuracy or the ability to do this in several languages accurately (given the pledge to provide this service in many more languages and given their focus on speech recognition and speech synthesis this won't be hard to pull off) without the cloud.
post #30 of 36
Quote:
Originally Posted by nbcbubba View Post

What is everyone's experiences using the paid apps that do SMS and email via voice please?

Works decently on Android. There are times it'll give you a false reading of your speech. But all in all for small day to day stuff, both voice actions and general text entry (Android enables speech to text for every textbox), works quite well. I rarely type long emails or SMS anymore. Anything more than the most basic sentence I do by speech. Same for any short SMS when I'm somewhat pre-occupied (like when driving).
post #31 of 36
Quote:
Originally Posted by fecklesstechguy View Post

...Your cloud is not available. Clouding is nice as long as you have robust access, marginalized connectivity or interruptions in service immediately disable your lovely command controls. Google is set-up to provide fairly robust cloud services and we will see more capacity as they build out support for the Chrome OS, but ask any BB user what happens when the cloud is offline and you get the drift. Or you forget to pay your cell bill, or a hurricane takes out a coverage area - how much do you want to have to rely on cloud when there are so many vulnerabilities? How about someone deciding a DDOS is in the works because they're pissed about something Google has done? The network infrastructure serving public use is not as robust as you need for reliable voice control to depend on. Now if you setup a tiered structure where simple commands are locally resident and more complex controls are clouded, then you at least retain some level of serviceability. And no Google are NOT the first people to make voice commands that are actually useful - please leave the hyperbole for the actual Android/Google fora, not here.


I seriously doubt you'll be worried about not having voice commands on your phone if a Hurricane knocks out cellular coverage. And if you forget to pay your bill, how useful would your iPhone be anyway (even if it has voice commands processess natively). And then there's DDOS attacks against Google? Really? They survived the trained hackers of the Chinese government. I'll take my chances that they can protect a server that does voice commands. Talk about FUD....unpaid bills, hurricanes, DDOS attacks....when you going to bring up the locusts and the taking of the first-born?

As for who did it first....who gives a rat's ass? Do you care whether Apple was the first to have a candybar touchscreen phone? Maybe you only buy Ford because Henry Ford made the first mass produced automobile? First does not mean best.
post #32 of 36
Quote:
Originally Posted by AppleInsider View Post

Apple could improve voice commands on the iPhone by making them contextualized, asking users to select, by hand, an application before dictating commands aloud.


They might want to make Voice Command actually work before they make it contextualized.

Just a suggestion.
post #33 of 36
Quote:
Originally Posted by tjw View Post

Yes but this is an invalid argument since your smart phone is not 'smart' when you don't have a data connection anyway. You are not going to send an email or use navigation that requires data in those cases anyway, whether you have voice commands or not.

calling someone using voice control
adding items to a todo list

just 2 that come to mind that have affected me recently.
post #34 of 36
Quote:
Originally Posted by tjw View Post

Shape of things to come I guess: apple playing catch up to google.


Google also beat Apple to the punch with NFC. We might get that with the iP5, but I wouldn't hold my breath. It was rumored to be part of iP4, and we know that that did not happen.

With cool new Google Phones coming out one after another, it will be very difficult for the iPhone to stay ahead of the pack. Impossible, really.

But the iPhone has never been about having every available feature. The iPhone has a great user experience, even if it is not able to boast the tech in the Google Phones. It is easy to use, and that is the only thing that really matters.
post #35 of 36
Quote:
Originally Posted by solipsism View Post

How is that a voice command?

The user employs his voice to issue commands.
post #36 of 36
Quote:
Originally Posted by Jetz View Post

It'll be interesting to see how Apple implements voice commands.

They already have. It is a standard feature on the iPhone. It remains unused due to poor performance.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
AppleInsider › Forums › Mobile › iPhone › Apple exploring improved, contextual voice commands for iPhone