or Connect
AppleInsider › Forums › Mobile › iPhone › Investigation into Siri reveals how the iPhone 4S service talks to Apple
New Posts  All Forums:Forum Nav:

Investigation into Siri reveals how the iPhone 4S service talks to Apple

post #1 of 32
Thread Starter 
A mobile developer investigating Apple's Siri voice assistant has dissected the service's protocol to develop tools for playing with the service outside of the iPhone 4S.

Testing by Applidium notes that iPhone 4S uses standard HTTPS network requests to communicate with Apple's servers, but sends data using an "ACE" command rather than regular web GET requests.

Each Siri request an iPhone 4S makes also involves a unique host identifier that appears to be based on the hardware UUID, preventing unauthorized devices from sending requests to Apple's servers.

Applidium reports some success in copying an iPhone 4S host identifier into requests sent from other devices, including a test Mac environment. By examining how iPhone 4S packages speech recognition requests, the developer was able to send a similarly packaged request and obtain a correct response.

The testing proves that Siri sends raw audio captures of the user's voice, compressed with the Speex audio codec optimized for VoIP. Previously, it had been speculated that iPhone 4S was performing preprocessing of the audio and sending only the results to Apple's servers.

While Siri may perform other preprocessing tasks that use the additional horsepower of the iPhone 4S, Applidium's discovery indicates that any iPhone should be able to support at least Siri's basic voice recognition features, although Apple has indicated that it has no plans to release such capabilities for earlier iOS 5 models, including iPhone 4 and iPhone 3GS.

Both models can support third party speech-to-text services, but Apple offers no way in iOS to integrate such third party services into any app system wide, meaning that users have to dictate into one app and then copy and paste the results elsewhere. Both Android and Windows Phone 7 offer system wide, integrated voice recognition features.

Siri, however, goes far beyond simple voice recognition. Rather than just converting audio to text, Siri evaluates the meaning of requests and maintains a understanding of the user's relationships with specific contacts and a contextual session of the location and other details of a request.



So far, Applidium's investigation has revealed that Siri packages requests in compressed property lists, but further exploration of the protocol is hampered by a number of issues, including the complexity of requests, the fact that they are tied to a hardware key, and that they are subject to change.

Apple could at any time stop responding to a given hardware key were it to suspect that it was being used to exploit its servers; additionally, because the Siri service is proprietary to Apple, the company can change how it transmits data rather quickly by simply sending out a client update.

Applidium says "anyone could now write an Android app that uses the real Siri! Or use Siri on an iPad!" However, in order to access Siri at all, a user would have to sniff out the unique user key of an actual iPhone 4S, and then reuse that key until it expired or was blocked by Apple.

Apple has been perfecting its Siri service as a "beta" feature exclusive to the iPhone 4S, but has experience some downtime in ramping up services to accommodate the demands of millions of users who have rushed to buy the latest iPhone model. The new service also makes use of hardware unique to the iPhone 4S.

It is expected that Siri will eventually find its way to new models of the iPod touch, iPad and perhaps even Macs, with some speculating that it could eventually server as a living room interface employed by Apple TV, doing away with the need for a button or touch-based remote control for navigating television programing.
post #2 of 32
Siri can work with every other network-based device in the world yet Apple only allows the iPhone 4S to use it¡ OMG¡ OMG¡
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #3 of 32
Quote:
Originally Posted by solipsism View Post

Siri can work with every other network-based device in the world yet Apple only allows the iPhone 4S to use it¡ OMG¡ OMG¡

Watch Apple have to shut the entire service down from the load this reverse engineering could potentially cause, as the servers were designed to handle the amount of 4S' sold.
post #4 of 32
it sounds like IBM's watson...
post #5 of 32
Quote:
Originally Posted by mbarriault View Post

Watch Apple have to shut the entire service down from the load this reverse engineering could potentially cause, as the servers were designed to handle the amount of 4S' sold.

Yes, what a great public service this nerd is providing. Why can't people just let it be?
A.k.a. AppleHead on other forums.
Reply
A.k.a. AppleHead on other forums.
Reply
post #6 of 32
By limiting Siri to just iPhone 4S, Apple can test it with a smaller sample of users. As it happened they experienced a service outage. Can you imagine how quickly it would have melted down if they had let iPhone 4 and 3GS users access it too?
post #7 of 32
Quote:
Originally Posted by mbarriault View Post

Watch Apple have to shut the entire service down from the load this reverse engineering could potentially cause, as the servers were designed to handle the amount of 4S' sold.

Why? Apple allready has a filter for valid devices. If they believe a device is invalid they can either blacklist it or do some other identification. My guess is that they will blacklist and use legal tactics to make sure this doesn't last.
post #8 of 32
Quote:
Originally Posted by Robin Huber View Post

Yes, what a great public service this nerd is providing. Why can't people just let it be?

Why can't I be curious on how Siri does work? As an avid programmer, I love to just expand my knowledge of technology and I especially love to see how Apple's technology for iOS works, including Siri; there's no harm in knowing.

Besides, this information is pretty basic, just because it was explained how it works doesn't mean that there are a plethora of people at the gates with this knowledge now able to mimic Siri requests. Nothing will really change.
post #9 of 32
Quote:
Originally Posted by mbarriault View Post

Watch Apple have to shut the entire service down from the load this reverse engineering could potentially cause, as the servers were designed to handle the amount of 4S' sold.

Nah... Apple will just send out a patch that lets Siri know whether to answer an iPhone 4s device in the normal Siri voice or if it is not an Apple device, code Siri to respond in that generic male computer voice "Does not compute! Does not compute!"

Oh, and anyone that tinkers with their none Apple iPhone 4s by fooling Siri into thinking it's an iPhone 4s, well I heard Apple legal just got a box of 1,000,000,000 business cards...

Hello, this is Apple legal calling!
/
/
/

Ten years ago, we had Steve Jobs, Bob Hope and Johnny Cash.  Today we have no Jobs, no Hope and no Cash.

Reply

Ten years ago, we had Steve Jobs, Bob Hope and Johnny Cash.  Today we have no Jobs, no Hope and no Cash.

Reply
post #10 of 32
Siri should be on iPod nano. That would totally create a new market segment which Apple's competitors cannot even match. Imagine being able to place and receive calls on a watch-sized nano. Make appointments. Have it read your email etc. Track your fitness like that Jawbone UP thing. And sell it for $299.
post #11 of 32
Quote:
Originally Posted by brisance View Post

Siri should be on iPod nano.

This can't happen until storage is large enough cheaply enough to house all of the server-side software on every device.

Quote:
Imagine being able to place and receive calls on a watch-sized nano.

Oh, so not an iPod at all, an iPhone with a 1" screen. Somehow I don't think that'll work out.

Quote:
And sell it for $299.

Battery life: 30 minutes!

Originally Posted by Slurpy

There's just a TINY chance that Apple will also be able to figure out payments. Oh wait, they did already… …and you’re already fucked.

 

Reply

Originally Posted by Slurpy

There's just a TINY chance that Apple will also be able to figure out payments. Oh wait, they did already… …and you’re already fucked.

 

Reply
post #12 of 32
Quote:
Originally Posted by Tallest Skil View Post

Battery life: 30 minutes!

It can go two ways. You can have these newfangled new concepts fairly soon with no realistic battery life, or have them with realistic battery life in some future that makes the whole idea too distant to be worth conceiving. It's like me saying I want a tablet that is thinner and light with 10x the battery life. I'm sure that will happen, but not for a very, very long time.
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #13 of 32
Here's a video explanation of Siri I found on YouTube.

http://www.youtube.com/watch?v=C_r-30oLdH0

Interesting.

"The power of this technology is NOT in voice recognition - it is being able to UNDERSTAND complex statements and deduce MEANING."
post #14 of 32
Quote:
Originally Posted by Tallest Skil View Post

This can't happen until storage is large enough cheaply enough to house all of the server-side software on every device.

Not to mention that the Nano has no network connectivity.
post #15 of 32
Quote:
Originally Posted by Robin Huber View Post

Yes, what a great public service this nerd is providing. Why can't people just let it be?

Heres to the crazy ones. The misfits. The rebels. The troublemakers. The round pegs in the square holes.
post #16 of 32
Quote:
Originally Posted by Tommy Wasabi View Post

Here's a video explanation of Siri I found on YouTube.

http://www.youtube.com/watch?v=C_r-30oLdH0

Interesting.

"The power of this technology is NOT in voice recognition - it is being able to UNDERSTAND complex statements and deduce MEANING."

Exactly!

The speech recognition is provided by Nuance and although complex and challenging, speech recognition is fairly well understood. Siri provides natural language processing with contextual, semantic and syntactical awareness. Siri is capable of far more than the functionality currently available on iPhone 4S. Essentially, Siri could integrate with any number and variety of products and services to extend the capabilities to order movie tickets, restaurant reservations or requesting taxis. In fact, Siri was previously capable of doing all those activities.
post #17 of 32
Quote:
Originally Posted by TokyoJimu View Post

Not to mention that the Nano has no network connectivity.

That's EXACTLY what the other poster WAS mentioning.

No network connectivity => all processing happens locally => local storage and processing power must increase dramatically.

Thompson
post #18 of 32
I can't picture anyone having had such a knee-jerk, this-is-bad-for-Apple, don't-do-it, Apple-Legal-will-find-you, blacklist-the-bastards attitude toward finding out how Apple technologies work and incorporating them in useful ways into their daily lives, back in the mid '90s when I started using Macs.

Nowadays, these forums are full of people who, whenever someonewhether a developer like this, or another company, or a security researcherfinds something interesting, something novel, something reverse-engineered, something flawedare quick to shut it down, declare it out of bounds, say it's useless and call for Apple to block it.

People, this is technology! It's ones and zeroes. Examine them, decode them, rebuild them, restructure them, bask in their ingenuity! A savvy programmer tinkers with Siri and gets it working on another device, figures out how requests are sent, decodes the authentication mechanism and finds a workaround. This is brilliant, and it illuminates the way the technology we love works.

Maybe it's just your desire to preserve the aura of magic that Jobs so cunningly cultivated, but when you call for Apple Legal to break out the blocklists (as if a tiny spike in traffic on one iPhone 4S UUID would even attract their notice), you're not helping Apple, you're hurting the kind of people who get excited about its technologies and want to develop them further, even maybe if they have different ideas about how they are to be used than those in Apple's engineering department.

Apple (and mainly Jobs) created this mindset where things are locked down, where everything is packaged so neatly and cleanly and does what it's supposed to do well that many of the Mac faithful have been sucked into believing that Apple's way is the only way. This used to be the platform that called for "thinking differently" but in some Apple fans' calls to follow Apple in every situation, I think we run the risk of becoming merely sheep. Now that Tim Cook is in charge, hopefully an environment more open to free-thinking can be cultivated.

I don't mean any personal offense with this post, and I'm not calling out anyone in particular, but sometimes the lack of imagination and the rush to defense makes me sad. This coming from someone who's been using Mac since elementary school (1994).
post #19 of 32
There is always some one out there who will want to break the code or hack the data to get it functioning on something other than what it was extended. Yes it would be nice to see it on my iPhone 4. No I won't be trying to hack my iPhone 4 to get Siri to work on it. I will wait till my upgrade option is available and but an iPhone 4S or 5 to get the new Siri.
An Apple man since 1977
Reply
An Apple man since 1977
Reply
post #20 of 32
Quote:
Originally Posted by brisance View Post

Siri should be on iPod nano. That would totally create a new market segment which Apple's competitors cannot even match. Imagine being able to place and receive calls on a watch-sized nano. Make appointments. Have it read your email etc. Track your fitness like that Jawbone UP thing. And sell it for $299.

And it could be clipped on as a watch. It will of course take another round of miniaturizing to fit in bluetooth, WiFi, GSM and GPS, but it will be the ultimate communication device.

J.
post #21 of 32
Quote:
Originally Posted by thompr View Post

That's EXACTLY what the other poster WAS mentioning.

No network connectivity => all processing happens locally => local storage and processing power must increase dramatically.

Thompson

Once upon a time, the iPod was the size of a deck of cards, held 5GB, had a click wheel, no video etc.

In 1997, a roomful of 10,000 Pentium Pros drew 500 KW of power and required another 500 KW just for cooling. Ten years later, Intel had got that down to the size of a fingernail drawing just 62W.

I don't think it's that far-fetched. All the nano needs is network connectivity. Without a screen, battery demands are far less.
post #22 of 32
Quote:
Originally Posted by djsherly View Post

Heres to the crazy ones. The misfits. The rebels. The troublemakers. The round pegs in the square holes.

Didn't notice thieves in your list. Nor was Assange one of the faces of the greats in Apple's famous commercial.
A.k.a. AppleHead on other forums.
Reply
A.k.a. AppleHead on other forums.
Reply
post #23 of 32
Quote:
Originally Posted by jnjnjn View Post

And it could be clipped on as a watch. It will of course take another round of miniaturizing to fit in bluetooth, WiFi, GSM and GPS, but it will be the ultimate communication device.

It may be a lot closer than you realize. There's an Android-powered "watch" module available now that includes "a touchscreen, Wi-Fi and Bluetooth connectivity, accelerometer, magnetometer, creative alerts, and a 14-pin connector. . . ."

http://www.androidauthority.com/andr...s-do-it-26014/
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #24 of 32
You know they've hit upon something revolutionary when the entire world seems obsessed in how it works and how they can circumvent the technology behind it.
post #25 of 32
Quote:
Originally Posted by Robin Huber View Post

Didn't notice thieves in your list. Nor was Assange one of the faces of the greats in Apple's famous commercial.

Blue box? How do you think apples founders made their start in life? And I don't know why you're bringing Assange into this?
post #26 of 32
Quote:
Originally Posted by djsherly View Post

Blue box? How do you think apples founders made their start in life? And I don't know why you're bringing Assange into this?

Ha. The free long distance gizmos. Good point. And now Apple is way bigger than those telephone companies back then.
post #27 of 32
Quote:
Originally Posted by Nobodyy View Post

Why can't I be curious on how Siri does work? As an avid programmer, I love to just expand my knowledge of technology and I especially love to see how Apple's technology for iOS works, including Siri; there's no harm in knowing.
.

Because you're supposed to be a blind sheep and just accept things for what they are, er, for what you're told.
post #28 of 32
Another developer has figured out how to port Siri to any device, fooling it into thinking it's communicated thru Apple servers when instead it's a proxy. Good stuff.

http://9to5mac.com/2011/11/21/develo...vice-via-siri/
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #29 of 32
Quote:
Originally Posted by Gatorguy View Post

Another developer has figured out how to port Siri to any device, fooling it into thinking it's communicated thru Apple servers when instead it's a proxy. Good stuff.

http://9to5mac.com/2011/11/21/develo...vice-via-siri/

You really think Siri being accessible to a billion or more possible devices is a good thing a month after its launch as a Beta? At best, we have Apple putting more checks into Siri on the server and a more elaborate handshake that will slow down its ability to communicate to the user.

You really want the entire system to less functional? That just makes you sound like an anti-Apple fanatic that wants to see Apple fail. Why not instead hope for other companies to build their own version of Siri that will push Apple and this technology ahead the way the iPhone has revamped the smartphone industry in just a few years?
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #30 of 32
Quote:
Originally Posted by solipsism View Post

You really think Siri being accessible to a billion or more possible devices is a good thing a month after its launch as a Beta? At best, we have Apple putting more checks into Siri on the server and a more elaborate handshake that will slow down its ability to communicate to the user.

You really want the entire system to less functional? That just makes you sound like an anti-Apple fanatic that wants to see Apple fail. Why not instead hope for other companies to build their own version of Siri that will push Apple and this technology ahead the way the iPhone has revamped the smartphone industry in just a few years?

By "good stuff" I meant the article and explanation of how he did it. You're absolutely correct that from Apple's viewpoint it's definitely not a good thing, and depending on how Apple tweaks authentication it could impact legitimate users too.

As far as other companies working on voice control, I thought I read that Amazon, like Apple, recently made a similar tech purchase and Google was already rumored months ago to be finalizing their extended voice-search service, expected in the spring/2012.
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #31 of 32
Quote:
Originally Posted by Gatorguy View Post

As far as other companies working on voice control, I thought I read that Amazon, like Apple, recently made a similar tech purchase and Google was already rumored months ago to be finalizing their extended voice-search service, expected in the spring/2012.

A couple things to consider Siri was founded in 2007 and Apple bought Siri in 2010. On top of that, the foundations of Siri come from a long history of AI development.
With Siri, Apple is using the results of over 40 years of research funded by DARPA via SRI Internationals Artificial Intelligence Center through the Personalized Assistant that Learns Program and Cognitive Agent that Learns and Organizes Program CALO.

This includes the combined work from research teams from Carnegie Mellon University, the University of Massachusetts, the University of Rochester, the Institute for Human and Machine Cognition, Oregon State University, the University of Southern California, and Stanford University. This technology has come a long way with dialog and natural language understanding, machine learning, evidential and probabilistic reasoning, ontology and knowledge representation, planning, reasoning and service delegation
It's easier to just quote the Wiki paragraph but there is a footnote link to a more indepth source for those that wish to delve deeper.

I can't find much about Yap so I assume it doesn't have the same pedigree as Siri and potentially is much more rudimentary in nature.

»

http://www.forbes.com/sites/mobiledi...te-with-apple/ What Google has is a lot of data mining, but I don't know if they have yet been able to build out the natural language support. As usual, the competition is usually 5 years behind Apple, though I'm sure you'll disagree with that.
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #32 of 32
Quote:
Originally Posted by solipsism View Post

. . .What Google has is a lot of data mining, but I don't know if they have yet been able to build out the natural language support. As usual, the competition is usually 5 years behind Apple, though I'm sure you'll disagree with that.

No, not necessarily disagreeing with the basic premise that Apple is an innovation leader. They certainly are. In the case of Google's upcoming natural voice search, I'd imagine it's homegrown rather than purchased (Siri/Yap), same as their current voice search and translation services. Google's been using voice search (and control for that matter) for some time, since at least 2008. In fact some of the features were pinned in the old Google 411 service back in early 2007, in part used for gathering voice samples for their upcoming voice control in 2008. So Google obviously had voice search plans underway prior to Siri entering the picture in 2007.
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
  • Investigation into Siri reveals how the iPhone 4S service talks to Apple
AppleInsider › Forums › Mobile › iPhone › Investigation into Siri reveals how the iPhone 4S service talks to Apple