or Connect
AppleInsider › Forums › Mobile › iPhone › Street test measures Siri comprehension at 83%, accuracy at 62%
New Posts  All Forums:Forum Nav:

Street test measures Siri comprehension at 83%, accuracy at 62% - Page 3

post #81 of 141
Quote:
Originally Posted by Rogifan View Post


How long is a service allows to be called beta? As long as its not working well we just call it beta to excuse away its problems.

Or as long Apple hasn't find a way to monetize it, that can scale! Here is my idea: publish an api so websites can be listed in the Siri service. I looked at the wolfram service and basically they won't allow anything coming from outside.

post #82 of 141
Quote:
Originally Posted by Gatorguy View Post

 

He doesn't pronounce it fur'ners??  Just kidding... lol.gif

 

Well technically speaking, I'll be a foreigner soon too. All of the Fandroids on this forum will be relieved to know that I'm going away for the summer and probably won't get a chance to post much at all, because I'll be in a real remote place with very limited internet access. I will be back though.

post #83 of 141
Quote:
Originally Posted by markbyrn View Post

I guess Siri will be in beta for years to come.

 

lsqgia26.jpg

 

Like Gmail?

 

Siri has only been out for 8 months... Gmail, an email service was out for years before it went off beta. Email was a tried and true technology for over 20 years at the time!!! Why in the hell was it ever in beta in the first place?

 

Siri is a learning technology, meaning it needs to learn your voice and enunciation. Sorry do you not know what that means? It needs to learn how you talk, slurs and all.

 

I think int he past 7 months since I've had my iPhone, Siri has dissed me a handful of times (maybe four or five).

Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
post #84 of 141
Quote:
Originally Posted by Hellacool View Post

When I was in highschool many moons ago, understanding the teacher 86% of the time and only getting 62% of my answer's right was a big fat F, of course I just wrote BETA on the top of my tests and all was forgiven.

 

Software has many stages for testing purposes.

 

There's alpha, meaning very early.

There's beta, meaning in a test phase.

Then there's gold master, meaning about to be released.

 

 

Since Siri is a social/learning feature it needs to be tested on a mass scale before it can be really be let loose.

Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
post #85 of 141
Quote:
Originally Posted by Apple ][ View Post

 

Well technically speaking, I'll be a foreigner soon too. All of the Fandroids on this forum will be relieved to know that I'm going away for the summer and probably won't get a chance to post much at all, because I'll be in a real remote place with very limited internet access. I will be back though.

Good for you! Work or play?

melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #86 of 141
Quote:
Originally Posted by mjtomlin View Post

 

Like Gmail?

 

Siri has only been out for 8 months... Gmail, an email service was out for years before it went off beta. Email was a tried and true technology for over 20 years at the time!!! Why in the hell was it ever in beta in the first place?

 

Siri is a learning technology, meaning it needs to learn your voice and enunciation. Sorry do you not know what that means? It needs to learn how you talk, slurs and all.

 

I think int he past 7 months since I've had my iPhone, Siri has dissed me a handful of times (maybe four or five).

Once Apple included Siri with all shipping iPhone 4S devices, it should no longer be called "beta". That is a final shipping product to customers.

post #87 of 141
Quote:
Originally Posted by Negafox View Post

Once Apple included Siri with all shipping iPhone 4S devices, it should no longer be called "beta". That is a final shipping product to customers.

Then you have a different definition of beta than most, I think.

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply
post #88 of 141
Quote:
Originally Posted by mjtomlin View Post

 

You're an idiot.

 

 

Software has many stages for testing purposes.

 

There's alpha, meaning very early.

There's beta, meaning in a test phase.

Then there's gold master, meaning about to be released.

 

 

Since Siri is a social/learning feature it needs to be tested on a mass scale before it can be really be let loose.


I have an iTissue if your feelings are hurt by the truth.  Advertising to the masses Siri as a main feature with no indication of beta is flat out lying.  We know it is beta because we hang out on tech forums but the majority of folks have no idea.

post #89 of 141
Quote:
Originally Posted by Tallest Skil View Post


Then you have a different definition of beta than most, I think.

I would like to hear your definition then. Beta software is generally feature complete but released as an optional download for users to test. Siri is GA, not beta, as it is a bullet feature in a retail product.

post #90 of 141
Quote:
Originally Posted by Tallest Skil View Post


Then you have a different definition of beta than most, I think.


As an uniformed consumer, one that has no idea what beta even means, a product they purchase is expected to be the final product.  So to say most, you mean most in the tech world because no way would most consumers have a clue.

post #91 of 141
Quote:
Originally Posted by Hellacool View Post

Advertising to the masses Siri as a main feature with no indication of beta is flat out lying.

Saying Apple doesn't indicate it's a beta is flat out lying, too.

216

246

I really shouldn't have to link to something as simple as Apple.com to debunk the things you say. Doing so also proves that a much larger number of people than you assume know it's a beta.

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply
post #92 of 141
Quote:
Originally Posted by Gatorguy View Post

Good for you! Work or play?

It's mostly play, but I'm hoping to get a little work done at least.

 

There will be no shortage of Apple devices there, because I'm bringing a Macbook, two iPads, Apple BT keyboard, Magic Trackpad and some recording gear with me. I'll buy some cheap 24-27" monitor when I get there, and that should be a decent set up to use while I'm away.

post #93 of 141
Quote:
Originally Posted by Hellacool View Post


Have fun in jail.

 

Nah, people like me don't end up in jail.

post #94 of 141

"Remind me to pick up eggs and milk when I leave work."

 

Siri can do this for me... can Google? No, but we didn't test that kind of thing.

post #95 of 141
Quote:
Originally Posted by Gatorguy View Post

Dude, don't start out the day failing to read before posting, particularly when it's your own link. Only in your world could your article "proving" Siri more accurate than Google Voice Actions be misconstrued as a scientific study "with appropriate controls".  The article itself plainly says: 

 

"The study is small and not scientific, therefor no quantitative conclusions should be drawn

http://blog.thearorareport.com/2011/11/14/aaple-siri-voice-recgnition-trumps-android-voice/

 

That's your link!! Please be careful today. It's a long fall off your high-horse.

 

 

The fact that he didn't realize what was said is no real evidence that he didn't read it.  Indeed, it would par for the course.

post #96 of 141
Quote:
Originally Posted by Tallest Skil View Post


Saying Apple doesn't indicate it's a beta is flat out lying, too.

I really shouldn't have to link to something as simple as Apple.com to debunk the things you say. Doing so also proves that a much larger number of people than you assume know it's a beta.

 

But Apple should, and they don't.   I know of no links on ads which do not identify it as beta (such as the celebrity video ads) which direct you to the Siri page with these graphics.  Who makes regular trips to Apple's site for info *just in case* there's more to it?  Us maybe, not most of Apple users, which makes it some pretty fine print regardless of how front and center it is on their Siri page.  Is this not the only place where it is clarified?  If not, point out where else it is, in adverts for the iPhone which mention Siri, and if it's not there point out what directs one to the Siri site for such status.   Not saying it doesn't exist, just that I've never seen it, though I've seen lots of ads, print and video, which contain no reference to this.

post #97 of 141
Quote:
Originally Posted by robogobo View Post

It's "allowed" to be called beta as long as it's in beta.  And it's still in beta because they haven't worked out the problems completely.  As soon as they're worked out, it will probably be out of beta.  Make sense?

Not really. It really means that 'he guys, we have something that's called Siri, but please, don't mind its countless errors"

post #98 of 141
Quote:
Originally Posted by jragosta View Post


 it was a scientific survey.
 
"The study is small and not scientific, therefor no quantitative conclusions should be drawn
http://blog.thearorareport.com/2011/11/14/aaple-siri-voice-recgnition-trumps-android-voice/

 

 

 

See what I mean?

post #99 of 141
Quote:
Originally Posted by Tallest Skil View Post


Saying Apple doesn't indicate it's a beta is flat out lying, too.
216
246
I really shouldn't have to link to something as simple as Apple.com to debunk the things you say. Doing so also proves that a much larger number of people than you assume know it's a beta.


Sigh, I know it is hot out and your brain may be running slow but I am referencing the televised commercials, the commercials majority see, the ones with the celebs, the ones Apple pours all its advertising dollars into, you know the ones you are ignoring.  on top of the fact that most consumers have zero idea what beta means.  Keep grasping straws, it is entertaining.

post #100 of 141
Quote:
Originally Posted by Apple ][ View Post

 

Nah, people like me don't end up in jail.


That's what all the guilty say.

post #101 of 141
Quote:
Originally Posted by Tallest Skil View Post


Saying Apple doesn't indicate it's a beta is flat out lying, too.
216
246
I really shouldn't have to link to something as simple as Apple.com to debunk the things you say. Doing so also proves that a much larger number of people than you assume know it's a beta.

Well, as a former project manager, I can tell you this: a beta is never intended for a large audience.
It's meant for a review group sending back findings on using the software.

- does Siri beta have such reporting functions?

- is it intended for a review group?

- was it launched as a feature for the whole of the intended audience from day 1?

- does it implicate that when Siri is out of beta, that it will be flawless (as current errors are so vividly attributed to a beta status?

 

Now, for me, Siri works quite well. I do believe it's slightly better than the 62% and I can live with that.

But please, don't BS me with a beta status. Never can I inform Apple why the 38% failed and what I should have had.

post #102 of 141
Quote:
Originally Posted by Blitz1 View Post

Well, as a former project manager, I can tell you this: a beta is never intended for a large audience.
It's meant for a review group sending back findings on using the software.

- does Siri beta have such reporting functions?

- is it intended for a review group?

- was it launched as a feature for the whole of the intended audience from day 1?

- does it implicate that when Siri is out of beta, that it will be flawless (as current errors are so vividly attributed to a beta status?

 

Now, for me, Siri works quite well. I do believe it's slightly better than the 62% and I can live with that.

But please, don't BS me with a beta status. Never can I inform Apple why the 38% failed and what I should have had.


Shhhh, Shhhhh no logic or facts allowed here.

post #103 of 141
Quote:
Originally Posted by JerrySwitched26 View Post

 

 

 

See what I mean?


There's nothing good to be gained by intentionally trying to stir trouble. Not a great idea if you wish to stick around longer than last time. Way too many other members post trollish comments which add nothing to the discussion but trouble, I guess erroneously thinking they're being cute or creative. Instead they come off as juvenile and petty. Don't join them.

melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #104 of 141
Quote:
Originally Posted by ankleskater View Post


Such a comparison does not make sense. There's no way it would have been published in a peer-reviewed journal. A proper test would compare apples to apples, with control built in. If it makes sense to compare accuracy of speech processing to text input, why not include sign language and smoke signals too?

 

This has nothing to do with the utility of Siri. The reported test just does not pass any meaningful scientific scrutiny.

 

I don't mind the comparison. Its basically all about comparing the info you are asking Siri and the results vs. what you would get if you inputed the same info in a Google search box. I did not read the full study so I assume that it was the Exact same info with no additional help with the use of parameters in search. If the latter, its not a fair analysis. 

 

Then there is another thing to consider. Currently, Apple has been using Google for the search. Question is are the different results because of Siri not transcribing accurately or is Google intentionally not passing the best results back to Siri? My guess is Google has to know where the searches are coming from and Google. Remember, No ads are passed, like a regular search engine so Google sees no benefit form providing results so who knows. Only way to know for sure is to compare the text output that Siri passes to Google. Something they do not see in the test. 

 

As far as the Beta tag for Siri, I have no problem with it. I think its two fold. One is that Siri has a limited set of functionality/commands. Apple doesn't want those to feel its full featured. Second, is that Siri needed to build up its vocabulary and learn speech patterns and dialects from thousands of users to increase accuracy across the board. This also goes to point 1. How frustrating would it be to have a wider range of commands and features, which will be present in iOS6, then to get poor results and apps not launching. My guess is that if someone studied the current functions of Siri and the vocabulary it can respond to, you will probably find that those utilize required verbiage to quickly build out better speech recognition and nuances in speech.  I am also guessing that while Siri still might show Beta, the commercials may not mention it since Siri may have built up its speech recognition past the point of "Beta". Thus, issues in recognition for some after launch are no longer present today.  

Thus, the online moniker would be removed to then coincide with the iOS6 release.    

post #105 of 141
Quote:
Originally Posted by jragosta View Post



I have a PhD in science.
 

 

 

This reminds me of the scene in Good Will Hunting where Ben Afflek is trying to pick up a Harvard girl by telling her that he thinks that they have a class together.

 

"Oh?  Which class?"

 

"History".

 

LOL

 

Me, personally, I've never heard of a PhD saying that they have a degree in a vague, general area. 

post #106 of 141
Quote:
Originally Posted by MacBook Pro View Post


My wife is born and raised Japanese, listening to her trying to coax information from Siri in English is hysterically funny. Siri in Japanese seems to be fairly reliable for her though.

 

Interesting point.  Siri in iOS 6 has several dialects of most languages, e.g.  Spanish (Spain), Spanish (Mexico), Spanish (United States).

 

Likely, Siri will evolve to understand mixed languages... Eso es Scheiße! (SpanDeutsche).

"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #107 of 141
Quote:
Originally Posted by Blitz1 View Post

Well, as a former project manager, I can tell you this: a beta is never intended for a large audience.
It's meant for a review group sending back findings on using the software.

- does Siri beta have such reporting functions?

- is it intended for a review group?

- was it launched as a feature for the whole of the intended audience from day 1?

- does it implicate that when Siri is out of beta, that it will be flawless (as current errors are so vividly attributed to a beta status?

 

Now, for me, Siri works quite well. I do believe it's slightly better than the 62% and I can live with that.

But please, don't BS me with a beta status. Never can I inform Apple why the 38% failed and what I should have had.

While I posted some possible reasons for calling it Beta, your post "A Beta is never intended for a large audience" needs clarification. What is a large audience. remember, while Siri when released it could handle some foreign languages. However, the functionality in those countries was limited as well. Thus, while the iPhone 4S was and is released in lots of countries, Siri can't work fully or at all in many. Google used to release everything as Beta and stayed that was for years. The audience was huge. Also, we are talking iPhone 4S users here which is a subset of a larger audience of total iPhone users with a significant number still on iPhone 4 and 3Gs. At the next launch, many if not most of those will be able to upgrade with their carrier to the new phone and thus most iPhone users will then be on a Siri enabled device. 

post #108 of 141
Quote:
Originally Posted by mjtomlin View Post

 

Like Gmail?

 

Siri has only been out for 8 months... Gmail, an email service was out for years before it went off beta. Email was a tried and true technology for over 20 years at the time!!! Why in the hell was it ever in beta in the first place?

 

Siri is a learning technology, meaning it needs to learn your voice and enunciation. Sorry do you not know what that means? It needs to learn how you talk, slurs and all.

 

I think int he past 7 months since I've had my iPhone, Siri has dissed me a handful of times (maybe four or five).

 

Exactly!   And Siri does learn!

 

When I first started using Siri, she had trouble with family names...

 

Especially my oldest grandson -- First name: Braden;  Last name Starts with "Gi".

 

Siri pronounced these wrong -- she used a short 'a' in Braden instead of a long 'a';  and a 'Ji' instead of a hard 'G' sound for the last name.

 

Over time, Siri learned the correct pronunciation by keeping track of my usage.

 

 

Siri, What's my name?

 

Siri:  "Your  Dick, but since we're friends, I get to call you 'Oh, Omnipotent One'".

 

 

She had trouble with "Omnipotent" (don't they all), but eventually got it right.

"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #109 of 141
Quote:
Originally Posted by Dick Applebaum View Post

 

When I first started using Siri, she had trouble with family names...

 

 

 

You should hear her butcher some of my contact names.

 

Xianli  has to be pronounced Zi-on-lee when actually phonetically it is only two syllables - jian-li

 

Also my friend Venicio of course sounds like "B" not "V" so again you have to say it horribly wrong to make it work.

 

When the Spanish version comes out you will be known as 'Dee-ca'


Edited by mstone - 6/29/12 at 3:07pm

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #110 of 141

Personally, I find that Google's accuracy and reliability when entering queries with text have been on a downhill slide for at least a year — without Siri being involved at all.  I wouldn't want to base a study of anything's reliability or accuracy when using Google.

post #111 of 141

I guess I missed this until just now, but read at theVerge that voice recognition and dictation now works without a data connection on Google's latest OS, processing directly on your device. Fairly surprising. 

melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #112 of 141
Quote:
Originally Posted by jragosta View Post


Are you a flipping idiot or what?
I have a PhD in science. I know what is scientific.
Now, a scientific study may or may not have validity, but if it meets the premises of science, it is a scientific study. That is, it must have:
1. A testable hypothesis. Check
2. A methodology for testing the hypothesis. Check.
3. A control. Check.
4. Results. Check.
5. A conclusion. Check.
It meets the criteria. They are being careful to ensure that it's not being used as the be-all and end-all of research on the subject, but to say that it's not scientific just because it doesn't have a lot of replication is just plain wrong.
And, no matter how you slice it, it's infinitely more valid than a Motorola advertisement saying that Motorola's phones are better than Apple's phones. That should be obvious to anyone thinking at above a 5 year old's level. Sadly, that seems to leave you out.

 

Pretty sad statement for a PhD I think.  You are completely forgetting (or worse--ignoring) that 1, 2, 3 & 5 need to be properly executed and carefully crafted to remove bias from the measurements.  Without that your 1-5 just give us pseudo science, which is  worse than something that is merely unscientific.  Why worse?  Pseudo science is a lie cloaked in the trappings and lingo of science in an attempt to deceive.  

 

I vote that Munster was unscientific. He wasn't attempting to lie, he just doesn't know how to craft an unbiased study.

.
Reply
.
Reply
post #113 of 141
Quote:
Originally Posted by Hiro View Post

Pretty sad statement for a PhD I think.  You are completely forgetting (or worse--ignoring) that 1, 2, 3 & 5 need to be properly executed and carefully crafted to remove bias from the measurements.  Without that your 1-5 just give us pseudo science, which is  worse than something that is merely unscientific.  Why worse?  Pseudo science is a lie cloaked in the trappings and lingo of science in an attempt to deceive.  

So tell us what was done with bias in the comparison I provided.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #114 of 141
Quote:
Originally Posted by imt1 View Post

While I posted some possible reasons for calling it Beta, your post "A Beta is never intended for a large audience" needs clarification. What is a large audience. remember, while Siri when released it could handle some foreign languages. However, the functionality in those countries was limited as well. Thus, while the iPhone 4S was and is released in lots of countries, Siri can't work fully or at all in many. Google used to release everything as Beta and stayed that was for years. The audience was huge. Also, we are talking iPhone 4S users here which is a subset of a larger audience of total iPhone users with a significant number still on iPhone 4 and 3Gs. At the next launch, many if not most of those will be able to upgrade with their carrier to the new phone and thus most iPhone users will then be on a Siri enabled device. 

His statement is false.

There is not an industry standard for the audience of a beta.

There isn't even an industry standard for the term "beta" (although "beta" is generally accepted to be a version for limited distribution which may have issues not generally acceptable to the public).

There is a trend in the last decade for very large "public beta" releases of highly complicated software with complicated algorithms (artificial intelligence or computer graphics), constrained resources (to evaluate performance at load) or expansive environments (simulated "worlds"). Siri arguably meets all three criteria for a large public beta as listed.

Some businesses have added gamma and omega releases.
post #115 of 141

I don't understand why Siri will not accept written requests.

 

Voice in a noisy environment will always be challenging.

 

The value of Siri is not just voice.  Siri is smarter than Google cause it editorializes.

 

For example "remind me when i leave"

post #116 of 141

Siri is in beta because the technology requires significant use to to develop and understand it's user base to eventually be as comprehensive and intuitive as possible.  All requests are sent to Apple's servers to be analyzed and processed. It's a system of learning and can't simply be mastered in a few revisions. This is a work in progress. If Siri wasn't in beta, people would lose their minds if it went down or didn't handle a request as the wished. I personally have no gripes to think of (I own a 4S) and am anticipating some welcome changes in iOS 6.

 

If the argument is that you don't think Siri works as well as it does in the commercials where dramatizations speed up requests, then maybe you should be upset with the rest of the marketing world as well.  Do a fancy pair of basketball shoes make you better at basketball? Does an expensive liquor make you classy? Will a Happy Meal actually make you happy when you finally consume it?  More people than you'd think are walking around with their intellect off and respond to pure emotional stimulation.  It's an advertisers job to cater to that market since they're the mass majority. There will always more stupid people (less inquisitive and more dependent) than smart people and stupid people are an easy sale.  Smart people are more inquisitive and require more stimulation to their intellect, rather their senses and emotions. The "green" market is one example for the more intellectual -- which also helps Apples market share being that their products use highly recyclable components.  The Toyota Prius is another example for that market base.

 

When intelligent people are forced to consume, they do research and use their knowledge to determine what is best for them.

If you need all you information from a commercial and are mad at Apple for somehow misleading you with theirs, I know where you stand.

 

As fas as tests go, I require specifics. I don't trust percentages.  For example, if you were to hear on the news or read poll results that say 80% of the voters are against legalizing marijuana, I'd want to know how many voters there were and what was their demography? Meanwhile, the polls may have been answered by elderly republicans or religious fanatics (more republicans, lol) that still think the movie Reefer Madness holds water and had no idea that the paper companies were behind the misleading propaganda to eliminate the competition (hemp).

 

Off topic..

How many haters actually use Apple products? I've met a few Apple haters in my time and they've all converted because their tired old rhetoric was just that.  They saw the light when their friends were using MBPs while they were replacing or repairing laptops too frequently and wasting money. Not to mention all the bloatware, spyware and other common malicious attacks that are inherent to nearly all Windows machines. Unfortunately, most are still forced to use Windows for work, but that works on the same hardware so there's no problem there.

 

People commonly make the mistake of buying a PC thinking the higher spec'd hardware will outperform and Mac.  This is still true in some areas.  Apple uses quality hardware that run at a more comfortable pace.  PCs put a bunch of top performing hardware in a plastic shell.  Time after time the higher performing hardware will burn itself out.  Apple products are a more refined technology and manage performance in a more optimal and reliable manner.  Personally, I wouldn't buy any other machine unless, like others have mentions, they totally screw up big like Nokia and RIM, which seems unlikely being that Apple is the one putting them in their place.  My first computers were hand-me-down PCs. My first purchase was a Mac.

 

I can still remember the day I was driving home with my new iPod classic in '04 and I said to my passenger "Now if they'd only make a phone out of this so I won't have to carry both."  Sure enough, Apple did and wow has the market changed because of them.

post #117 of 141
Quote:
Originally Posted by AppleInsider 

While the convenience of Apple's Siri comes with searching by voice, a new test has found that Apple's personal assistant is not yet a reliable enough search engine to entirely replace text input with Google.
Of the 83 percent of queries Siri could comprehend, 21 percent were answered inaccurately. Some of the questions that stumped Siri were:
  • "What team does Peyton Manning play for?" Siri would respond with the answer to the previous query.
  • "Where is Elvis buried?" Siri looked for a person named "Elvis Buried."
  • "Where am I?" Sometimes a map pin would be dropped in the wrong place.
  • "When did the movie Cinderella come out?" Siri would attempt to find current theater movie times via Yelp.
  • "How do I get from Boston to New York?" Siri responded: "I can only give directions from your current location. I can't give you directions to a place you are not in."

In Munster's test, he gave Google an A+ for comprehension and B+ for accuracy, while Apple's Siri was graded with a B for comprehension and a D for accuracy.
His testing also found that Siri relies on Google for 60 percent of its answers in iOS 5. But this fall, with the launch of iOS 6, Munster projects that Siri's reliance on Google will drop to 48 percent, replacing the search giant with Apple's own solutions for mapping and sports scores.
"While Siri is two-plus years behind Google today, we are optimistic about its future," Munster wrote. "With the iOS 6 release in the fall, we expect Siri to improve meaningfully while reducing its reliance on Google."
 
Interesting questions. But unless I haven't done my due diligence, most of the questions exampled above do not fall into the capabilities presented on the Apple Siri feature page, except for the links to the listed internet sites it queries. As such, perhaps the protocol, of which we have not be aprised, provides the answers that all qualified researchers understand, i.e., "Ask a stupid question and you get a stupid answer."
 
Certainly a Google text-search should give more accurate answers than Siri is or every will be capable. But giving more accurate answers does not mean they are all accurate. For example Googling, "where is elvis buried" and you get over 6 million hits. And not all answers given, e.g., "Paul The Octopus 'To Record Elvis Tribute Album," mentions anything about where Elvis is buried.
 
As for, "How do I get from Boston to New York?" Siris does exactly as it is supposed to do. Ask the exact question via Google and much of the 1.4 billion results aren't accurate. To be picky, my youngest grandson would have simply answered, "By walking, flying, car, bus, train, boat, swimming, sailing…." My older granddaughter would have been more guiding, but would not have outlined the "Getting From New York City to Boston" route first as Google did.
 
When I showed this article to my wife, her response was immediate and had me Google three questions while she Siri'd her iPhone.
 
  1. When is our anniversary?
  2. When is my birthday?
  3. Where is your iPhone?
 
Needless to say, Google couldn't answer correctly to any of the three as Siri couldn't on the third question. But as she pointed out, "Siri is beta. Just like most men will always be. By the way, your iPhone is where you left it, in the…and I suggest you use be nice to Siri in the next few weeks."
 
post #118 of 141
Quote:
Originally Posted by Blitz1 View Post

Well, as a former project manager, I can tell you this: a beta is never intended for a large audience.

It's meant for a review group sending back findings on using the software.
- does Siri beta have such reporting functions?
- is it intended for a review group?
- was it launched as a feature for the whole of the intended audience from day 1?
- does it implicate that when Siri is out of beta, that it will be flawless (as current errors are so vividly attributed to a beta status?

Now, for me, Siri works quite well. I do believe it's slightly better than the 62% and I can live with that.
But please, don't BS me with a beta status. Never can I inform Apple why the 38% failed and what I should have had.

I think you can blame Google. They rolled out "beta" products for the world to use and had that designation out there for years. I never noticed a bug submission button anywhere with those products.

The part I am disappointed with is the only place the beta is noted is on the web site. The TV ads I've seen certainly don't say it's beta, and I hadn't seen it noted on the store displays. It just seems wrong to advertise it as your flagship feature and not make it clear in all your marketing materials that it's beta.
Edited by JeffDM - 6/30/12 at 6:14am
post #119 of 141
Quote:
Originally Posted by jragosta View Post


So tell us what was done with bias in the comparison I provided.

You simply ignored all the contradictory evidence or lack of evidence.  The blindness to protocol led to confirmation bias in your assessment that Munster's study was scientific.  

.
Reply
.
Reply
post #120 of 141
Quote:
Originally Posted by q2h View Post

 

 

When intelligent people are forced to consume, they do research and use their knowledge to determine what is best for them.

If you need all you information from a commercial and are mad at Apple for somehow misleading you with theirs, I know where you stand.

 

This skirts the issue.  If you were shopping for food in the store and found what seemed to be a prepared vegetarian dish, and nothing on the package stated otherwise, you would assume that you shouldn't need to go home and go to the manufacturer's site, and look up the dish in their index to find that it actually was not truly vegetarian.  And the single place you could find this would be to go to this page, it wouldn't be on the posters in the stores or commercials on TV.   Fortunately, here in the US you couldn't sell food this way.  That in electronics it is another story simply means just that, it doesn't mean anyone who bought an iPhone and never saw this disclaimer is an idiot.  

 

Half the people I know who have iPhones have never been to the Apple site for this purpose.  When you say "When intelligent people are forced to consume they do research and use their knowledge to determine what is best for them" you're making a huge sweeping generalization that Aunt Minnie and Grandpa Joe, who don't spend much time tinkering online, are exactly like us.  We have a bucketload of iPhones in my extended family, including some oldsters.  The fact is that they don't really care if Siri works perfectly or not and they love their phones but none of them could answer the "Is Siri beta?" question, and I doubt they've ever been to the Apple site to look up iPhone info.  They didn't know they had to!!!  Idiots?  Nope.  Like I say, they don't feel misled because they love their iPhones, don't rely on Siri and half of them don't know what beta means.  But that's not the point.  It's been successfully shifted off the radar.  

 

Ask the next 50  people you meet on the street with iPhones the Siri/beta question.  How many could answer it?   According to you those that couldn't are dolts (and deserve what they get?)  I disagree with that generalized assumption.

New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
AppleInsider › Forums › Mobile › iPhone › Street test measures Siri comprehension at 83%, accuracy at 62%