Street test measures Siri comprehension at 83%, accuracy at 62%

245678

Comments

  • Reply 21 of 143
    gatorguygatorguy Posts: 24,771member

    Quote:

    Originally Posted by jragosta View Post



    Fortunately, we don't have to rely on guesses from Apple haters and google shills.

    One side by side comparison shows the opposite - Siri is more accurate than Google Voice:

    http://blog.thearorareport.com/2011/11/14/aaple-siri-voice-recgnition-trumps-android-voice/


    I believe I suggested Munster would do a future side-by-side with Google Now, not Google Voice. Read more carefully please. So yes, even you still have to rely on guesses for now on how they'll compare.


     


    BTW, this is a much more recent test putting the two head-to-head. One test doesn't prove anything of course, whether this one or Munsters.


    http://www.youtube.com/watch?v=kDsOtdRtG0Q&feature=player_embedded

     0Likes 0Dislikes 0Informatives
  • Reply 22 of 143
    lightknightlightknight Posts: 2,312member

    Quote:

    Originally Posted by sr2012 View Post



    Statistics can be used to prove anything. 80% of people know this.




    Made-up statistics don't prove anything. 99% of people agree with me.

     0Likes 0Dislikes 0Informatives
  • Reply 23 of 143
    sr2012sr2012 Posts: 896member
    I think it depends  as much on the ability of the sexor as the sexee.

    Well, me and Fluffy Farah, it's 100% better than real-life then... LOL
     0Likes 0Dislikes 0Informatives
  • Reply 24 of 143
    lightknightlightknight Posts: 2,312member

    Quote:

    Originally Posted by jragosta View Post



    The problem is that this is a limited test and can not be easily verified. If someone uses a sample of native English speakers vs. a bunch of people just off the boat who learned English from a guidebook, the answer will be different.

    The only real indication is what the millions of Siri users think about the product. One survey (using a random sample which should be representative) found that 96% if iPhone 4S users were somewhat satisfied or very satisfied with their phone and when asked about the best thing about the phone, 50% said 'Siri'.

    http://www.uswitch.com/mobiles/news/2011/12/iphone_4s_siri_score_big_for_customer_satisfaction/

    Or, you could do a side by side comparison using the same speakers and the same conditions. Siri had 96% accuracy compared to Google Voice at 93%:

    http://blog.thearorareport.com/2011/11/14/aaple-siri-voice-recgnition-trumps-android-voice/

    Of course, the real benefit is not just the accuracy, it's what you can do with it. If you're driving, there's some value in "Siri, make an appointment with John for 9 am next Wednesday". Even if you have to repeat it, it's better than having to rely on your memory until you get home or trying to manually enter the appointment while driving.




    So, basically since (for once) Munster has a result that doesn't proclaim Apple's dominance, his experiment is null and void? He explains how he did it, it's therefore verifiable. Moreover, your comment smells of racism and/or xenophobia. Let me highlight this for you: "bunch of people just off the boat who learned English from a guidebook". I _hope_ for you nobody on Munster's team is from foreign origin and willing to take you to court. Provided of course, they've got nothing better to do than read your comments, like me ;)

     0Likes 0Dislikes 0Informatives
  • Reply 25 of 143
    jragostajragosta Posts: 10,473member
    gatorguy wrote: »
    I believe I suggested Munster would do a future side-by-side with Google Now, not Google Voice. Read more carefully please. So yes, even you still have to rely on guesses for now on how they'll compare.

    BTW, this is a much more recent test putting the two head-to-head. One test doesn't prove anything of course, whether this one or Munsters.
    http://www.youtube.com/watch?v=kDsOtdRtG0Q&feature=player_embedded

    Yes, but this does point out the difference in philosophy. Go to the section where he asks it to set an alarm. Google Voice displays a page where you have to manually enter the information. Siri asks you to tell it what time to set the alarm and so on. If I'm driving or my hands are full, there's a world of difference.

    The huge functionality difference is far more important than the 0.1 second time difference he keeps bragging about.

    But, nonetheless, that comparison does nothing to answer the question of accuracy which is what this thread and the post you responded to was all about. Read the link I provided which shows that Siri is more accurate than the existing Google product. Since neither iOS 6.0 or Google Voice is currently available to users, we have to rely on existing products.
     0Likes 0Dislikes 0Informatives
  • Reply 26 of 143
    gatorguygatorguy Posts: 24,771member

    Quote:

    Originally Posted by sr2012 View Post



    But iOS6 along with a super sexy iPhone (imagine iPhone 4S but with only as thick as the stainless steel band) is going to be... Pretty darn significant.


    If they add the expected larger 4"+ display along with LTE support, I completely agree. The next iPhone is probably a killer. 

     0Likes 0Dislikes 0Informatives
  • Reply 27 of 143
    daharderdaharder Posts: 1,580member
    robogobo wrote: »
    It's "allowed" to be called beta as long as it's in beta.  And it's still in beta because they haven't worked out the problems completely.  As soon as they're worked out, it will probably be out of beta.  Make sense?

    Then Apple should stop touting it as a primary 'feature' of the iPhone 4S in those ads/TV commercials until the 'feature' is complete i.e. Out of Beta.
     0Likes 0Dislikes 0Informatives
  • Reply 28 of 143
    jragostajragosta Posts: 10,473member

    So, basically since (for once) Munster has a result that doesn't proclaim Apple's dominance, his experiment is null and void?

    I guess critical thinking isn't your thing. I didn't object because of the result but because it was a stupid test. There were major flaws in his test:
    1. Comparing a voice recognition system to manual text entry is inane.
    2. His experiment is not reproducible because we don't know what queries he used and there is no control.
    He explains how he did it, it's therefore verifiable. Moreover, your comment smells of racism and/or xenophobia. Let me highlight this for you: "bunch of people just off the boat who learned English from a guidebook". I _hope_ for you nobody on Munster's team is from foreign origin and willing to take you to court. Provided of course, they've got nothing better to do than read your comments, like me ;)

    Once again, you obviously never took a critical thinking course. Nothing I said is grounds for any kind of lawsuit. What I said was:
    "If someone uses a sample of native English speakers vs. a bunch of people just off the boat who learned English from a guidebook, the answer will be different."

    That is a true, completely factual statement. I didn't say that anyone working for Munster fell into either category. I simply pointed out that voice recognition systems will depend on how similar the voice is to the average speaker for that language. If you want to do a real comparison, use the same speakers reading the same text to Siri and Google's voice recognition - and Siri comes out ahead

    As to 'racist and xenophobic', that's one of the more bizarre comments you've ever made (and that's saying a lot). Simply stating that a voice recognition system will have more trouble understanding someone who doesn't speak good English is xenophobic? Pretty bizarre conclusion.
     0Likes 0Dislikes 0Informatives
  • Reply 29 of 143
    rogifanrogifan Posts: 10,669member
    robogobo wrote: »
    It's "allowed" to be called beta as long as it's in beta.  And it's still in beta because they haven't worked out the problems completely.  As soon as they're worked out, it will probably be out of beta.  Make sense?
    Well you have to admit Apple is touting it as a major feature of the iPhone. I mean how many companies advertise on beta product/service?
     0Likes 0Dislikes 0Informatives
  • Reply 30 of 143
    tallest skiltallest skil Posts: 43,388member
    rogifan wrote: »
    Well you have to admit Apple is touting it as a major feature of the iPhone. I mean how many companies advertise on beta product/service?

    Google. Or maybe you've all forgotten Gmail.
     0Likes 0Dislikes 0Informatives
  • Reply 31 of 143
    ankleskaterankleskater Posts: 1,287member

    Quote:

    Originally Posted by jragosta View Post





    Yep. Not very intelligent.

    Aside from the fact that he's comparing a text input to voice recognition, there are no controls or no way to compare how Siri stacks up against the competition.

    Fortunately, we don't have to rely on guesses from Apple haters and google shills.

    One side by side comparison shows the opposite - Siri is more accurate than Google Voice:

    http://blog.thearorareport.com/2011/11/14/aaple-siri-voice-recgnition-trumps-android-voice/




    Not just unintelligent. Totally misleading. Why not compare Siri to talking to a human? Why not compare Siri to training a dog? While we have come to expect silliness from professional analysts, this is lowering the bar again. Of course, Munster will become "the man" if and when Apple releases a flat panel TV, because he predicts its emergence every year.

     0Likes 0Dislikes 0Informatives
  • Reply 32 of 143
    gatorguygatorguy Posts: 24,771member

    Quote:

    Originally Posted by jragosta View Post





    Yes, but this does point out the difference in philosophy. Go to the section where he asks it to set an alarm. Google Voice displays a page where you have to manually enter the information. Siri asks you to tell it what time to set the alarm and so on. If I'm driving or my hands are full, there's a world of difference.

    The huge functionality difference is far more important than the 0.1 second time difference he keeps bragging about.

    But, nonetheless, that comparison does nothing to answer the question of accuracy which is what this thread and the post you responded to was all about. Read the link I provided which shows that Siri is more accurate than the existing Google product. Since neither iOS 6.0 or Google Voice is currently available to users, we have to rely on existing products.


    And this link says just the opposite:


    http://www.huffingtonpost.com/2012/02/27/motorola-siri-android-iphone_n_1304984.html


     


    One, or even a couple of tests is not definitive proof. Also as you mentioned, some users might be better served by Google Voice Actions which has mor feature control functions, while Siri may fit the bill jsut fine for others. There's no clear "better" solution regardless of any current testing

     0Likes 0Dislikes 0Informatives
  • Reply 33 of 143
    anonymouseanonymouse Posts: 7,123member

    Quote:

    Originally Posted by Gatorguy View Post


    I've no doubt that Mr. Munster will do a side-by-side test between Siri and Google Now once it's in release, as will hundreds of others. I don't expect miracles but would not be at all surprised that Google would offer the more accurate results between the two. I'd personally expect comprehension to be a draw. Both iOS and Android have pretty good voice recognition already.



     


    Siri isn't voice recognition, it's natural language processing, so, the state of Android's voice recognition (a relatively trivial problem at this point) isn't really relevant. But, thanks for the shill talk.


     




    By the way, isn't Munster generally a pro-Apple guy? Certainly no sign that he cares much for Google AFAIK. I'd be surprised if his tests were intended to make Siri look bad.



     


    No, he's an analyst who makes a lot of incorrect predictions about what Apple and AAPL will do. With Munster, the issue isn't malice, but rather whether he know what he's doing. But, it's amusing that you try to paint him as pro-Apple, as if you don't know exactly who he is and what he does.

     0Likes 0Dislikes 0Informatives
  • Reply 34 of 143
    zeromeuszeromeus Posts: 182member

    Quote:

    Originally Posted by ankleskater View Post


    Did I understand this correctly - Munster directly compared Siri to ... Google search by text input? 

     





    Yes!  What a stupid test!  Of course the text input would be more accurate.  This isn't exactly news.

     0Likes 0Dislikes 0Informatives
  • Reply 35 of 143
    q2hq2h Posts: 18member

    Quote:

    Originally Posted by Tallest Skil View Post





    Google. Or maybe you've all forgotten Gmail.


    That's exactly what I was about to say, but I decided to search 'gmai' before posting.  


     


    TS, of all the tech sites that I follow and see you on, I must say I usually agree with your arguments.  The problem is that people like us think before we act and as a result, we become outnumbered by people that don't and have no idea what they're talking about.


     


    Real life Idiocracy..


    www.imdb.com/title/tt0387808/

     0Likes 0Dislikes 0Informatives
  • Reply 36 of 143
    jragostajragosta Posts: 10,473member
    gatorguy wrote: »
    And this link says just the opposite:
    http://www.huffingtonpost.com/2012/02/27/motorola-siri-android-iphone_n_1304984.html

    One, or even a couple of tests is not definitive proof. Also as you mentioned, some users might be better served by Google Voice Actions which has mor feature control functions, while Siri may fit the bill jsut fine for others. There's no clear "better" solution regardless of any current testing

    ROTFLMAO.

    Only in your world would a paid advertisement with no methodology or controls be a valid response to an independent side-by-side comparison with appropriate controls.

    You know you've lost when you have to resort to an advertisement to make a claim that a product is better.
     0Likes 0Dislikes 0Informatives
  • Reply 37 of 143
    postulantpostulant Posts: 1,272member


    In my tests, the only thing that Siri gets consistently right with a 100% accuracy is setting timers and alarms. 


     


    In fact, I haven't used my alarm clock since getting the 4S.

     0Likes 0Dislikes 0Informatives
  • Reply 38 of 143

    Quote:

    Originally Posted by jragosta View Post





    I guess critical thinking isn't your thing. 




    Once again, you obviously never took a critical thinking course.


    that's one of the more bizarre comments you've ever made (and that's saying a lot).


     


     


     


    This stuff is completely uncalled for.

     0Likes 0Dislikes 0Informatives
  • Reply 39 of 143
    jragostajragosta Posts: 10,473member

    This stuff is completely uncalled for.

    Why? He posted inane personal attacks which had no basis in reality and I called him on it.
     0Likes 0Dislikes 0Informatives
  • Reply 40 of 143
    newbeenewbee Posts: 2,055member

    Quote:

    Originally Posted by   View Post


    If they add the expected larger 4"+ display along with LTE support, I completely agree. The next iPhone is probably a killer. 



    Judging from the dominance of the current iPhone (leading seller, by all accounts) .... I'd have to say it's already a "killer".

     0Likes 0Dislikes 0Informatives
Sign In or Register to comment.