Siri 'whistleblower' details drug deals & sex heard during manual reviews

Posted:
in iPhone edited December 2019
A 'whistleblower' has taken issue with Apple's lack of disclosure that it has contractors listening to anonymized Siri queries -- but the company has said all along that it does.




Siri has repeatedly been the subject of improvements over the years, as Apple continues to make the voice recognition systems it uses work better, to minimize the possibility of incorrect answers to questions. As part of this, it is a selection of recordings where potential issues are detected are passed along to workers for analysis.

A report from the Guardian reveals a small number of recordings are passed on to contractors working for Apple, tasked with determining if the Siri activation was accidental or on purpose, if it was a query within the range of Siri's capabilities, and whether Siri acted properly.

The main thrust of the report claims that Apple does not explicitly disclose to consumers that recordings are passed along to contractors -- but Apple does tell users that some queries are manually reviewed, and has since the release of the service. Despite the information having been public-facing for at least six iterations of iOS, the "whistleblower" advised that they were concerned over the lack of disclosure, especially considering the contents of some recordings containing "extremely sensitive personal information."

The nature of the information, sometimes unintentional and not part of the query, is wide-ranging, the whisteblower said.

"You can definitely hear a doctor and patient, talking about the medical history of the patient," said the source. "Or you'd hear someone, maybe with car engine background noise - you can't say definitely, but it's a drug deal. You can definitely hearing it happening," they advised.

The whistleblower goes on to state there are many recordings "featuring private discussions between doctors and patients, business deals, seemingly criminal dealings, sexual encounters and so on. These recordings are accompanied by user data showing location, contact details, and app data."

Allegedly, there isn't a procedure in place to deal with sensitive recordings, with the whistleblower stepping forward over the suggestion the data could be easily misused. Citing a lack of vetting for employees and the broad amount of data provided, it is suggested "it wouldn't be difficult to identify the person you're listening to, especially with accidental triggers" like names and addresses, especially for "someone with nefarious intentions."

Apple confirmed "A small portion of Siri requests are analysed to improve Siri and dictation," but added it was kept as secure as possible.

"User requests are not associated with the user's Apple ID," the company continued, "Siri responses are analysed in secure facilities and all reviewers are under the obligation to adhere to Apple's strict confidentiality requirements."

Apple added that a random subset of less than 1% of daily Siri activations are used for grading, recordings typically only a few seconds in length.

The report follows after similar privacy-related stories about Google Assistant and Amazon's Alexa, where teams had access to some customer data logs with recordings, for similar review purposes.

In the Amazon case, the captured voice data was associated with user accounts. In regards to Google, a researcher provided a voice snippet that Google retained to analysis to the reporter that made the request -- but Google says that the samples aren't identifiable by user information. How the user was identified by the researcher isn't clear.
«134

Comments

  • Reply 1 of 74
    AppleExposedAppleExposed Posts: 1,805unconfirmed, member
    No Apple IDs are stored and they're one random clip at a time.

    I can see the concerns but keeping it anonymous means we shouldn't worry.
    racerhomie3jony0
  • Reply 2 of 74
    If {sounds like a drug deal}, go back to sleep


    AppleExposedcornchip
  • Reply 3 of 74
    zoetmbzoetmb Posts: 2,654member
    There's a big difference between 'hearing" a Siri request ("where's the closest pizza place") and hearing conversations when a request isn't specifically being made.  Which is it?   The former doesn't bother me, but the latter does.  

    But if they were listening to me using Siri, all they would hear is a lot of annoyance and cursing after I asked the initial query.   
    netlingAppleExposedcgWerkscornchipFileMakerFeller
  • Reply 4 of 74
    Mike WuertheleMike Wuerthele Posts: 6,858administrator
    zoetmb said:
    There's a big difference between 'hearing" a Siri request ("where's the closest pizza place") and hearing conversations when a request isn't specifically being made.  Which is it?   The former doesn't bother me, but the latter does.  

    But if they were listening to me using Siri, all they would hear is a lot of annoyance and cursing after I asked the initial query.   
    I'm sure there's some caught that are inadvertent triggers. That's the whole point of process improvement.
    1STnTENDERBITSdewme
  • Reply 5 of 74
    mystigomystigo Posts: 183member
    I bet they hear Siri being called "idiot" a lot. That's my go to response when it tries to take me to an address 3 states away instead of the same address 3 miles away. I have literally had it complain that it could not find a route to England from the US. The assumption that I was looking for a way to drive across the Atlantic to the UK is just flawed on so many levels. It has zero sense of geographical context. And it seems like it would be so easy to fix. Just assume I want the closest match. Idiot.
    space2001mobirdentropysviclauyychungovermuthuk_vanalingamFileMakerFellerurahara
  • Reply 6 of 74
    zoetmb said:
    There's a big difference between 'hearing" a Siri request ("where's the closest pizza place") and hearing conversations when a request isn't specifically being made.  Which is it?   The former doesn't bother me, but the latter does.  

    But if they were listening to me using Siri, all they would hear is a lot of annoyance and cursing after I asked the initial query.   
    I'm sure there's some caught that are inadvertent triggers. That's the whole point of process improvement.
    Yes.  I've occasionally had Siri activate when I haven't said the key phrase.  I've always assumed that I said something that the algorithm interprets in the same way as the key phrase.  Similar to the way in which two different strings can (however unlikely it might be) result in the same hash code.
  • Reply 7 of 74
    AppleExposedAppleExposed Posts: 1,805unconfirmed, member
    zoetmb said:
    There's a big difference between 'hearing" a Siri request ("where's the closest pizza place") and hearing conversations when a request isn't specifically being made.  Which is it?   The former doesn't bother me, but the latter does.  

    But if they were listening to me using Siri, all they would hear is a lot of annoyance and cursing after I asked the initial query.   

    The idea is to minimize false queries. So this is why the 'whistleblower' mentioned these. His job was to determine whether or not the user intended to talk to Siri. If anything he says is to be believed.

    They're not intentionally turning on your microphone to eavesdrop.
    chasmlolliver
  • Reply 8 of 74
    Hey sir I just wanted to tell you about the important confidential matter that I hope hasn't been accidentally recorded because I have a funny way of addressing people. Oh, and Alexa says hi!
    n2itivguy
  • Reply 9 of 74
    anantksundaramanantksundaram Posts: 20,403member
    This is not good.

    The issue has nothing to do with whether they are anonymized or not. The issue is permissions and disclosure. Period.

    I expect we'll hear from Apple more on this.
    viclauyyclogic2.6bb-15chemengin1
  • Reply 10 of 74
    bonobobbonobob Posts: 382member
    This is not good.

    The issue has nothing to do with whether they are anonymized or not. The issue is permissions and disclosure. Period.

    I expect we'll hear from Apple more on this.
    I suspect you agreed to what Apple is doing when you clicked through the terms & conditions as you set up your iDevice.  I know I did.
    n2itivguycornchipwatto_cobralolliveruraharajony0
  • Reply 11 of 74
    Mike WuertheleMike Wuerthele Posts: 6,858administrator
    This is not good.

    The issue has nothing to do with whether they are anonymized or not. The issue is permissions and disclosure. Period.

    I expect we'll hear from Apple more on this.
    FTA: "The main thrust of the report claims that Apple does not explicitly disclose to consumers that recordings are passed along to contractors -- but Apple does tell users that some queries are manually reviewed, and has since the release of the service. Despite the information having been public-facing for at least six iterations of iOS, the "whistleblower" advised that they were concerned over the lack of disclosure, especially considering the contents of some recordings containing "extremely sensitive personal information."
    pbruttoentropysn2itivguywatto_cobra
  • Reply 12 of 74
    FolioFolio Posts: 698member
    There are more serious things with Siri than this Guardian clickbait. I changed Siri's voice on my iphone X to British female. For several months, out of the blue she'll startle me by blurting a repeat of prior day personal response. Often comes when recharging via MBP. Also noticed degradation of Siri direction prowess on maps. But still a grace period for new Siri team. many users like my 23 year old EE friend never use Siri and still addicted to Apple. But I think siri and ML will be key to Apple's stature-- good or bad--five years from now.
  • Reply 13 of 74
    LordeHawkLordeHawk Posts: 168member
    I call bullshit, location and contacts never get sent with Siri requests.  Sounds like someone wants the public to associate Siri as being as bad as Google and Amazon.

    If you turn Siri off and then back on in settings, a new anonymous Siri ID is issued.
    StrangeDaysDAalsethgregoriusmmwhiteSydNn2itivguycornchipwatto_cobralolliverurahara
  • Reply 14 of 74
    hentaiboyhentaiboy Posts: 1,252member
    “ A 'whistleblower' has taken issue with Apple's lack of disclosure that it has contractors listening to anonymized Siri queries -- but the company has said all along that it does.”

    So what percentage of Siri users are aware that humans may listen to their conversations? No-one reads software license agreements. 
    FileMakerFeller
  • Reply 15 of 74
    Mike WuertheleMike Wuerthele Posts: 6,858administrator
    hentaiboy said:
    “ A 'whistleblower' has taken issue with Apple's lack of disclosure that it has contractors listening to anonymized Siri queries -- but the company has said all along that it does.”

    So what percentage of Siri users are aware that humans may listen to their conversations? No-one reads software license agreements. 
    You'd think at the least the "whistleblower" would, or the Guardian would check.
    AppleExposedradarthekatFileMakerFellerwatto_cobramagman1979
  • Reply 16 of 74
    hentaiboyhentaiboy Posts: 1,252member
    “ FTA: "The main thrust of the report claims that Apple does not explicitly disclose to consumers that recordings are passed along to contractors -- but Apple does tell users that some queries are manually reviewed, and has since the release of the service.”

    Followed your link. Where is the pertinent advice?
    gatorguy
  • Reply 17 of 74
    AppleExposedAppleExposed Posts: 1,805unconfirmed, member
    LordeHawk said:
    I call bullshit, location and contacts never get sent with Siri requests.  Sounds like someone wants the public to associate Siri as being as bad as Google and Amazon.

    If you turn Siri off and then back on in settings, a new anonymous Siri ID is issued.

    Article states it's anonymous and no info on the user is ever seen.
    watto_cobra
  • Reply 18 of 74
    anantksundaramanantksundaram Posts: 20,403member
    bonobob said:
    This is not good.

    The issue has nothing to do with whether they are anonymized or not. The issue is permissions and disclosure. Period.

    I expect we'll hear from Apple more on this.
    I suspect you agreed to what Apple is doing when you clicked through the terms & conditions as you set up your iDevice.  I know I did.
    If you know you did, it would be helpful and educational if you could post the relevant section.
    mike54
  • Reply 19 of 74
    anantksundaramanantksundaram Posts: 20,403member
    This is not good.

    The issue has nothing to do with whether they are anonymized or not. The issue is permissions and disclosure. Period.

    I expect we'll hear from Apple more on this.
    FTA: "The main thrust of the report claims that Apple does not explicitly disclose to consumers that recordings are passed along to contractors -- but Apple does tell users that some queries are manually reviewed, and has since the release of the service. Despite the information having been public-facing for at least six iterations of iOS, the "whistleblower" advised that they were concerned over the lack of disclosure, especially considering the contents of some recordings containing "extremely sensitive personal information."
    Where does it say that it is manually reviewed by outside contractors? Could you provide a link to that specific wording? Thanks.
Sign In or Register to comment.