Facebook's attempted 'revenge porn' solution: Trust us with your nude pics

Posted:
in General Discussion edited November 2017
Social networking giant Facebook is trialing a unique and likely controversial solution for the internet's "revenge porn" problem -- asking some users to send themselves their own naked images via Facebook Messenger, so the company's algorithms can anonymously scan and block sharing of those pics in the future.




Facebook's efforts are being spearheaded in Australia with the aid of the eSafety Office, which is looking to curb "image-based abuse" before it even occurs. In an interview with ABC News, eSafety Commissioner Julie Inman Grant said people who are worried their private photos may be shared on Instagram or Facebook can contact her office, and they may be asked to send the images to themselves on Facebook Messenger.

"It would be like sending yourself your image in email, but obviously this is a much safer, secure end-to-end way of sending the image without sending through the ether," Inman Grant said.

By sending the image through Facebook's servers, the company's artificial intelligence and photo matching technologies are said to be able to create a digital fingerprint for racy images, without actually saving the picture itself.

A Facebook official told ABC that four countries are currently testing the "industry-first" pilot.

So-called "revenge porn," typically in reference to images captured consensually in a relationship but shared by a vengeful ex after a breakup, has become a major problem on the internet, further exacerbated by social networking sites that make it easy to disseminate and shame. Often by the time the victim is able to have the image removed, the damage has already been done.




A study published in late 2016 by the Data & Society Research Institute found that one in 25 Americans were a victim of "revenge porn." The most popular target is younger women, where one in 10 under the age of 30 have experienced threats of nonconsensual image sharing.

Apple has had its own hurdles in addressing the problem, as dozens of celebrities who were victims of iCloud account hacking have had private photos shared across the internet. Though Apple's own security was not to blame, users showed poor password practices combined with automatic uploads of images to the iCloud Photo Library, putting sensitive images on the web in a vulnerable fashion.

The Federal Bureau of Investigation continues to look into the so-called "Celebgate" iCloud hacking scandal, and to date three people have been charged with the crimes committed. Two of them have already been found guilty and sentenced to prison.
«134

Comments

  • Reply 1 of 66
    What could possibly go wrong?
    longpathsdw2001OutdoorAppDeveloperzroger73baconstangDavidAlGregorydysamoriajony0watto_cobracornchip
  • Reply 2 of 66
    So Facebook wants people to donate their own pictures to Facebook's porn collection.
    Then when a disgruntled employee or hacker hacks their server, your nude pictures are going to be released to the world.

    longpathmacseekerzroger73baconstangDavidAlGregorywatto_cobracornchipbeowulfschmidtSpamSandwich
  • Reply 3 of 66
    WOW indeed. Surely, if the AI is smart enough to recognise naked t*ts or *ss for example, then should it not automatically block that content? They don't need to verify WHO the person was surely.

    I don't get the whole sexting thing. Surely young people these days are clued up on how the internet works and putting something online is pretty much irreversible, regardless of where and to whom they sent it, and whether or not it was "meant to be private", as if there is such a thing.
  • Reply 4 of 66
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    caliunphocuscornchiprandominternetpersonSpamSandwich
  • Reply 5 of 66
    nhughesnhughes Posts: 770editor
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. There are plenty of other forums to discuss these matters, including Twitter. It's our house, those are our rules.

    As for this story, I think it's pretty clear that anyone who would post "revenge porn" to hurt someone else is a despicable human being. Anyone here who disagrees with that basic assessment can feel free to let me know and get a swift ban.
    peterharttallest skilfotoformatking editor the gratelongpathmknelsononeof52OutdoorAppDeveloperzroger73macseeker
  • Reply 6 of 66
    So go ahead and trust FB with those nude pics that you don't what anyone else to see? Lol.
    watto_cobracornchip
  • Reply 7 of 66
    So Facebook wants people to donate their own pictures to Facebook's porn collection.
    Then when a disgruntled employee or hacker hacks their server, your nude pictures are going to be released to the world.

    Yes but in a tasteful manner. 
    peterhartbaconstangwatto_cobracornchip
  • Reply 8 of 66
    calicali Posts: 3,494member
    Imagine if this was an Apple story........

    nhughes said:
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. There are plenty of other forums to discuss these matters, including Twitter. It's our house, those are our rules.

    As for this story, I think it's pretty clear that anyone who would post "revenge porn" to hurt someone else is a despicable human being. Anyone here who disagrees with that basic assessment can feel free to let me know and get a swift ban.

    I think abusing your power is despicable. I research gender inequality daily and see men treated like sh** in public, in law, in relationships, online, everywhere. I see comments like yours where it’s only bad because the “victim” is a female who did something first to hurt a man.

    I won’t disclose my gender but it’s not fair to attack one gender while giving a pass to the other for whatever reason.
  • Reply 9 of 66
    nhughesnhughes Posts: 770editor
    cali said:
    Imagine if this was an Apple story........

    nhughes said:
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. There are plenty of other forums to discuss these matters, including Twitter. It's our house, those are our rules.

    As for this story, I think it's pretty clear that anyone who would post "revenge porn" to hurt someone else is a despicable human being. Anyone here who disagrees with that basic assessment can feel free to let me know and get a swift ban.

    I think abusing your power is despicable. I research gender inequality daily and see men treated like sh** in public, in law, in relationships, online, everywhere. I see comments like yours where it’s only bad because the “victim” is a female who did something first to hurt a man.

    I won’t disclose my gender but it’s not fair to attack one gender while giving a pass to the other for whatever reason.
    I'm not really sure what this comment is about, as I didn't even specify a gender in my comment, yet you seem to be accusing me of saying otherwise.

    I did, however, specify a gender in the story, simply noting that younger women under the age of 30 are by far the most likely to be victims of "revenge porn." That's a fact backed by statistics.

    In no way did I ever, at any point, suggest that "revenge porn" is somehow less of a crime if a woman were to do it to a man. Not sure where you got that idea.
    edited November 2017 king editor the gratelongpathoneof52baconstangbonobobrhinotuffstompydysamoriawatto_cobracornchip
  • Reply 10 of 66
    TBH this plan makes a lot of sense.

    You could add a thing to the Facebook or Messenger app that reads a photo, makes a hash (the "signature" as described by the article), and then sends that string back to Facebook. Building that hash in the phone is child's play, no need for the pix to ever reach the server.

    Keeping Facebook honest is easy enough. Tons of folks will be doing packet scanning on their home routers; even over SSL it would be obvious when the app is sending a multi-megabyte JPEG instead of a 2kb hash string.

    On the UI side, you surface this feature as some kind of "safety" button, the same way Chrome and Safari have their winkingly named "incognito mode." 

    ("By sending the image through Facebook's servers, the company's artificial intelligence and photo matching technologies are said to be able to create a digital fingerprint for racy images, without actually saving the picture itself." It's funny how they're playing this off as some kind of high tech thing, though. Calculating similarity between two differently hashed images isn't trivial but it's not rocket science either.)
    edited November 2017
  • Reply 11 of 66
    retrogustoretrogusto Posts: 1,109member
    Until my TrueDepth camera can do this automatically, I’m just going to have the Amazon delivery guy send them from my computer when he’s over to deliver my packages.  
  • Reply 12 of 66
    nhughes said:
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. There are plenty of other forums to discuss these matters, including Twitter. It's our house, those are our rules.

    As for this story, I think it's pretty clear that anyone who would post "revenge porn" to hurt someone else is a despicable human being. Anyone here who disagrees with that basic assessment can feel free to let me know and get a swift ban.
    If Google's algorithm is policing the site, maybe you could hire their algorithms to police the forums. I am kidding: both Google's and Facebook's heavily reliance on algorithms, (which can be easily manipulated) is why they're on Capitol Hill answering questions from angry senators. I don't believe an AI (Artificial Intelligence) is yet capable of nuanced contextual reading of forum comments to where they can discern the writer's probable intent. Dead pan sarcasm, for example.

    I know the coders at both Google and Facebook have an unnatural trust in their code, which is why Facebook think's it's completely rational and socially acceptable to ask its members to send nude photos of themselves to Facebook for machine learning. You think Apple would ever do something so tone-deaf and bone-headed?

    I also don't think Google should be policing AppleInsider unless they want to sell you their policing services as some kind of API you can use to evaluate posted comments as they are posted (and not after the fact). But, as I said, I don't trust any AI (artificial intelligence) to do this with enough nuance as to be useful. As far as I'm concerned, Google wasn't deputized to be the Internet Police, and their threats should not create a chilling effect on smaller websites, but unfortunately, that appears to be the case here. I am fully aware of sites' need to play nice with Google.
    edited November 2017 nhugheslongpathmacseekermacky the mackymontrosemacscornchip
  • Reply 13 of 66
    StrangeDaysStrangeDays Posts: 12,834member
    nhughes said:
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. 
    Can you shed light on this Google warning? I’m a developer but I can’t think of what sort of warning Google would send your site, especially considering the type of political bickering is generally pretty tame and doesn’t contain graphic images of beheadings or whatnot. Surely the bickering is no different than on popular news sites all over the web, including other leading Apple rumor sites?

    House-rules is fine, but I’m interested in what the Google concern is. 
    edited November 2017 watto_cobraanantksundaram
  • Reply 14 of 66
    nhughes said:
    (we actually got a warning from Google regarding offensive content in the comments)
    I understand fully, from a business perspective, why AI’s decision on this matter has occurred. I ask you (and others) personally, though: WHO THE FUCK IS GOOGLE TO SAY WHAT WE CAN AND CANNOT DISCUSS? Google is the arbiter of speech itself, with powers far beyond that of any government (yet dictated to them, in part, thereby). This is beyond reproach.
    There are plenty of other forums to discuss these matters, including Twitter.
    They don’t allow political discussion either.  :p
    macseekerwatto_cobraanantksundarambestkeptsecretrandominternetpersonjcs2305
  • Reply 15 of 66
    nhughesnhughes Posts: 770editor
    nhughes said:
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. 
    Can you shed light on this Google warning? I’m a developer but I can’t think of what sort of warning Google would send your site, especially considering the type of political bickering is generally pretty tame and doesn’t contain graphic images of beheadings or whatnot. Surely the bickering is no different than on popular news sites all over the web, including other leading Apple rumor sites?

    House-rules is fine, but I’m interested in what the Google concern is. 
    nhughes said:
    (we actually got a warning from Google regarding offensive content in the comments)
    I understand fully, from a business perspective, why AI’s decision on this matter has occurred. I ask you (and others) personally, though: WHO THE FUCK IS GOOGLE TO SAY WHAT WE CAN AND CANNOT DISCUSS? Google is the arbiter of speech itself, with powers far beyond that of any government (yet dictated to them, in part, thereby). This is beyond reproach.
    I wasn't involved in the receipt of the warning, but our developer mentioned it here:

    https://forums.appleinsider.com/discussion/202133/were-removing-political-outsider

    You wouldn't believe the number of emails and tweets we receive from people who are outraged (usually incorrectly saying something about us stifling free speech) when they see that we have turned off comments on a political article. I think these people have some sort of vision of AppleInsider as being owned by a mega corporation, with a big headquarters and a huge staff and tens of millions of dollars backing us. That's just not the case.

    The truth is, we have a small staff, we do the best we can, and we can't spend our time policing the forums. That's really all there is to it — no agenda and no conspiracy at play.
    king editor the gratelongpathdysamoriauktechiewatto_cobra
  • Reply 16 of 66
    Even if only a hash is sent instead of the full picture, this is still a bad idea, given that this is all algorithm..

    Because people will use this for any picture they want to see blocked, not just nude pictures of themselves..

    Don't like Obama? Have ask his pictures blocked.. Don't like Taylor Swift? Get her banned..

    All nice ideas in theory if people use it in good faith.. Reality is far less benevolent unfortunately..
    llamawatto_cobra
  • Reply 17 of 66
    sdw2001sdw2001 Posts: 18,015member
    nhughes said:
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. There are plenty of other forums to discuss these matters, including Twitter. It's our house, those are our rules.

    As for this story, I think it's pretty clear that anyone who would post "revenge porn" to hurt someone else is a despicable human being. Anyone here who disagrees with that basic assessment can feel free to let me know and get a swift ban.

    You got a warning about a four year old thread where someone posted graphic images.  So, that's nonsense.  It was an excuse to delete PoliticalOutsider, which had been around for 15+ years.  You now post stories with some political content and disallow comments because some people engage in ad hominem attacks.  This doesn't foster discussion.  It doesn't help your audience engage.  AI was once a thriving community for Apple rumors, products, news, information and unrelated discussion.  It's now nothing but a news site.  Anyone here as long as I have been knows exactly what I'm talking about.  
    ben20anantksundaramStrangeDays
  • Reply 18 of 66
    sdw2001sdw2001 Posts: 18,015member

    Eric_WVGG said:
    TBH this plan makes a lot of sense.

    You could add a thing to the Facebook or Messenger app that reads a photo, makes a hash (the "signature" as described by the article), and then sends that string back to Facebook. Building that hash in the phone is child's play, no need for the pix to ever reach the server.

    Keeping Facebook honest is easy enough. Tons of folks will be doing packet scanning on their home routers; even over SSL it would be obvious when the app is sending a multi-megabyte JPEG instead of a 2kb hash string.

    On the UI side, you surface this feature as some kind of "safety" button, the same way Chrome and Safari have their winkingly named "incognito mode." 

    ("By sending the image through Facebook's servers, the company's artificial intelligence and photo matching technologies are said to be able to create a digital fingerprint for racy images, without actually saving the picture itself." It's funny how they're playing this off as some kind of high tech thing, though. Calculating similarity between two differently hashed images isn't trivial but it's not rocket science either.)

    Yeah, sending nudes to Facebook makes tons of sense.  I mean, seriously.  I'm not exactly a privacy freak, but you want to send naked pictures of yourself to a media giant?  This might be the single dumbest idea ever in technology.  
    ibillben20dysamoriamacky the mackyuktechiewatto_cobra
  • Reply 19 of 66
    sdw2001sdw2001 Posts: 18,015member

    nhughes said:
    nhughes said:
    Oh how nice.  We can post comments here but the one yesterday on the locked phone for the Texas shooter was closed because it was too political.  Porn is more appropriate than politics I guess. 
    Commenters have repeatedly shown that they cannot behave or be respectful toward one another or the site when the topic is a hot button political issue. Rather than constantly policing it, we decided to shut down comments on those stories. The comments degrade the quality of the site (we actually got a warning from Google regarding offensive content in the comments), and it is not worth our time. 
    Can you shed light on this Google warning? I’m a developer but I can’t think of what sort of warning Google would send your site, especially considering the type of political bickering is generally pretty tame and doesn’t contain graphic images of beheadings or whatnot. Surely the bickering is no different than on popular news sites all over the web, including other leading Apple rumor sites?

    House-rules is fine, but I’m interested in what the Google concern is. 
    nhughes said:
    (we actually got a warning from Google regarding offensive content in the comments)
    I understand fully, from a business perspective, why AI’s decision on this matter has occurred. I ask you (and others) personally, though: WHO THE FUCK IS GOOGLE TO SAY WHAT WE CAN AND CANNOT DISCUSS? Google is the arbiter of speech itself, with powers far beyond that of any government (yet dictated to them, in part, thereby). This is beyond reproach.
    I wasn't involved in the receipt of the warning, but our developer mentioned it here:

    https://forums.appleinsider.com/discussion/202133/were-removing-political-outsider

    You wouldn't believe the number of emails and tweets we receive from people who are outraged (usually incorrectly saying something about us stifling free speech) when they see that we have turned off comments on a political article. I think these people have some sort of vision of AppleInsider as being owned by a mega corporation, with a big headquarters and a huge staff and tens of millions of dollars backing us. That's just not the case.

    The truth is, we have a small staff, we do the best we can, and we can't spend our time policing the forums. That's really all there is to it — no agenda and no conspiracy at play.

    The forums were well moderated years ago...by members with moderator privileges.  What you've done now is stifle discussion. And while I'm sure the staff is small, AI is indeed acting like a corporate conglomerate.  "Corporate" is the perfect word.   There was a time that the only things ever taken down were the result of Cease and Desist letters from Apple.  Now Google sends you a nasty e-mail and you delete an entire forum with hundreds of thousands of replies.   
    anantksundarampscooter63StrangeDays
  • Reply 20 of 66
    linkmanlinkman Posts: 1,035member
    Facebook/Instagram might be successful in blocking these images from its servers, leaving about 10,000 other web sites available for posting those revenge photos.
    bonobobdysamoriasuddenly newton
Sign In or Register to comment.