Apple's first AI research paper focuses on computer vision

Posted:
in General Discussion edited December 2016
Apple last week published its first scholarly research paper, an article covering methods of improving recognition in computer vision systems, marking a new direction for the traditionally secretive company.




The paper, titled "Learning from Simulated and Unsupervised Images through Adversarial Training," was submitted for review in mid-November before seeing publication through the Cornell University Library on Dec. 22.

Apple's article arrives less than a month after the company said it would no longer bar employees from publishing research relating to artificial intelligence.

Spotted by Forbes on Monday, the first of Apple's public research papers describes techniques of training computer vision algorithms to recognize objects using synthetic, or computer generated, images.

Compared to training models based solely on real-world images, those leveraging synthetic data are often more efficient because computer generated images are usually labelled. For example, a synthetic image of an eye or hand is annotated as such, while real-world images depicting similar material are unknown to the algorithm and thus need to be described by a human operator.

As noted by Apple, however, relying completely on simulated images might yield unsatisfactory results, as computer generated content is sometimes not realistic enough to provide an accurate learning set. To help bridge the gap, Apple proposes a system of refining a simulator's output through "Simulated+Unsupervised learning."

In practice, this particular flavor of S+U learning combines unlabeled real image data with annotated synthetic images. The technique is based in large part on Generative Adversarial Networks (GANs), which applies two competing neural networks -- generator and discriminator -- against each other to better discern generated data from real data. A fairly recent development, the process has seen success in the generation of photorealistic "super-resolution" images.

While not necessarily presaging upcoming consumer technology, it is interesting to note Apple elected to apply its modified GAN to the evaluation of gaze and hand pose estimation. In addition, the company says it hopes to one day move S+U learning beyond static images to video input.

Apple's first public research paper was penned by vision expert Ashish Shrivastava and a team of engineers including Tomas Pfister, Oncel Tuzel, Wenda Wang, Russ Webb and Apple Director of Artificial Intelligence Research Josh Susskind. Of note, Susskind made the announcement of Apple's newfound interest in scholarly pursuits earlier this month, a move some believe will help future recruitment efforts.

Comments

  • Reply 1 of 16
    calicali Posts: 3,494member
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?
  • Reply 2 of 16
    While we are at it Apple.. Can we get the most basic of AI: Apple Spell Check and contextual word recognition working...beyond the Jurassic state it is in now. Many Thanks , in advance .
    edited December 2016 neo-techRayz2016StrangeDayspatchythepiratepalomine
  • Reply 3 of 16
    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?

    A lot of people working on cutting edge stuff want to publish and get recognition for their work. And people in specific fields might want to work for someone (like Apple) if they know more about things they're working on. Especially if it aligns with their own ideas.

    But that doesn't mean they're giving away ALL their secrets when they publish. This report is only a 15 page summary. I would think the data Apple has accumulated along with the source code for the software they're using would span 10's of thousands of pages. It's sort of like Apple explaining how their A Series processors work without giving away the actual blueprints to make one.
    edited December 2016 davenSoliRayz2016patchythepirate
  • Reply 4 of 16
    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?
    Apple's AI ambitions require the expertise of some talented PhDs on computer science. Good luck getting those kinds of people if they can't talk about any of their work, especially when there are so many jobs in computer science. 
    Rayz2016
  • Reply 5 of 16
    Hurts them if anything releasing the data I think. Look at the rumors on hardware, etc ahead of time. People think Apple is doing badly and behind times....yea because we all know what is coming ahead of time! We don't get the surprises like we used too.
  • Reply 6 of 16
    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?
    Knowledge begets knowledge.
  • Reply 7 of 16
    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?


    As stated by others, attracting the greatest talent means that Apple has to give them the leeway to talk about their work.

    I really wouldn't worry about people knowing these things. We have seen examples of people hearing rumors about what Apple is working on and rushing out products to beat Apple to the punch. The results were always pathetic.

    Remember the first Samsung Gear Watch? It has a freakin' camera on it and unaligned screws! Compare that to the Apple Watch and the difference is night and day.


    watto_cobracanukstorm
  • Reply 8 of 16
    cali said:
    imagine if the U.S. military published papers on what we were working on?
    They do all the time. Are you actually unaware of this? Look in google.

    You seem to have a complete lack of understanding how research works. If you don't share, you never know if you're simply fooling yourself. It's one of the most fundamental problems in science, and publishing is supposed to be the cure.
  • Reply 9 of 16
    So this is great and all, but maybe they should get Siri to work? Last night I tried to get it to call the local pizza shop here in Ajax Ontario, outside of Toronto...

    Me: Call pino's pizza
    Siri: do you mean Pinot's Pizza in Hamburg?
    M: Call pino's pizza in ajax
    S: Do you mean (something in toronto)
    M: Call pino's pizza in ajax
    S: I didn't find any matching restaurants.

    Go to google and type in "pinos pizza ajax". Now I'm entirely happy to give Siri the confusion between "pinos" and "pinots". But go to google and type in "pinots pizza ajax".

    Maps often completely ignores your current location when making suggestions. I'm sure you've all seen it suggest a location on the far side of the planet when you make even a trivial spelling mistake. Add to that that minor spelling mistakes are impossible to avoid when using voice, and that Apple's location database is based on Yelp and thus hopelessly limited, and you get the horrible mess that is Siri+Maps.

    So let's see them fix that, then I'll start reading their other papers.
    patchythepirate
  • Reply 10 of 16
    nhtnht Posts: 4,522member
    vvswarup said:
    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?
    Apple's AI ambitions require the expertise of some talented PhDs on computer science. Good luck getting those kinds of people if they can't talk about any of their work, especially when there are so many jobs in computer science. 
    There's a lot of dev work available. That's not the same thing. PhD's are constantly looking for grants and working with someone with deep pockets is often easier than writing a lot of proposals.

    In this case it's just keeping up with Google and MS vs academia.

    The military and three letter agencies significantly limit publishing too.
  • Reply 11 of 16
    nhtnht Posts: 4,522member
    cali said:
    imagine if the U.S. military published papers on what we were working on?
    They do all the time. Are you actually unaware of this? Look in google.

    You seem to have a complete lack of understanding how research works. If you don't share, you never know if you're simply fooling yourself. It's one of the most fundamental problems in science, and publishing is supposed to be the cure.
    A lot stuff ends up in the DTIC reports and there are papers and conferences but all the important stuff ends up in either classified briefings at very limited conferences or stays in a vault.

    Most of what you see in DoD conferences is really fluffy or really incomplete or basic science (6.1 stuff).  None of the cutting edge 6.2 stuff of importance gets out (materials, radars, etc).

    And you don't have to share to read what others are publishing.
  • Reply 12 of 16
    nhtnht Posts: 4,522member

    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?
    Knowledge begets knowledge.
    Fear leads to anger...
  • Reply 13 of 16
    flaneurflaneur Posts: 4,526member
    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?
    Knowledge begets knowledge.
    Exponentially.
  • Reply 14 of 16
    flaneurflaneur Posts: 4,526member

    nht said:

    cali said:
    I don't get what the advantage of spilling your work to the enemy has.

    imagine if the U.S. military published papers on what we were working on?
    Knowledge begets knowledge.
    Fear leads to anger...
    ...and then to alienation, which leads to more fear.

    The antidote to fear is knowledge.

    Rayz2016
  • Reply 15 of 16
    So this is great and all, but maybe they should get Siri to work? Last night I tried to get it to call the local pizza shop here in Ajax Ontario, outside of Toronto...

    Me: Call pino's pizza
    Siri: do you mean Pinot's Pizza in Hamburg?
    M: Call pino's pizza in ajax
    S: Do you mean (something in toronto)
    M: Call pino's pizza in ajax
    S: I didn't find any matching restaurants.

    Go to google and type in "pinos pizza ajax". Now I'm entirely happy to give Siri the confusion between "pinos" and "pinots". But go to google and type in "pinots pizza ajax".

    Maps often completely ignores your current location when making suggestions. I'm sure you've all seen it suggest a location on the far side of the planet when you make even a trivial spelling mistake. Add to that that minor spelling mistakes are impossible to avoid when using voice, and that Apple's location database is based on Yelp and thus hopelessly limited, and you get the horrible mess that is Siri+Maps.

    So let's see them fix that, then I'll start reading their other papers.


    Siri is just plain terrible.* How is Apple so blind to this. I've been taken across the world without spelling mistakes. I'm taken across the country frequently.. without making any spelling errors. Siri fails on day 1 promised functionality. It's absurd.

    I try to ask something like, "give me directions to my event tonight" and I got nonsense replies, even after asking 6 different ways. There are SO MANY examples of this, small things, but they happen all the time. If I wasn't an Apple fan I would give up on siri completely for anything other than making calendar additions and reminders, which it does pretty well (although I often have to go back and edit them). Siri's deficits are also glaringly obvious on Apple TV, where you need exact phrasing. They were just talking about it on the iMore podcast, how "Christmas movies" didn't bring up any results, but "holiday movies" did. It's beyond absurd how terrible that is.. it's so so basic.

    Why can't Apple understand that it doesn't do much good to make Siri more sophisticated, or give Siri more knowledge, if the #&%$ can't do basic things correctly.

    *FWIW, I'm not comparing Siri to alternative AIs, because I don't use them. But it doesn't matter if the other AI's are better or worse, Siri is inexcusably bad.

    edited December 2016
  • Reply 16 of 16
    nhtnht Posts: 4,522member
    flaneur said:

    nht said:

    cali  Knowledge begets knowledge.
    Fear leads to anger...
    ...and then to alienation, which leads to more fear.

    The antidote to fear is knowledge.

    Epic Star Wars reference fail.
Sign In or Register to comment.