Latest demos with iOS 11 ARKit show plated food, 3D sculpting with Apple Pencil

Posted:
in iPhone edited December 2019
Two new ARKit demonstrations by developers are showing the rapid advancement of what is possible with the new technology, with one showing food on a plate prior to ordering, and the other illustrating sculpting and painting with the iPad for use in other AR applications.




The demonstration of a new sculpting app illustrates a face being crafted in the forthcoming Maker Studio, including color and texture.





Potentially more impressive is a proof of concept by Kabaq. The demonstration depicted a series of very-near photographic quality foods, including two desserts and a hamburger on a plate.





Properly executed, the app can give the user a good feel for not only appearance of the food, but portion size as well. Beyond just menus, the company notes that the app can be used for cookbooks, and wider marketing initiatives as well.

These demos, and many more can be found on the MadewithARKit Twitter account. Updated frequently, the feed curates the best of what it sees, and appears to be the most frequently updated stream at this time.

Apple Vice President Craig Federighi announced ARKit during the 2017 WWDC keynote. The project contains a developer toolset that it will make available to nearly instantaneously make the iPhone and iPad the largest AR platform in the world, according to the company.

In a demonstration of software produced by the new ARKit, the software identified a table surface, and applied a virtual coffee cup properly scaled to the surface. Following a lamp's addition to the surface, the lighting model adjusted dynamically as Federighi moved the lamp around the cup.

A statement in Apple's developer information limits compatibility to A9 processor and newer, with little other amplifying information. Apple notes that the "breakthrough performance" in the two processors allow for "fast scene understanding" without breaking down why the A8 can't technically accomplish the same feat.
«1

Comments

  • Reply 1 of 24
    foggyhillfoggyhill Posts: 4,767member
    At will explode in the 2 year and again it will be thanks to apple's large ecosystem of phones with similar capabilities and giving devs  the tools to easily put ar in front of those people. 

    This will of course help everyone involved in the field, even ms and google.
    tmaywatto_cobra
  • Reply 2 of 24
    My first thought on the menu AR thingy: cool. And then the real plate arrives and you start crying ;)
    SpamSandwichGG1fastasleepwatto_cobrabonobobcornchip
  • Reply 3 of 24
    radarthekatradarthekat Posts: 3,842moderator
    2018 might just be dubbed, The Year of Augmented Reality.  With Apple leading the charge.
    Metriacanthosaurustmaydoozydozenmacky the mackyslprescottwatto_cobra
  • Reply 4 of 24
    2018 might just be dubbed, The Year of Augmented Reality.  With Apple leading the charge.
    Mmm...

    I wonder if the [yet-to-be-announced] New AppleTV will have the requisite A9 chip and directly support ARKit (instead of via AirPlay from an iDevice)...


    tmayjony0doozydozenwatto_cobracornchip
  • Reply 5 of 24
    SpamSandwichSpamSandwich Posts: 33,407member
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 
    edited August 2017 doozydozenbonobob
  • Reply 6 of 24
    jSnivelyjSnively Posts: 429administrator


    But ... can it do that? :)


    doozydozen
  • Reply 7 of 24
    Oh, boy, that dirty iPad must have tasted yummy to that toddler. I have to say that those virtual meals certainly do look appetizing. I'm licking my drool away as I type.
    edited August 2017 watto_cobra
  • Reply 8 of 24
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 
    A video clip would be just as good? Hey, why are you trying to kill the virtual everything buzz? You must realize Apple and a host of other companies are spending billions of dollars on the next new thing. A video clip is so 90's. All these new expensive technologies must be developed so consumers will spend lots more money. VR goggles for everyone is a must. /s
  • Reply 9 of 24
    jdgazjdgaz Posts: 403member
    A picture is worth a thousand words.
  • Reply 10 of 24
    My first thought on the menu AR thingy: cool. And then the real plate arrives and you start crying ;)
    Just keep looking at the virtual image while you're eating and you won't feel so bad.
    doozydozenfastasleepwatto_cobracornchip
  • Reply 11 of 24

    I didn't realize it, but with iOS 11 Apple kinda' included AR in the Apple Maps app.





    What you need to do is:
    1. locate a city or area that has 3d Flyover
    2. select Flyover from the menu
    3. do not select Start [City] Tour

    You are in a 3D virtual tour with AR Heads-up highlights.  You can navigate by walking around...  But even better, you can pan and zoom with your fingers from the comfort of your chair.

    It's a little clunky -- but it works!

    It would be great if you could use Siri to refine Flyover, say, define a custom tour -- tell Siri where you want to go, the route you wish to take, how fast you want to travel, etc.

    Here's a a video that illustrates what I mean -- a tour of Sydney Harbor featuring the Opera House.  (This video was made over 4 years ago, and Flyover has improved quite a bit).



    watto_cobra
  • Reply 12 of 24
    coolfactorcoolfactor Posts: 2,239member
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 

    How does one customize what is viewed in a video? Recording *every* possible combination that a customer might choose? Not only would that be expensive time wise, but what if you added *one* new ingredient option? Then you'd need to record several more videos including that ingredient. It would be exponentially insane!

    AR solves that.
    doozydozenwatto_cobra
  • Reply 13 of 24
    tmaytmay Posts: 6,312member
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 
    I'm old, restaurants are poorly lit, and generally understaffed. Giving me a device with AR images to look at, and with text befitting my poor eyesight, isn't going add any time to the dining experience, and may facilitate increased sales. If anything, it might improve the experience for fussy eaters that need to have their order defined precisely.
    watto_cobra
  • Reply 14 of 24
    SpamSandwichSpamSandwich Posts: 33,407member
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 

    How does one customize what is viewed in a video? Recording *every* possible combination that a customer might choose? Not only would that be expensive time wise, but what if you added *one* new ingredient option? Then you'd need to record several more videos including that ingredient. It would be exponentially insane!

    AR solves that.
    Customize a sandwich in AR? That would be idiotic. The point of looking at a menu is to order the food. Screwing around with a virtual plate of food would be an insane impediment to doing that.

    Profitability is the difference between a restaurant that succeeds and one that fails.
    edited August 2017
  • Reply 15 of 24
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 

    How does one customize what is viewed in a video? Recording *every* possible combination that a customer might choose? Not only would that be expensive time wise, but what if you added *one* new ingredient option? Then you'd need to record several more videos including that ingredient. It would be exponentially insane!

    AR solves that.
    Build your own Pizza -- on a much higher level... Now, if they'd make the crust extra-crispy so it could carry the weight of a garbage pizza...

    Or for a different use:  A 3D weather map showing the possible effects of a category 4 hurricane hitting the Texas coast at various points; stalling with torrential rainfall; and potential evacuation route options for 3 Million plus people.
  • Reply 16 of 24
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 

    How does one customize what is viewed in a video? Recording *every* possible combination that a customer might choose? Not only would that be expensive time wise, but what if you added *one* new ingredient option? Then you'd need to record several more videos including that ingredient. It would be exponentially insane!

    AR solves that.
    Customize a sandwich in AR? That would be idiotic. The point of looking at a menu is to order the food. Screwing around with a virtual plate of food would be an insane impediment to doing that.

    Profitability is the difference between a restaurant that succeeds and one that fails.

    You make a good point...  Just tell the waiter:  "I'll have the same thing I had last time!"

    I can see some value in bringing an Open Kitchen to the customers' table to show how the different dishes are prepared -- especially at fine restaurants.

    A variant of this could even work for fast food -- with lots of menu items. There could be kiosks (maybe an iPad at each table) where the customers could visualize the menu items, choose their purchase, place the order, pay for it -- and get called when it is ready.  Less stress for the customers -- no waiting in line, etc.  More productive for the restaurant -- fewer cashiers, etc.  Not much need for AR in this example, tho.


  • Reply 17 of 24
    My first thought on the menu AR thingy: cool. And then the real plate arrives and you start crying ;)
    Just keep looking at the virtual image while you're eating and you won't feel so bad.
    Paving the way for AR glasses. ;)
    red pill anyone? Or the blue pill?
    cornchip
  • Reply 18 of 24
    jSnively said:


    But ... can it do that? :)


    All I want is my $2.00!
  • Reply 19 of 24
    kevin keekevin kee Posts: 1,289member
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 

    How does one customize what is viewed in a video? Recording *every* possible combination that a customer might choose? Not only would that be expensive time wise, but what if you added *one* new ingredient option? Then you'd need to record several more videos including that ingredient. It would be exponentially insane!

    AR solves that.
    Customize a sandwich in AR? That would be idiotic. The point of looking at a menu is to order the food. Screwing around with a virtual plate of food would be an insane impediment to doing that.

    Profitability is the difference between a restaurant that succeeds and one that fails.

    You make a good point...  Just tell the waiter:  "I'll have the same thing I had last time!"

    I can see some value in bringing an Open Kitchen to the customers' table to show how the different dishes are prepared -- especially at fine restaurants.

    A variant of this could even work for fast food -- with lots of menu items. There could be kiosks (maybe an iPad at each table) where the customers could visualize the menu items, choose their purchase, place the order, pay for it -- and get called when it is ready.  Less stress for the customers -- no waiting in line, etc.  More productive for the restaurant -- fewer cashiers, etc.  Not much need for AR in this example, tho.


    Good idea and I think McDonald here already have that although I hate their lagging touch screen. Now if I wanted to go further, like to customize my order, reducing the portion of certain elements of food, adding other elements, adding the quantities of say lettuce, etc. I think AR would solve that problem easily. How many times you tried to explain to the waitress what you wanted just to get the wrong one because he or she didn’t get you. A visual order will not just cut the time but get exactly what you wanted.
    edited August 2017 tmay
  • Reply 20 of 24
    MarvinMarvin Posts: 15,310moderator
    The ideal way to lose weight! Virtual food.

    As a practical matter, it makes no sense to waste time and money developing virtual foods when a simple video clip would suffice. There's no advantage for the customer and it would slow the ordering process.

    It's a solution looking for a problem, even though its eye-popping as a demo. 
    One advantage listed on the website is the ability to show portion size. Sometimes when you order something like a burger, it's bigger than you expect. Videos and photos aren't easy to see in proportion to your plate or hand. They have an interactive demo on their site:

    http://www.kabaq.io

    They also mention ingredients and cookbooks so you could have a recipe in a cookbook and get a preview of the dish on the table and compare how bad your attempt was. Supermarkets and online shopping can have this too because sometimes when you order by weight or get ingredient packs, it's not obvious what that looks like. What does 50g of yogurt look like vs 250g of cheese?

    The visuals are a bit more realistic in the setting you're in than video because they have real-time lighting and you can move around the models. Making the models is more time-consuming than video but if upcoming iOS devices have depth sensors, they can do 3D model acquisition almost as quickly as video. Picking shaders would be hard but some sensors would be able to build the shader from the way the light bounces off the object and they can have pre-made ones.

    It's hard enough to get pictures of food from restaurants so I doubt this more advanced presentation will be widely used but one of the hardest problems for restaurants is getting people in the door. If some tech can help with that, all the better. There will be a lot of AR experiments to see what will be practical uses for it.
Sign In or Register to comment.