New video shows what the iPad Pro's LIDAR scanner is capable of

Posted:
in iPad edited March 2020
A new demonstration video has surfaced, showing how the new iPad Pro, with the help of its powerful new LIDAR scanner, can incorporate itself into the real world.

Apple has included a LiDAR scanner on the back of the 2020 iPad Pro
Apple has included a LiDAR scanner on the back of the 2020 iPad Pro


Apple on Wednesday unveiled two new iPad Pro models that come equipped with a LiDAR Scanner, which will offer major improvements to ARKit and photography.

Now, a new video has emerged, highlighting all the incredible things made possible by the new LIDAR scanner. In one clip, a user uses the LIDAR scanner and the Complete Anatomy app to measure the range of motion in someone's arm in real-time.

Another clip shows how with an iPad Pro and a bit of free space, a user could turn their living room into an immersive game of Hot Lava.

View this post on Instagram

A post shared by Rene Ritchie (@reneritchie) on Mar 18, 2020 at 6:58pm PDT

The Shapr3D app shows how the iPad Pro can scan a room and convert it into an accurate 3D model, which then can be edited and placed back into the AR space. It also shows off an improved AR-based retail experience, as a user utilizes the Ikea Place app to pick out furniture for their home.

The pair of 2020 iPad Pro models and the new Magic Keyboard with Trackpad will launch on March 24, with pre-orders in process now. Broader mouse and trackpad support are coming in iPadOS 13.4, which arrives on March 24 as well.

Comments

  • Reply 1 of 14
    This is really sick!! I can't wait to get my hands on this iPad Pro, good thing I waited last year going from 2017 iPad Pro 10.5 to last year's model. The LIDAR is a real game-changer as AR/VR meet worlds. I play a ton of games in VR and always wondering how I can bring real spaces into virtual reality. This is one step closer!
    jony0watto_cobra
  • Reply 2 of 14
    dewmedewme Posts: 5,362member
    Wow. The potential uses of this technology are bounded only by human imagination. About the only challenge I see is that holding the iPad Pro looks a bit uncomfortable. I’d imagine you could use a tripod system, maybe one with a stabilized gimbal, or even a motorized gimbal, that can be used to capture a 360 degree 3D model of a space or scene. Putting this technology into a iPad Mini would be awesome. 
    edited March 2020 watto_cobra
  • Reply 3 of 14
    sphericspheric Posts: 2,560member
    Is this really relevant to the mainstream though? 

    I’m not knocking the tech; I’m asking because I have not seen any use case example so far that is at all relevant to my personal experience. 

    It’s nice to be able to see the new Mac Pro in VR sitting on the bathroom floor, but beyond that, the technology hasn’t enriched my life. 

    Gaming I get, home improvement and 3D models, I get — but those really aren’t mass interests, are they? 
    macpluspluswatto_cobra
  • Reply 4 of 14
    spheric said:
    Is this really relevant to the mainstream though? 

    I’m not knocking the tech; I’m asking because I have not seen any use case example so far that is at all relevant to my personal experience. 

    It’s nice to be able to see the new Mac Pro in VR sitting on the bathroom floor, but beyond that, the technology hasn’t enriched my life. 

    Gaming I get, home improvement and 3D models, I get — but those really aren’t mass interests, are they? 
    Have you seen how popular IKEA is? Have you been to one on a weekend? Potentially every time every person or family visits one is a session or sessions of using AR beforehand to help decide which piece of furniture to buy. The entire high tech industry has been built on “we don’t know all that folks will use this for yet”. The Apple Watch wasn’t initially introduced as a health device. Right now it requires an iPad or iPhone to use as a “universe window” (props to the tv show ”Fringe”), but a whole new set of uses become available when it’s more passive (glasses, for example).
    Rayz2016StrangeDaysfastasleepwatto_cobra
  • Reply 5 of 14
    spheric said:
    Is this really relevant to the mainstream though? 

    I’m not knocking the tech; I’m asking because I have not seen any use case example so far that is at all relevant to my personal experience. 

    It’s nice to be able to see the new Mac Pro in VR sitting on the bathroom floor, but beyond that, the technology hasn’t enriched my life. 

    Gaming I get, home improvement and 3D models, I get — but those really aren’t mass interests, are they? 
    I'm with you on this and have been saying for years that the tech is cool. "See what the developers do", is the usual line. Well, I'm still waiting. Sure there are a few apps that utilize ARKit but even fewer do it well. The on-stage demos are always neat but that's almost as far as it goes. As far as gaming is concerned (or that model of Apple Park in that allow an iPad to get more detail using ARKit), I don't see how playing a game that looks like it is on my dining room table can't be played basically the same without the AR view. What does the AR view do to complement the game?

    spheric said:
    Is this really relevant to the mainstream though? 

    I’m not knocking the tech; I’m asking because I have not seen any use case example so far that is at all relevant to my personal experience. 

    It’s nice to be able to see the new Mac Pro in VR sitting on the bathroom floor, but beyond that, the technology hasn’t enriched my life. 

    Gaming I get, home improvement and 3D models, I get — but those really aren’t mass interests, are they? 
    Have you seen how popular IKEA is? Have you been to one on a weekend? Potentially every time every person or family visits one is a session or sessions of using AR beforehand to help decide which piece of furniture to buy. The entire high tech industry has been built on “we don’t know all that folks will use this for yet”. The Apple Watch wasn’t initially introduced as a health device. Right now it requires an iPad or iPhone to use as a “universe window” (props to the tv show ”Fringe”), but a whole new set of uses become available when it’s more passive (glasses, for example).
    You're right, tons of people go to IKEA. Aside from the fact that, outside of tech circles, I have never heard anyone mention AR on their phone or iPad, most people don't even know the functionality exists to preview furniture in their home. 

    I think the point @spheric is trying to make is that nothing you or this article mentions is driving mass market demand. Do you know of anyone who has said something along the lines of, "Holy cow! Have you seen the AR preview of furniture in IKEA's app?!?! I need to get a new iPhone because I want to do that!!" I haven't. On the other hand, it is not uncommon at all when a new iPhone gets released that I hear people say they want the new camera capabilities. I don't have any hard facts but I'd be willing to bet that significantly more people purchased a new iPhone for the camera than they did to be able to have ARKit 2.

    I agree with you that all this ARKit stuff is just setting the foundation for glasses, in-car HUD-type things or something else that hasn't been leaked. Doing most of these things with an iPhone or iPad is just too cumbersome, which is why we don't see people doing it or lots of apps coming out with AR.
    spheric
  • Reply 6 of 14
    bulk001bulk001 Posts: 764member
    I will admit that my initial comments on AR in iOS devices was short sighted. The addition of lidar and the ability to tie it to things like arm movement or room layout is very powerful. Putting a couch into the   Champs-Élysées didn’t seem that useful. To the developers who saw the potential and Apple for adding it, good on you! Thanks for the video link BTW very interesting. 
    coolfactorjony0
  • Reply 7 of 14
    coolfactorcoolfactor Posts: 2,241member
    dewme said:
    Wow. The potential uses of this technology are bounded only by human imagination. About the only challenge I see is that holding the iPad Pro looks a bit uncomfortable. I’d imagine you could use a tripod system, maybe one with a stabilized gimbal, or even a motorized gimbal, that can be used to capture a 360 degree 3D model of a space or scene. Putting this technology into a iPad Mini would be awesome. 

    This is a step in the direction towards AR glasses. You know it.

    fastasleep
  • Reply 8 of 14
    StrangeDaysStrangeDays Posts: 12,877member
    Every time I remodel a room in my house I want an app that produces a model of the room I’m in, then allows me to edit the dimensions to actuals, and put in standard/common elements like cabinets, counters, shelves, closets, furniture, etc. Would be useful for general reno design, and materials estimates.

    Shapr3D looks like it does this, but it’s a full CAD app for mechanical engineers, so I fear it will be too complicated. Something easy designed for interior space would be great (but I’ll check it out...oh yay, expensive monthly rental payments)
    edited March 2020 watto_cobra
  • Reply 9 of 14
    sphericspheric Posts: 2,560member
    spheric said:
    Is this really relevant to the mainstream though? 

    I’m not knocking the tech; I’m asking because I have not seen any use case example so far that is at all relevant to my personal experience. 

    It’s nice to be able to see the new Mac Pro in VR sitting on the bathroom floor, but beyond that, the technology hasn’t enriched my life. 

    Gaming I get, home improvement and 3D models, I get — but those really aren’t mass interests, are they? 
    Have you seen how popular IKEA is? Have you been to one on a weekend? Potentially every time every person or family visits one is a session or sessions of using AR beforehand to help decide which piece of furniture to buy. The entire high tech industry has been built on “we don’t know all that folks will use this for yet”. The Apple Watch wasn’t initially introduced as a health device. Right now it requires an iPad or iPhone to use as a “universe window” (props to the tv show ”Fringe”), but a whole new set of uses become available when it’s more passive (glasses, for example).
    Yes, I've been to Ikea. How many times does one remodel a room during the lifespan of an iDevice? It's really not something that is really relevant to most people more than like once a year. Maybe. 

    I agree that use cases evolve as technology becomes adopted, and your example with the Apple Watch is a good one. 

    It's just that AR has been around for quite some time now, and I have yet to see anything that clicks, where I think "Oh yeah, people are gonna really want/need that!" the way the fitness aspect did with the Apple Watch, which then morphed into this massive health push. 

    The fitness tracker part isn't part of my life, but I understood immediately why it would be important to millions of people. I didn't even see the health part, with heart tracking etc. at the time.

    I'm not seeing anything comparable with AR tech at this point. 

    What makes most sense to me is the idea that this is just transitional tech until we get AR glasses to the mainstream. But I'm skeptical of that, given how the public reacted to Google's fucked-up attempt.
    watto_cobra
  • Reply 10 of 14
    thttht Posts: 5,443member
    Is this AR convincing to you?



    Not bad. Probably needs ray tracing to hit a sweet spot. The LiDAR feature will probably improve the sizing and positioning better. It's bigger than it should be.

    I could only move the AR object in-display. The LiDAR may get good enough for my to move it in virtual space? That will get freaky. The glasses will make the reality bending even better if they can get that to work.

    The folding display form factors are cool and all, but if AR glasses get good enough, good enough that you can manipulate virtual objects in "virtual" space, watch out. If I can type on that AR keyboard, that's when you begin to think the next big think is AR glasses. A trillion dollar big thing.
    watto_cobra
  • Reply 11 of 14
    sphericspheric Posts: 2,560member
    Okay, now THAT is getting interesting. The point where computing drops the two-dimensional display entirely and moves into 3D space, integrated with your surroundings through AR interface on glasses. 

    Probably a decade off, though, no?
    watto_cobra
  • Reply 12 of 14
    thttht Posts: 5,443member
    spheric said:
    Okay, now THAT is getting interesting. The point where computing drops the two-dimensional display entirely and moves into 3D space, integrated with your surroundings through AR interface on glasses. 

    Probably a decade off, though, no?
    The scenario of typing on an AR keyboard? I don't know. There are systems that do this today with projected keyboards and some kind of shadow based photogrammetry, and it's not new technology. It's an old idea. But everything miniaturized in a pair of super light, battery powered glasses sounds challenging.

    A minimum viable AR glasses product could be one that shows a AR display next to your laptop display. You control what's on that AR display with a hardware keyboard and mouse, and maybe even virtual touching of the AR display (AR touch can work for real displays too). There are other features like navigation, all the status information that the Apple Watch shows, and yes, the boring interior decorating stuff. The laptop or phone will be doing all the number crunching. The glasses is just a display.

    Latency is going to be a big thing to tackle. LiDAR may be able to precisely place the AR display so that it appears rock solid next to the real display. People will be wasting energy and just have videos on the side playing like background noise.

    ihatescreennames
  • Reply 13 of 14
    SpamSandwichSpamSandwich Posts: 33,407member
    The LIDAR could allow moviemakers to do live background replacement, tracking and special effects which would rival theatrical releases. Very interesting.
  • Reply 14 of 14
    y2any2an Posts: 187member
    Who’s to say this is only on the iPad? Imagine this on a phone for games, and throw in AR glasses...
    watto_cobra
Sign In or Register to comment.