Apple's push for 3D will require voxel map of the world, claims VR evangelist

Posted:
in Future Apple Hardware edited May 2021
Apple is preparing for another "paradigm shift" that embraces 3D, according to VR and AR evangelist Robert Scoble, with predictions that Apple's strategy involves creating a realtime 3D map of the entire planet, as well as more work on 3D audio and a headset like the often-rumored "Apple Glass."




Apple is thought to be working on "Apple Glass," wearable augmented reality or smart glasses that can provide users with a mix of digitally-produced and real-world views. While the bulk of rumors and speculation have revolved around the hardware, Robert Scoble has forecast the kind of user experience to expect from Apple over the coming years.

Starting at WWDC 2021, Apple will start to make announcements throughout the year about technological advances that it will be incorporating into its products, writes Scoble on Saturday. The announcements will apparently lead into "many new products, services, and experiences that will come for decades."

According to Scoble, this will be a fourth paradigm shift for Apple, following its introduction of the personal computer, the graphical user interface, and the iPhone. While many of the changes will center around headsets and glasses, Scoble believes it will be "way deeper" than the hardware, with Apple moving into a number of new areas.

The list of changes is vast, with arguably the most ambitious being a "realtime 3D map of the entire world." The map will be a "basis for the new Apple," and will be used to power its products and user experiences.

The sources, identified as "developers who are building for Apple," also apparently offered that many shipping products have elements that could help Apple's new strategy take form. Such items include the Neural Engine included in the M1 chip that is dedicated to AI workloads, but "hasn't really been used much yet," as well as Apple's buy-in to Ultra Wide-band.

Scoble believes the Apple Mapping car is being used to map the world with voxels, dividing Earth into small cubes. Each cube allegedly has a unique identifier, a virtualized microphone and speaker, a virtualized display, a virtualized database, and a virtualized computer.

The scans would also be performed in buildings by its products, taking advantage of things like computer vision and LiDAR on an iPhone or iPad.

By building the map and introducing hardware that can view and take advantage of the data, Apple will be able to offer other major applications.

As part of this, Scoble says Apple will be working to overhaul Siri to react to what the user is looking at through a headset or another device. By taking advantage of computer vision or the map, Siri will be able to infer meaning from what the user is looking at, and to answer questions like "How much are 20 of these on Amazon?"

The overhauled Siri will also be more flexible and hooked into more data sources. For example, Siri may access a Foursquare API to answer a question about check-ins at a hotel.

The "HoloDeck" will be a "TV killer" application, Scoble suggests, when the headset becomes available in 2022. Claiming to have been given a "sneak look" at software being built for the headset, he believes it will introduce "a stunning set of capabilities that will blow away any physical TV you have ever seen"

The headset or visor will "cover your eyes and your ears," showing a representation of the living room and playing a representation of environmental audio to the user. It is offered that this could be similar to the "transparency" mode of the AirPods Max.

The screens will apparently be made of two 8K "Sony chips," which will allow Apple to virtualize TVs to be bigger than any in a store, and in high quantities. By taking advantage of Ultra Wideband, these screens may also be better locked into position in the environment, and for devices to automatically know where they are located in the world without requiring resources.

Scoble offers that this is the equivalent of Apple putting a TV "on every inch of the world," which can enable new experiences. For example, a Super Bowl halftime show could pop out of the TV and show the 3D performer in front of the user, rather than on a screen.

A volumetric football service is also apparently being prepared, giving the experience of being sat at the stadium through a volumetric surface alongside 2D screens.

On sound, Scoble believes the experience will offer a system with extremely high numbers of channels. In the case of a VR band, a user would feasibly be able to stand in the middle and hear instruments in different positions.

The Holodeck-style experience will apparently be announced sometime before the end of 2021, and could be teased at WWDC itself, the VR evangelist believes.

For CarPlay, a 3D audio experience can allow individual buttons or parts of a road to "talk to you." While such a system could be included in the "Apple Car," Scoble insists it could land in current vehicles in 2021.

Scoble also suggests a new portable gaming device is on the way that would be able to work with headsets, as well as a next-level noise-cancellation system.

"Put it all together and you can see Apple is about to unleash a new paradigm-shifting strategy. One that will change all of our lives very deeply and bring us many exciting new things," says Scoble. "This has deep implications on the future of a number of companies. Spotify looks threatened. So does Google."

Stay on top of all Apple news right from your HomePod. Say, "Hey, Siri, play AppleInsider," and you'll get latest AppleInsider Podcast. Or ask your HomePod mini for "AppleInsider Daily" instead and you'll hear a fast update direct from our news team. And, if you're interested in Apple-centric home automation, say "Hey, Siri, play HomeKit Insider," and you'll be listening to our newest specialized podcast in moments.

Comments

  • Reply 1 of 19
    indieshackindieshack Posts: 328member
    You had me interested until mentioning “Siri”. Also, if this was from another source than Scoble I would take it more seriously.
    gregoriusmVermelhoFileMakerFeller
  • Reply 2 of 19
    entropysentropys Posts: 4,152member
    To roll out as fast as the look around feature of Apple maps.

    elijahgbonobob
  • Reply 3 of 19
    mobirdmobird Posts: 752member
    Can I post my wishlist as well?

    gregoriusmwatto_cobra
  • Reply 4 of 19
    Fidonet127Fidonet127 Posts: 507member
    Ummm..... it is hard enough keeping 2D map of the world updated because of construction, etc, yet supposedly they will have a realtime 3D map of the world? That is a lot of data to gather, transmit and crunch. This more of dreaming, than an actual prediction.
    edited May 2021 macpluspluswatto_cobra
  • Reply 5 of 19
    Google Earth VR is pretty addictive. It is so interesting to explore locations that have voxel maps. Google already has much of the planet scanned in 3D. The app give the viewer a sort of gods eye view of the planet as everything is shrunk down to look like a table top model (there is a realistic scale too which is also worth trying). This would be a great area for Apple to get into as they already have 3D scans of many cities. Currently they do not allow developers to use them in VR mode sadly but this seems to be a direction they are going in.
    gregoriusmwatto_cobra
  • Reply 6 of 19
    JapheyJaphey Posts: 1,767member
    lmasanti said:
    Although —almost— all what he says seems ‘plausible,’ I do not know why I have a feeling that he just found that ‘putting Apple’ in a site/blog/etc. attracks lots of page views.

    Bloomberg has an article about Apple vs. Microsoft in AR/VR war… and there is no mention of Microsoft? Only Spotify and Google?

    But this is only my feeling…
    Agreed. This is like chum in the water for all the fanboys and girls. Fortunately with WWDC only three  weeks away, we won’t have to wait very long to find out if this is bullshit or not. Strange that Spotify was specifically called out without explaining why. Stranger still is that neither Microsoft nor Facebook were even mentioned. 
    edited May 2021 watto_cobra
  • Reply 7 of 19
    Apple will start employing many people around the world to update the map in real time. You will be able to make money (paid through your Apple Card) by go out with your phone and update locations. Your boss will be  an A.I. Bot giving jobs to tens of thousands all over the world at minimum wage to spend their day recording and uploading. Eventually people will realize that that is what this world is... we are all AI bots recording the world with our eyes and ears for our masters to play the game in the prior dimension. 
    watto_cobra
  • Reply 8 of 19
    boboliciousbobolicious Posts: 1,139member
    Given what I understand of human nature is this potentially problematic...?



    edited May 2021 bennettvista
  • Reply 9 of 19
    hydrogenhydrogen Posts: 314member
    "The primary use of Geography is war" - Napoleon Bonaparte
    Japheywatto_cobra
  • Reply 10 of 19
    WgkruegerWgkrueger Posts: 352member
    Apple will start employing many people around the world to update the map in real time. You will be able to make money (paid through your Apple Card) by go out with your phone and update locations. Your boss will be  an A.I. Bot giving jobs to tens of thousands all over the world at minimum wage to spend their day recording and uploading. Eventually people will realize that that is what this world is... we are all AI bots recording the world with our eyes and ears for our masters to play the game in the prior dimension. 
    Sounds great! When can I sign up?
    watto_cobra
  • Reply 11 of 19
    WgkruegerWgkrueger Posts: 352member
    So, Apple should be making an autonomous Apple Drone.
    watto_cobra
  • Reply 12 of 19
    byronlbyronl Posts: 356member
    so excited!!!
    watto_cobra
  • Reply 13 of 19
    mattinozmattinoz Posts: 2,299member
    Ummm..... it is hard enough keeping 2D map of the world updated because of construction, etc, yet supposedly they will have a realtime 3D map of the world? That is a lot of data to gather, transmit and crunch. This more of dreaming, than an actual prediction.
    Sell it as a service to property developers and Apple can get paid to keep the data up to date. 
    We’d already pay them for Apple maps 3D file of the area surrounding our sites if the option was there. 
    jony0watto_cobrabyronl
  • Reply 14 of 19
    byronlbyronl Posts: 356member
    mattinoz said:
    Ummm..... it is hard enough keeping 2D map of the world updated because of construction, etc, yet supposedly they will have a realtime 3D map of the world? That is a lot of data to gather, transmit and crunch. This more of dreaming, than an actual prediction.
    Sell it as a service to property developers and Apple can get paid to keep the data up to date. 
    We’d already pay them for Apple maps 3D file of the area surrounding our sites if the option was there. 
    yea but REAL TIME?
    watto_cobra
  • Reply 15 of 19
    fastasleepfastasleep Posts: 6,408member
    Given what I understand of human nature is this potentially problematic...?

    Everything about human nature is potentially problematic. 
    watto_cobrabyronl
  • Reply 16 of 19
    fastasleepfastasleep Posts: 6,408member
    lmasanti said:
    Although —almost— all what he says seems ‘plausible,’ I do not know why I have a feeling that he just found that ‘putting Apple’ in a site/blog/etc. attracks lots of page views.

    Bloomberg has an article about Apple vs. Microsoft in AR/VR war… and there is no mention of Microsoft? Only Spotify and Google?

    But this is only my feeling…
    Japhey said:
    lmasanti said:
    Although —almost— all what he says seems ‘plausible,’ I do not know why I have a feeling that he just found that ‘putting Apple’ in a site/blog/etc. attracks lots of page views.

    Bloomberg has an article about Apple vs. Microsoft in AR/VR war… and there is no mention of Microsoft? Only Spotify and Google?

    But this is only my feeling…
    Agreed. This is like chum in the water for all the fanboys and girls. Fortunately with WWDC only three  weeks away, we won’t have to wait very long to find out if this is bullshit or not. Strange that Spotify was specifically called out without explaining why. Stranger still is that neither Microsoft nor Facebook were even mentioned. 
    He specifically mentions Microsoft in the context of Hololens once, twice in reference to things that happened when he worked there, and once in context of his stock holdings. Anyone read the source?
    watto_cobra
  • Reply 17 of 19
    mattinozmattinoz Posts: 2,299member
    byronl said:
    mattinoz said:
    Ummm..... it is hard enough keeping 2D map of the world updated because of construction, etc, yet supposedly they will have a realtime 3D map of the world? That is a lot of data to gather, transmit and crunch. This more of dreaming, than an actual prediction.
    Sell it as a service to property developers and Apple can get paid to keep the data up to date. 
    We’d already pay them for Apple maps 3D file of the area surrounding our sites if the option was there. 
    yea but REAL TIME?
    People in construction already take 100's of photos a day of the work in progress, the design team takes 1000's during planning phases. We now runs Drones to capture an existing building then turn that in to a model including surrounding areas. Builders are running drones daily to capture progress of works. There is a lot of information out there already being captured and could be at Apple's disposal with the right commercial agreements.  Sure not getting to real time but certainly to the point that most digital twins of constructions could be no less than a day behind if not a couple of hours. The sort of processing Apple has on device with the right API could be under a second. 

    Real time is the "ambitious goal" not a pass/fail. it doesn't need to get there anytime in the next decade to still be successful and valuable to Apple's cashflow.
    watto_cobrabyronl
  • Reply 18 of 19
    crowleycrowley Posts: 10,453member
    Nerd has a nerdgasm.

    I don’t believe a word of it. Total flight of fancy.
    edited May 2021
Sign In or Register to comment.