Apple's future AR smart glasses could help you find things in your house

Posted:
in General Discussion edited May 2020
Apple is working on an AR mapping app for an iPhone or smart glasses, with identification of not just distant locations, but also on a smaller scale, like a car dashboard or items in your house.

A concept image of Google Maps using AR for navigation
A concept image of Google Maps using AR for navigation


Mapping and navigation systems are handy for many users in telling people where to go and how to get to their desired destination. While useful for those who can adequately read maps or those who have a general idea of their current location's geography, sometimes users want to be able to have a more visual guide to where things are in relation to their current position.

Granted by the U.S. Patent and Trademark Office on Tuesday, Apple's patent for a "Method for representing points of interest in a view of a real environment on a mobile device and mobile device therefor" effectively describes how an augmented reality system could point out important areas in a landscape for a user.

The patent is in fact one that surfaced in 2017, revealed by the USPTO in July of that year after being applied for by Apple in April.

An illustration of an AR overlay on an iPhone
An illustration of an AR overlay on an iPhone


According to the patent, the mobile device would capture an image of a view, likely one from a live camera feed, and overlay an indicator on the screen in the relative position of where a known point of interest is based. In a live video feed, the position of the indicator will change in relation to alterations in the view, so the indicator will continue to show the correct position.

With each indicator, extra supplemental information could also be offered in a separate and uncluttered part of the display, such as its name and distance. By interacting with the indicator, this could trigger functions such as navigating the user to it or offering more information or content, to learn more about the place without going there.

Illustrating how an iPad could be used to show features of a car in AR
Illustrating how an iPad could be used to show features of a car in AR


While the concept of an AR view for maps isn't an entirely new concept, there are some elements of the patent that differ from the usual idea.

First, it is suggested that the system could be used for small environments as well as larger spaces, such as things in a room. In one example image, an iPad-like tablet is used to show off key features of a car's interior, identifying and highlighting important items and potentially giving the user more opportunities to learn about each item. The text discusses location of items in a house, like pinpointing where your iPhone is in your house, instead of using "Find my iPhone" to make it play a sound.

The patent also references the use of a "semi-transparent display," in that rather than using a live camera feed, it could be graphics overlaying a real-world view seen through glass. Some of the example illustrations depict a head-mounted display view that would be expected from the usage of smart glassesor a VR headset.

Illustrating how the system could work in hypothetical smart glasses
Illustrating how the system could work in hypothetical smart glasses


As usual for patents and applications, Apple files many documents with the USPTO each week, and while they can be used as an indicator of Apple's fields of interest, they are not a guarantee that Apple will be working in that area in the future.

Apple has a number of patents and applications already under the field of Augmented Reality, to go with its existing and popular ARKit platform. In January 2017 it received two patents for a mobile AR system that could detect its surroundings and provide information to users in real-time, similar to the in-car function from the latest filing.

Apple also received a patent for "Augmented reality maps" in 2016, which laid out road names and routes on the streets and roads of the camera's view, as well as highlighting points of interest. More recently, a December 2018 patent application showed how AR could help drivers and passengers of a ride-hailing service find each other in crowded or busy locations.

The company has been rumored to be working on its own smart glasses and AR headsets for some time, with the suggestion WiGig and 8K-resolution eyepieces could be used in such headwear. Such a product is unlikely to ship anytime soon, as analysts believe the earliest a head-mounted AR device from Apple could ship is 2021.

Comments

  • Reply 1 of 12
    MplsPMplsP Posts: 3,911member
    Not sure how helpful this would be for me - the thing I need to find most often in my home is my iphone. Thankfully, the handset locate function on the Apple Watch is awesome for that!

    GeorgeBMacwatto_cobra
  • Reply 2 of 12
    MplsP said:
    Not sure how helpful this would be for me - the thing I need to find most often in my home is my iphone. Thankfully, the handset locate function on the Apple Watch is awesome for that!

    If it was possible to wear out that feature my wife would have done so long ago!

    I feel like finding objects around your home is a long way off, considering there are still plenty of issues finding known locations that haven’t moved for years.
    edited February 2019 muthuk_vanalingam
  • Reply 3 of 12
    Wearing some glasses like this right now feels to me like the idea of needing a foldable smartphone/tablet. 
  • Reply 4 of 12
    gatorguygatorguy Posts: 24,176member
    Wearing some glasses like this right now feels to me like the idea of needing a foldable smartphone/tablet. 
    Might be a bit odd walking around the house with 'em.  ;)
  • Reply 5 of 12
    GeorgeBMacGeorgeBMac Posts: 11,421member
    I can see something like this being integrated into Apples autonomous car program -- with AR being integrated into the car's mapping program so, particularly in close quarters such as a shopping center, you point to where you want the car to go.  Or another example: trying to park in an urban area and the lot you wanted to go to has a "Lot Full" sign up.   Integrate this with community wide cameras and 5G network and ya got something.
    watto_cobra
  • Reply 6 of 12
    gatorguy said:
    Wearing some glasses like this right now feels to me like the idea of needing a foldable smartphone/tablet. 
    Might be a bit odd walking around the house with 'em.  ;)
    At least, the number of people seeing you like this is limited - and possibly they’d know about your* funny nerdy moments. 😉

    *That’s not “you” personally, of course. 
  • Reply 7 of 12
    gatorguygatorguy Posts: 24,176member
    gatorguy said:
    Wearing some glasses like this right now feels to me like the idea of needing a foldable smartphone/tablet. 
    Might be a bit odd walking around the house with 'em.  ;)
    At least, the number of people seeing you like this is limited - and possibly they’d know about your* funny nerdy moments. 😉

    *That’s not “you” personally, of course. 
    No problem. :) My wife might say it fits. 
    wonkothesane
  • Reply 8 of 12
    At some point the social stigma of wearing AR glasses will fade*, and then it'll be nice to talk to people and know their name.  For some reason I have a hard time with that, and having an assistant just pop it up will be the thing for me.

    And finding things around the house would help too, as well as having it when driving or flying.  There are already apps with AR on them when flying, like FlyQ Insight, but pulling the ipad/iphone out and searching around for things is a bit... awkward, and could be deadly while driving.  (Flying is easier, as the airplane should be trimmed for straight and level flight, and there really isn't much to hit up at 2500 feet AGL)

    *like when only "geeks" wore BT headsets, or had cell phones, or computers...

    StrangeDayswatto_cobra
  • Reply 9 of 12
    gatorguygatorguy Posts: 24,176member
    At some point the social stigma of wearing AR glasses will fade*, and then it'll be nice to talk to people and know their name.  For some reason I have a hard time with that, and having an assistant just pop it up will be the thing for me.
    Re: Google Glass, identifying people was prohibited. It may be the same with early Apple (or whoever) eyewear. There's going be be a significant level of distrust early on, but like you mention people will adapt and it will become acceptable, perhaps even normal. 
    edited February 2019
  • Reply 10 of 12
    GeorgeBMacGeorgeBMac Posts: 11,421member
    At some point the social stigma of wearing AR glasses will fade*, and then it'll be nice to talk to people and know their name.  For some reason I have a hard time with that, and having an assistant just pop it up will be the thing for me.

    ....
    That could be 'face blindness'.  Or the medical name for it is:  Prosopagnosia.   It is quite common in Austistics and Aspergians but is not limited to those conditions.    Something like that could be a huge help to those with the condition.
  • Reply 11 of 12
    flaneurflaneur Posts: 4,526member
    At some point the social stigma of wearing AR glasses will fade*, and then it'll be nice to talk to people and know their name.  For some reason I have a hard time with that, and having an assistant just pop it up will be the thing for me.

    ....
    That could be 'face blindness'.  Or the medical name for it is:  Prosopagnosia.   It is quite common in Austistics and Aspergians but is not limited to those conditions.    Something like that could be a huge help to those with the condition.
    Oliver Sacks, of all people, was one of these prosopagnostics — he was the neurologist who wrote “The Man Who Mistook His Wife for a Hat” and much else.

    A bit of an odd duck, he just might have been one of the first to want to try out glasses for face recognition. But how this would work with current social norms is hard to figure. Who wants to be ID’d everywhere by AI and AR? This isn’t China, at least not yet.

    Anyway, realtime identification of places and objects by just looking at them will be as revolutionary as the cell phone itself. Or maybe the computer itself.
    edited February 2019 GeorgeBMac
  • Reply 12 of 12
    MplsP said:
    Not sure how helpful this would be for me - the thing I need to find most often in my home is my iphone. Thankfully, the handset locate function on the Apple Watch is awesome for that!

    You can also ask Siri on a device like HomePod to “ping my iphone” and it will start chiming. 
Sign In or Register to comment.