Apple patents augmented reality mapping system for iPhone

Posted:
in General Discussion edited November 2016
Apple on Tuesday was granted a patent detailing an augmented reality mapping system that harnesses iPhone hardware to overlay visual enhancements onto live video, lending credence to recent rumors suggesting the company plans to implement an iOS-based AR strategy in the near future.




As published by the U.S. Patent and Trademark Office, Apple's U.S. Patent No. 9,488,488 for "Augmented reality maps" describes a mapping app capable of tapping into iPhone's advanced sensor suite to present users with real-time augmented views of their surrounding environment.

In some embodiments, digital information like street names, points of interest and more are overlaid on top of live video pulled from a device's camera. Other variations include complex functions like GPS-based navigation.

For example, a user might open activate iPhone's rear-facing camera by navigating to an AR feature within the Maps app. The system automatically correlates data from onboard sensors, including gyroscope, digital compass and accelerometer components, with GPS data to pinpoint iPhone's position in space, tilt and alignment.

Using the newly gleaned information, the app is able to intelligently determine an approximation of the displayed live video scene, including field of view. From there, the captured image, or live video feed, is visually augmented with vital mapping information downloaded from an offsite server.

Certain embodiments offer simple POI identification, for example the name of buildings and monuments, though the system allows for more complex computations. In some examples, Apple's invention generates interactive route guidance complete with onscreen path indicators overlaid onto the live streamed video. The method can also pull data from the cloud to alert users that they are heading down a one-way street, for example.

Depending on device orientation, the system is capable of switching display formats from live video to a bird's-eye view, making transitioning into and out of AR mode a seamless experience.




Today's USPTO award comes five years to the day since Apple first applied for the patent grant in 2011. Only minor changes were made to the document to clarify and enhance claims.

Apple is known to be working on augmented reality solutions, though exactly what form that product will take is unclear. CEO Tim Cook in an quarterly investors conference call in July signaled Apple is particularly interested in AR, a sentiment he reiterated multiple times since then.

As tech industry competitors like Google, Microsoft and Facebook field their own AR/VR systems, pundits believe Apple plans to dip its toe into the AR/VR waters with an iPhone-based solution. Bolstering those claims are a multitude of AR patents, the latest covering device localization technology assigned to Apple from recent acquisition Flyby Media.

While an iPhone AR mapping app is intriguing, integrating the same technology into an in-car heads-up display would prove revolutionary. Analyst Ming-Chi Kuo recently hinted that Apple might use AR technology developed for mobile to break into other areas of interest like autonomous driving systems pulled from the ashes of Project Titan.

Apple's augmented map patent was first filed for in February 2010 and credits Jaron Waldman as its inventor.

Comments

  • Reply 1 of 9
    This is a time when I am hoping Apple technology is ready for showtime soon!
    watto_cobra
  • Reply 2 of 9
    blitz1blitz1 Posts: 412member
    Pokemon Go someone?
    birko
  • Reply 3 of 9





    Ha!  A 2016 bird's-eye view of Mission Dolores Park...


    How 'bout a 2013 astronaut's-eye view?



    edited November 2016
  • Reply 4 of 9
    I am frequently offended by the appaling writing standards on Apple Insider. A succession of cliches, jargon and tech-sad abreviations topped by content-free diarrhoea. Most of what appears could be more clearly expressed in 10% of the words. I can only infer that the writers get paid little and by the word. 
    SpamSandwich
  • Reply 5 of 9

    Thinking about this AR capability integrating today's iPhone camera and Apple Maps...

    Apple should ask the user if they would anonymously share [appropriate] camera images.  Apple could used these images to enhance Apple Maps -- essentially crowd-sourcing map data.  Potentially, it could be far superior to street-level drive-by mapping.

    watto_cobra
  • Reply 6 of 9
    Hopefully this is setting up for something new. I just don't see it useful to constantly be holding iPhone up to gain this information. Now if they use it somehow to tie into car systems that could be useful. Or other camera systems where this could be applicable...but to constantly hold up your phone to get this augmented view will not work for me.
  • Reply 7 of 9
    Hopefully this is setting up for something new. I just don't see it useful to constantly be holding iPhone up to gain this information. Now if they use it somehow to tie into car systems that could be useful. Or other camera systems where this could be applicable...but to constantly hold up your phone to get this augmented view will not work for me.
    I don't think it's meant to be used continuously -- rather, to periodically take a moment to get your bearings.  Then you could select your destination/route and let maps guide you there with notifications: e.g. Make a left turn onto 17th street from Dolores.

    edited November 2016
  • Reply 8 of 9
    MacProMacPro Posts: 17,998member
    Next time I'm in Amsterdam I know where I'll test this ;)
  • Reply 9 of 9
    Next time I'm in Amsterdam I know where I'll test this ;)
    Lol, @digi...

    AR window shopping is fine -- but no substitute for the real thing!

Sign In or Register to comment.