Patent filing suggests Apple is working on 'Street View' mapping technology
An Apple patent application discovered on Thursday hints that the company is looking to deploy on mobile devices a virtual navigation system based on panoramic location data, much like the popular "Street View" seen in Google Maps.
Published by the U.S. Patent and Trademark Office, Apple's "3D Position Tracking for Panoramic Imagery Navigation" describes a graphical user interface that leverages an iPhone or iPad's onboard sensors to navigate panoramic imagery.
Source: USPTO
According to the filing, Apple's invention improves upon current technology, like Google's Street View, which boxes users into a "bubble" that can only be navigated through an input device like a mouse or multitouch screen. For example, in a conventional GUI, the user must "jump" to a panoramic "bubble" at a given intersection and pan within said bubble to move in a desired direction; a tedious experience for mobile device users on the go.
Instead of the traditional approach, Apple proposes tracking subsystems and onboard sensors deployed within a mobile device be used to translate a user's physical motion into a panoramic navigation UI. In the examples that follow, data from accelerometers, cameras, gyroscopes and other sensors are used to "move" a user through virtual street-level panoramic space.
First, the invention notes a user must first enter the street-level view, which can be accomplished by "pinching in" on a map, or by selecting a dropped pin icon. Once in street view mode, a user can move their device up, down, left or right to view panoramic imagery supplied either by built-in storage or streamed wirelessly over cellular data networks. Movement is controlled by moving the device forward and back.
Illustration of device transitional movement from original position (104) with informational overlay (103b).
Throughout the process, onboard sensors are collecting movement data, including linear and velocity metrics, and translating the motions into the GUI.
Further, the filing notes informational bubbles can be displayed on the virtual environment to point out places of interest such as buildings or shops. Information is stored in layers, an example being "businesses," and can be displayed according to a user's preferences. In some embodiments, the bubbles can be hidden to reduce clutter on smaller device screens.
In an alternative implementation, the system can translate movement data from an imaging sensor in what is called "optical flow," which reads apparent patterns of motion of objects in a panoramic image in relation to an observer. By scaling distance data, a device can display the appropriate virtual location of a user within the environment.
One particularly intriguing idea is the use of multiple displays to increase the visible area of a panoramic image. Devices can communicate wirelessly to display concurrent information regarding the virtual environment.
Finally, the application mentions the use of interior imaging data for use in some implementations, allowing users to "walk into" a building using their device. When inside structures, other actions can be performed, such as "selecting an object for purchase," though further detail regarding that level of functionality was not discussed.
Apple's iOS Maps currently lacks a street-level viewing option as it simply doesn't have the imaging data. This feature, which is available on Google's mapping service thanks to its Street View initiative, was sorely missed by some iOS device users with the introduction of the Maps app in iOS 6.
It is unclear if and when Apple will implement the invention in a future iteration of Maps, but the filing shows the company is at least actively investigating a competitor to Google's solution.
Apple's patent application was filed in September of 2011 and credits Patrick Piemonte and Billy Chen as its inventors.
Published by the U.S. Patent and Trademark Office, Apple's "3D Position Tracking for Panoramic Imagery Navigation" describes a graphical user interface that leverages an iPhone or iPad's onboard sensors to navigate panoramic imagery.
Source: USPTO
According to the filing, Apple's invention improves upon current technology, like Google's Street View, which boxes users into a "bubble" that can only be navigated through an input device like a mouse or multitouch screen. For example, in a conventional GUI, the user must "jump" to a panoramic "bubble" at a given intersection and pan within said bubble to move in a desired direction; a tedious experience for mobile device users on the go.
Instead of the traditional approach, Apple proposes tracking subsystems and onboard sensors deployed within a mobile device be used to translate a user's physical motion into a panoramic navigation UI. In the examples that follow, data from accelerometers, cameras, gyroscopes and other sensors are used to "move" a user through virtual street-level panoramic space.
First, the invention notes a user must first enter the street-level view, which can be accomplished by "pinching in" on a map, or by selecting a dropped pin icon. Once in street view mode, a user can move their device up, down, left or right to view panoramic imagery supplied either by built-in storage or streamed wirelessly over cellular data networks. Movement is controlled by moving the device forward and back.
Illustration of device transitional movement from original position (104) with informational overlay (103b).
Throughout the process, onboard sensors are collecting movement data, including linear and velocity metrics, and translating the motions into the GUI.
Further, the filing notes informational bubbles can be displayed on the virtual environment to point out places of interest such as buildings or shops. Information is stored in layers, an example being "businesses," and can be displayed according to a user's preferences. In some embodiments, the bubbles can be hidden to reduce clutter on smaller device screens.
In an alternative implementation, the system can translate movement data from an imaging sensor in what is called "optical flow," which reads apparent patterns of motion of objects in a panoramic image in relation to an observer. By scaling distance data, a device can display the appropriate virtual location of a user within the environment.
One particularly intriguing idea is the use of multiple displays to increase the visible area of a panoramic image. Devices can communicate wirelessly to display concurrent information regarding the virtual environment.
Finally, the application mentions the use of interior imaging data for use in some implementations, allowing users to "walk into" a building using their device. When inside structures, other actions can be performed, such as "selecting an object for purchase," though further detail regarding that level of functionality was not discussed.
Apple's iOS Maps currently lacks a street-level viewing option as it simply doesn't have the imaging data. This feature, which is available on Google's mapping service thanks to its Street View initiative, was sorely missed by some iOS device users with the introduction of the Maps app in iOS 6.
It is unclear if and when Apple will implement the invention in a future iteration of Maps, but the filing shows the company is at least actively investigating a competitor to Google's solution.
Apple's patent application was filed in September of 2011 and credits Patrick Piemonte and Billy Chen as its inventors.
Comments
I'd love if you could somehow zoom in seamlessly from the 3D Flyover View into the Street View and then tie it in with Siri to either WalkThrough or DriveThrough to your destination.
Hold on there... Shouldn't E Broadway be on the right hand side and W Broadway on the left?
You know what they say... If the patent is wrong then... LOL.
You missed it - this is the new earthworm view. You're looking up from underground.
Quote:
Originally Posted by jimbo123
You can't but feel that's its all catchup to Google.
lol Not really. A street view like feature on a much superior application with much superior APIs that developers love (despite the fact that apple needs time to gather more infromation for the app... Google has been doing this for years, and apple is already close), together with other superior features, is nothing short of a knife on google's liver.
Apple is close to have:
- The absolute best hardware;
- The absolute best mobile and desktop OS;
- The absolute best pro software;
- The absolute best office software;
- The absolute best cloud services and maps.
There's only one thing left: Apply their superior search knowledge (spotlight, etc) and make a search engine.
That's a single company being better than whole industries.
Forget about patents, Apple needs to hire 200 drivers and start covering the earth.
Quote:
Originally Posted by pedromartins
lol Not really. A street view like feature on a much superior application with much superior APIs that developers love (despite the fact that apple needs time to gather more infromation for the app... Google has been doing this for years, and apple is already close), together with other superior features, is nothing short of a knife on google's liver.
Apple is close to have:
- The absolute best hardware;
- The absolute best mobile and desktop OS;
- The absolute best pro software;
- The absolute best office software;
- The absolute best cloud services and maps.
There's only one thing left: Apply their superior search knowledge (spotlight, etc) and make a search engine.
That's a single company being better than whole industries.
You want some wet-wipes after that?
So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it. I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.
Hmmm. . .
So instead of pinch-zooming or jumping to a bubble like in Google Streetview, Apple's version (if ever used) would be initiated by . . . uh. . . . pinch-zooming or by selecting a dropped pin (Is that like the Google "bubble"?). Gosh, that does sound a lot less tedious for the mobile device user on the go. /s
Now if only Google Streetview could make use of the mobile device's sensors to zoom or rotate the view by simply tilting and/or turning the phone then Apple's patent application would be really really similar to Google's current implementation.
Oh, wait. . .
Navigate within Street View
Use your finger to pan the map.
Drag the Pegman icon in the lower-left corner to a point in the image in order to zoom directly to that point.
Double-tap a point in Street View with one finger to zoom in; touch once with two fingers to zoom out.
Touch Compass mode in the Google Maps menu to use the phone itself to navigate Street View. Tilt, pan, or turn the phone to change the view. Touch Compass mode again to end it.
http://support.google.com/gmm/bin/answer.py?hl=en&answer=1650202
Quote:
Originally Posted by jimbo123
You can't but feel that's its all catchup to Google.
Google had a 7 year head start and within a year Apple has got pretty close.
Google were just resting on their laurels thinking they had no competition. Apple maps were a kick in the ass for Google who would have just carried on adding a few new things over the next 10 years. If Apple had been doing maps for 7 years there would be no hope for anyone else to catch up.
Quote:
Originally Posted by Ireland
Forget about patents, Apple needs to hire 200 drivers and start covering the earth.
Nope, just send out their fleet of top secret iRobots
Quote:
Originally Posted by jimbo123
You can't but feel that's its all catchup to Google.
Google hardly "invented" maps.
So the feature evolves, nature of tech.
Quote:
Originally Posted by cnocbui
So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it. I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.
They have enough data to start the process. After all, if they waited until they had a 100% perfect data set before doing anything, it would never happen
Quote:
Originally Posted by cnocbui
So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it. I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.
So, what you are saying is, it would be better for Apple to NOT plan ahead.
Brilliant!
They had a good example to work from.
Originally Posted by AppleInsider
... reads apparent patterns of motion of objects in a panoramic image in relation to an observer. By scaling distance data, a device can display the appropriate virtual location ...
This is why I think the A7 and its GPU both need to be quad core.
Whether they debut in the iPhone 5S or the iPhone 6.
That might be true - if Google were someone that you could trust.
Not to mention, of course, that the cost (both money and other terms) might be too high to do it with Google.
I suspect that Apple's management knew what they were doing when they decided to offer their own mapping solution.
Quote:
Originally Posted by Evilution
Google had a 7 year head start and within a year Apple has got pretty close.
Actually only two years head start when you consider that they have to redo every street scene and map on a two year refresh cycle to stay relevant. Kudos to Apple Maps for updating their aerial photography in my US location... and the resolution is a couple levels higher than it was before.
Quote:
Originally Posted by Gatorguy
Hmmm. . .
So instead of pinch-zooming or jumping to a bubble like in Google Streetview, Apple's version (if ever used) would be initiated by . . . uh. . . . pinch-zooming or by selecting a dropped pin (Is that like the Google "bubble"?). Gosh, that does sound a lot less tedious for the mobile device user on the go. /s
Now if only Google Streetview could make use of the mobile device's sensors to zoom or rotate the view by simply tilting and/or turning the phone then Apple's patent application would be really really similar to Google's current implementation.
Oh, wait. . .
Navigate within Street View
Use your finger to pan the map.
Drag the Pegman icon in the lower-left corner to a point in the image in order to zoom directly to that point.
Double-tap a point in Street View with one finger to zoom in; touch once with two fingers to zoom out.
Touch Compass mode in the Google Maps menu to use the phone itself to navigate Street View. Tilt, pan, or turn the phone to change the view. Touch Compass mode again to end it.
http://support.google.com/gmm/bin/answer.py?hl=en&answer=1650202
Your comment is a bit hard to understand but it seems to imply that Apple's solution is the same as what Google already has. It isn't.
If that's what you think, you need to re-read the article more carefully.