or Connect
AppleInsider › Forums › General › General Discussion › Patent filing suggests Apple is working on 'Street View' mapping technology
New Posts  All Forums:Forum Nav:

Patent filing suggests Apple is working on 'Street View' mapping technology

post #1 of 44
Thread Starter 
An Apple patent application discovered on Thursday hints that the company is looking to deploy on mobile devices a virtual navigation system based on panoramic location data, much like the popular "Street View" seen in Google Maps.

Published by the U.S. Patent and Trademark Office, Apple's "3D Position Tracking for Panoramic Imagery Navigation" describes a graphical user interface that leverages an iPhone or iPad's onboard sensors to navigate panoramic imagery.

Panoramic Image
Source: USPTO


According to the filing, Apple's invention improves upon current technology, like Google's Street View, which boxes users into a "bubble" that can only be navigated through an input device like a mouse or multitouch screen. For example, in a conventional GUI, the user must "jump" to a panoramic "bubble" at a given intersection and pan within said bubble to move in a desired direction; a tedious experience for mobile device users on the go.

Instead of the traditional approach, Apple proposes tracking subsystems and onboard sensors deployed within a mobile device be used to translate a user's physical motion into a panoramic navigation UI. In the examples that follow, data from accelerometers, cameras, gyroscopes and other sensors are used to "move" a user through virtual street-level panoramic space.

First, the invention notes a user must first enter the street-level view, which can be accomplished by "pinching in" on a map, or by selecting a dropped pin icon. Once in street view mode, a user can move their device up, down, left or right to view panoramic imagery supplied either by built-in storage or streamed wirelessly over cellular data networks. Movement is controlled by moving the device forward and back.

Panoramic Image Movement
Illustration of device transitional movement from original position (104) with informational overlay (103b).


Throughout the process, onboard sensors are collecting movement data, including linear and velocity metrics, and translating the motions into the GUI.

Further, the filing notes informational bubbles can be displayed on the virtual environment to point out places of interest such as buildings or shops. Information is stored in layers, an example being "businesses," and can be displayed according to a user's preferences. In some embodiments, the bubbles can be hidden to reduce clutter on smaller device screens.

In an alternative implementation, the system can translate movement data from an imaging sensor in what is called "optical flow," which reads apparent patterns of motion of objects in a panoramic image in relation to an observer. By scaling distance data, a device can display the appropriate virtual location of a user within the environment.

Panoramic Image Movement


One particularly intriguing idea is the use of multiple displays to increase the visible area of a panoramic image. Devices can communicate wirelessly to display concurrent information regarding the virtual environment.

Finally, the application mentions the use of interior imaging data for use in some implementations, allowing users to "walk into" a building using their device. When inside structures, other actions can be performed, such as "selecting an object for purchase," though further detail regarding that level of functionality was not discussed.

Apple's iOS Maps currently lacks a street-level viewing option as it simply doesn't have the imaging data. This feature, which is available on Google's mapping service thanks to its Street View initiative, was sorely missed by some iOS device users with the introduction of the Maps app in iOS 6.

It is unclear if and when Apple will implement the invention in a future iteration of Maps, but the filing shows the company is at least actively investigating a competitor to Google's solution.

Apple's patent application was filed in September of 2011 and credits Patrick Piemonte and Billy Chen as its inventors.
post #2 of 44
You can't but feel that's its all catchup to Google.

G5 2GHZ Power Mac, iPod Shuffle (1st Gen),iPod Nano (2nd Gen),iPod (5th Gen), Apple TV, Apple TV 2G x2, iPad 2,iPhone 4S, rMBP 15" 2.6

Reply

G5 2GHZ Power Mac, iPod Shuffle (1st Gen),iPod Nano (2nd Gen),iPod (5th Gen), Apple TV, Apple TV 2G x2, iPad 2,iPhone 4S, rMBP 15" 2.6

Reply
post #3 of 44
This is exactly what I hoped and thought Apple would do. If anything, this could go some way to improving the actual data in Apple Maps.

I'd love if you could somehow zoom in seamlessly from the 3D Flyover View into the Street View and then tie it in with Siri to either WalkThrough or DriveThrough to your destination.
post #4 of 44

Hold on there... Shouldn't E Broadway be on the right hand side and W Broadway on the left?

You know what they say... If the patent is wrong then...  LOL.

post #5 of 44
Quote:
Originally Posted by AppleSauce007 View Post

Hold on there... Shouldn't E Broadway be on the right hand side and W Broadway on the left?
You know what they say... If the patent is wrong then...  LOL.

You missed it - this is the new earthworm view. You're looking up from underground.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #6 of 44
Quote:
Originally Posted by jimbo123 View Post

You can't but feel that's its all catchup to Google.

lol Not really. A street view like feature on a much superior application with much superior APIs that developers love (despite the fact that apple needs time to gather more infromation for the app... Google has been doing this for years, and apple is already close), together with other superior features, is nothing short of a knife on google's liver.

 

Apple is close to have:

 

- The absolute best hardware;

- The absolute best mobile and desktop OS;

- The absolute best pro software;

- The absolute best office software;

- The absolute best cloud services and maps.

 

There's only one thing left: Apply their superior search knowledge (spotlight, etc) and make a search engine.

 

That's a single company being better than whole industries.

post #7 of 44

Forget about patents, Apple needs to hire 200 drivers and start covering the earth.

Citing unnamed sources with limited but direct knowledge of a rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of a rumoured device - Comedy Insider (Feb 2014)
Reply
post #8 of 44

So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it.  I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.
 

post #9 of 44
Quote:
Originally Posted by AppleInsider View Post


According to the filing, Apple's invention improves upon current technology, like Google's Street View, which boxes users into a "bubble" that can only be navigated through an input device like a mouse or multitouch screen. For example, in a conventional GUI, the user must "jump" to a panoramic "bubble" at a given intersection and pan within said bubble to move in a desired direction; a tedious experience for mobile device users on the go.

First, the (Apple) invention notes a user must first enter the street-level view, which can be accomplished by "pinching in" on a map, or by selecting a dropped pin icon.

Hmmm. . .
So instead of pinch-zooming or jumping to a bubble like in Google Streetview, Apple's version (if ever used) would be initiated by . . . uh. . . . pinch-zooming or by selecting a dropped pin (Is that like the Google "bubble"?). Gosh, that does sound a lot less tedious for the mobile device user on the go. /s

Now if only Google Streetview could make use of the mobile device's sensors to zoom or rotate the view by simply tilting and/or turning the phone then Apple's patent application would be really really similar to Google's current implementation.

Oh, wait. . .

Navigate within Street View

Use your finger to pan the map.

Drag the Pegman icon in the lower-left corner to a point in the image in order to zoom directly to that point.

Double-tap a point in Street View with one finger to zoom in; touch once with two fingers to zoom out.

Touch Compass mode in the Google Maps menu to use the phone itself to navigate Street View. Tilt, pan, or turn the phone to change the view. Touch Compass mode again to end it.


http://support.google.com/gmm/bin/answer.py?hl=en&answer=1650202
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #10 of 44
Quote:
Originally Posted by jimbo123 View Post

You can't but feel that's its all catchup to Google.

Google had a 7 year head start and within a year Apple has got pretty close.

 

Google were just resting on their laurels thinking they had no competition. Apple maps were a kick in the ass for Google who would have just carried on adding a few new things over the next 10 years. If Apple had been doing maps for 7 years there would be no hope for anyone else to catch up.

post #11 of 44
Quote:
Originally Posted by Ireland View Post

Forget about patents, Apple needs to hire 200 drivers and start covering the earth.

 

Nope, just send out their fleet of top secret iRobots

post #12 of 44
Quote:
Originally Posted by jimbo123 View Post

You can't but feel that's its all catchup to Google.

Google hardly "invented" maps.

 

So the feature evolves, nature of tech.

post #13 of 44
Quote:
Originally Posted by cnocbui View Post

So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it.  I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.
 

 

They have enough data to start the process. After all, if they waited until they had a 100% perfect data set before doing anything, it would never happen

post #14 of 44
Quote:
Originally Posted by cnocbui View Post

So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it.  I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.

 

So, what you are saying is, it would be better for Apple to NOT plan ahead.

 

Brilliant!

post #15 of 44
Quote:
Originally Posted by Evilution View Post

Google had a 7 year head start and within a year Apple has got pretty close.

They had a good example to work from. 1wink.gif
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #16 of 44
That would be great. But wouldn't it be easier to build a common database with google or just use their images just for this? It seems like they'd need to spend a lot of money, time and pollute a lot for something that others have already made.
post #17 of 44
Originally Posted by AppleInsider View Post

... reads apparent patterns of motion of objects in a panoramic image in relation to an observer. By scaling distance data, a device can display the appropriate virtual location ...

 

This is why I think the A7 and its GPU both need to be quad core.

Whether they debut in the iPhone 5S or the iPhone 6.

Sent from my iPhone Simulator

Reply

Sent from my iPhone Simulator

Reply
post #18 of 44
Quote:
Originally Posted by ClemyNX View Post

That would be great. But wouldn't it be easier to build a common database with google or just use their images just for this? It seems like they'd need to spend a lot of money, time and pollute a lot for something that others have already made.

That might be true - if Google were someone that you could trust.

Not to mention, of course, that the cost (both money and other terms) might be too high to do it with Google.

I suspect that Apple's management knew what they were doing when they decided to offer their own mapping solution.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #19 of 44
Quote:
Originally Posted by Evilution View Post

Google had a 7 year head start and within a year Apple has got pretty close.

 

Actually only two years head start when you consider that they have to redo every street scene and map on a two year refresh cycle to stay relevant. Kudos to Apple Maps for updating their aerial photography in my US location... and the resolution is a couple levels higher than it was before.

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #20 of 44
Quote:
Originally Posted by Gatorguy View Post


Hmmm. . .
So instead of pinch-zooming or jumping to a bubble like in Google Streetview, Apple's version (if ever used) would be initiated by . . . uh. . . . pinch-zooming or by selecting a dropped pin (Is that like the Google "bubble"?). Gosh, that does sound a lot less tedious for the mobile device user on the go. /s

Now if only Google Streetview could make use of the mobile device's sensors to zoom or rotate the view by simply tilting and/or turning the phone then Apple's patent application would be really really similar to Google's current implementation.

Oh, wait. . .

Navigate within Street View

Use your finger to pan the map.

Drag the Pegman icon in the lower-left corner to a point in the image in order to zoom directly to that point.

Double-tap a point in Street View with one finger to zoom in; touch once with two fingers to zoom out.

Touch Compass mode in the Google Maps menu to use the phone itself to navigate Street View. Tilt, pan, or turn the phone to change the view. Touch Compass mode again to end it.


http://support.google.com/gmm/bin/answer.py?hl=en&answer=1650202

 

Your comment is a bit hard to understand but it seems to imply that Apple's solution is the same as what Google already has.  It isn't.  

 

If that's what you think, you need to re-read the article more carefully. 

post #21 of 44
Quote:
Originally Posted by charlituna View Post

 

Nope, just send out their fleet of top secret iRobots

 

I know you're joking, but the data set could be generated faster and much cheaper using drones flying quickly up and down streets in a given city.  Apple could even make their own iOS controlled hovering drones with a feature that lets the user upload to the dataset in the cloud so as to crowdsource the images and create better coverage of rural areas.  

 

The biggest barrier to this much more efficient way of collecting the data, isn't the technology, but the inevitable outcry from seniors and old-thinkers over "privacy" and the inevitable outlawing of drone flights by the various city-halls for the same reason.  

 

It's similar to how the Segway scooter *could* have revolutionised sidewalk traffic if it wasn't immediately outlawed from the sidewalks in almost every city in North America within weeks of it's availability being announced.  Leaving a device designed for the sidewalk, to be stuck in the road fighting for a right of way with bicycles, simply because of the short-sighted pig-headedness of local "authorities."  

post #22 of 44
Quote:
Originally Posted by Gazoobee View Post

Quote:
Originally Posted by charlituna View Post

 

Nope, just send out their fleet of top secret iRobots

 

I know you're joking, but the data set could be generated faster and much cheaper using drones flying quickly up and down streets in a given city.  Apple could even make their own iOS controlled hovering drones with a feature that lets the user upload to the dataset in the cloud so as to crowdsource the images and create better coverage of rural areas.  

 

The biggest barrier to this much more efficient way of collecting the data, isn't the technology, but the inevitable outcry from seniors and old-thinkers over "privacy" and the inevitable outlawing of drone flights by the various city-halls for the same reason.  

 

It's similar to how the Segway scooter *could* have revolutionised sidewalk traffic if it wasn't immediately outlawed from the sidewalks in almost every city in North America within weeks of it's availability being announced.  Leaving a device designed for the sidewalk, to be stuck in the road fighting for a right of way with bicycles, simply because of the short-sighted pig-headedness of local "authorities."  

I think there are certain considerations for public safety as well as municipal liability insurance issues that you fail to take into account with regard to your comments on the Segway which similarly could be applied to the drone situation too. For example in many states and on all federal land, in order to fly a motorized model aircraft, one must obtain a permit and have proof of liability insurance. It is not just a privacy issue, but also a public safety concern.

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #23 of 44
Quote:
Originally Posted by cnocbui View Post

So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it.  I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.

 

Other companies have map data sets than Google -- and, likely, any of them are more trustworthy than Google!

Navteq, owned by Nokia, have excellent maps, street view and just acquired a 3D mapping company.
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #24 of 44
Quote:
Originally Posted by jimbo123 View Post

You can't but feel that's its all catchup to Google.

Right. Because Google never plays catch up to anyone, and they don't need competition /s

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply
post #25 of 44
Quote:
Originally Posted by mstone View Post

I think there are certain considerations for public safety as well as municipal liability insurance issues that you fail to take into account with regard to your comments on the Segway which similarly could be applied to the drone situation too. For example in many states and on all federal land, in order to fly a motorized model aircraft, one must obtain a permit and have proof of liability insurance. It is not just a privacy issue, but also a public safety concern.

 

Well I certainly simplified the situation for the sake of my argument and you make a good point about the liability which was a big issue at the time.  

 

I would argue in return that individuals are currently allowed to tear down the sidewalk at 20-30 clicks in one of those scooters for fat people, and no one has ever raised the liability issue or complained about that.  Segways when introduced, actually had a lower top speed than those things, take up less space, are far more maneuverable, and included a programmatic option for "capping" the speed because the inventor foresaw this very objection.  None of that seemed to matter. 

 

Personally, I thought the various city governments reaction to the Segway in the US was borderline hysteria. I would also argue that such things should be handled federally rather than locally.  There is a clear public interest in having the same rules of the road across the country.  

post #26 of 44
Quote:
Originally Posted by Gazoobee View Post

I would argue in return that individuals are currently allowed to tear down the sidewalk at 20-30 clicks in one of those scooters for fat people, and no one has ever raised the liability issue or complained about that.  

I know you see the problem as a 'fat person' or 'old person' issue but in many cases it is a disabled person in an electric wheel chair, which I agree is sometimes dangerous for regular pedestrians and I almost got run down by one just last week which really annoyed me how fast she was going, but nevertheless that is apparently a liability risk the city and public businesses are willing to accept in the course of enabling and not discriminating against disabled persons which IS a federal law. 

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #27 of 44
This has already been done before, how can you patent something like this. Layers had an app that does this exact thing. This is nothing new, sorry.
post #28 of 44
This is fantastic. I proposed this for 3D Flyover while in Beta.

Speaking of unmanned aerial drones. Apple might be able to dramatically increase the speed of capturing 3D Flyover data using unmanned aerial drones.
post #29 of 44
Quote:
Originally Posted by citycrazed25 View Post

This has already been done before, how can you patent something like this. Layers had an app that does this exact thing. This is nothing new, sorry.

No.


post #30 of 44
Quote:
Originally Posted by charlituna View Post

 

They have enough data to start the process. After all, if they waited until they had a 100% perfect data set before doing anything, it would never happen


Really?  How many world cities do they have street view images for so far?

post #31 of 44
Quote:
Originally Posted by Dick Applebaum View Post


Other companies have map data sets than Google -- and, likely, any of them are more trustworthy than Google!

Navteq, owned by Nokia, have excellent maps, street view and just acquired a 3D mapping company.


And that helps Apple, and is relevant to them, how?

post #32 of 44
Quote:
Originally Posted by Gazoobee View Post

 

I know you're joking, but the data set could be generated faster and much cheaper using drones flying quickly up and down streets in a given city.  Apple could even make their own iOS controlled hovering drones with a feature that lets the user upload to the dataset in the cloud so as to crowdsource the images and create better coverage of rural areas. 

 

And these drones won't, of course, run into or hit anything or cause anyone problems in any way?

post #33 of 44
Quote:
Originally Posted by cnocbui View Post

Quote:
Originally Posted by Dick Applebaum View Post

Other companies have map data sets than Google -- and, likely, any of them are more trustworthy than Google!


Navteq, owned by Nokia, have excellent maps, street view and just acquired a 3D mapping company.


And that helps Apple, and is relevant to them, how?

Apple could easily cut a deal with Nokia to provide map data. Apple and Nokia already have cross-license agreements to share each others' technology... so, not too far beyond the realm of possibility.

Also, Apple could buy Navteq outright -- Nokia recently sold their HQ buildings to raise cash... so, not too far beyond the realm of possibility.

Or Apple could invest in the Naviteq division of Nokia which is losing money and Nokia has never recovered the $8.1 billion it paid for Naviteq... so, not too far beyond the realm of possibility....

You must be blind (or lazy) not to realize the potential of companies to do business with other companies who they can trust (with proper contractual agreements).

Look it up -- Bing is your friend!


I'd much rather Apple cut a deal with Nokia than Google!
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #34 of 44
Quote:
Originally Posted by jimbo123 View Post

You can't but feel that's its all catchup to Google.

 

You can't help but feel it's all catch-up to and surpass Google.

post #35 of 44
Quote:
Originally Posted by mstone View Post

Quote:
Originally Posted by Evilution View Post

Google had a 7 year head start and within a year Apple has got pretty close.
Actually only two years head start when you consider that they have to redo every street scene and map on a two year refresh cycle to stay relevant. Kudos to Apple Maps for updating their aerial photography in my US location... and the resolution is a couple levels higher than it was before.

Yes! Orange County looks great! How about your former home in Central America!


I posted the following to another forum:
Quote:
Originally Posted by Dick Applebaum View Post

On a different note...

Much to my pleasant surprise, I noticed that Apple has updated iOS Maps to provide 3D Flyover of Pasadena, CA (and most of the surrounding cities)...

So you can: visit the Rose Bowl; follow the historic Pasadena Freeway to Dodger Stadium and beyond to Olvera Street in LA; tour Santa Anita Race Track (where I learned to drive); see El Monte Airport (where I learned to fly) follow the Spanish Missionaries on their quest up Mission Blvd / El Camino Real... or just do a 3D Flyover of Heidi Fleiss...
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #36 of 44
Quote:
Originally Posted by Dick Applebaum View Post

Yes! Orange County looks great! How about your former home in Central America!

 

Sorry no. Same low res outdated useless grayscale images. I'm currently in CA not CA. :)

 

@MacBook Pro : Great vid. Haven't watch that one for awhile!

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #37 of 44
Quote:
Originally Posted by AppleSauce007 View Post

Hold on there... Shouldn't E Broadway be on the right hand side and W Broadway on the left?
You know what they say... If the patent is wrong then...  LOL.
Don't know, but maybe it means you are going south and the north street appeared on accedent.

Well maybe we will see this in IOS 7, I expect a major feature push for it after all the complaints.
post #38 of 44
Quote:
Originally Posted by Gatorguy View Post


They had a good example to work from. 1wink.gif

And Gee what a suprise Gurggle bought the tech to get a start in maps.  

 

"Google Maps first started as a C++ program designed by two Danish brothers Lars and Jens Rasmussen at the Sydney-based company Where 2 Technologies. It was first designed to be separately downloaded by users, but the company later pitched the idea for a purely Web-based product to Google management, changing the method of distribution.[38] In October 2004 the company was acquired by Google Inc[39] where it transformed into the web application Google Maps".

 

So I guess they bought there example and innovated¡

post #39 of 44
Quote:
Originally Posted by cnocbui View Post

So Apple have this great idea on how to better use a data set they don't have, while Google actually have a data set and ways of using it.  I thing Apple have overlooked one or two minor details, but that's pretty consistent with their whole Maps initiative.
 

 

SO because Apple maps is not already superior in every single way to a product that Google has had on the market for more than a decade and has been working on since it first launched, that means Apple 'overlooked' things and you're angry at them for working hard to improve it further? A few months ago Apple was completely reliant on Google for mapping, and Google maps is considered untouchable. Now, the vast majority of iOS users are using Apple maps and are perfectly happy with it- I'd say that's progress. Even Apple can't conjure up magic and map the world perfectly and instantly, along with its hundreds of millions of POIs all from imperfect data sources. Give them a shred of credit at least instead of spitting on the fact that they submitting patents to improve the app further. 

post #40 of 44
Quote:
Originally Posted by Mechanic View Post

And Gee what a suprise Google bought the tech to get a start in maps
So I guess they bought there example and innovated

I agree. Looks like innovation to me too. Start with a rough idea and build on it.
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: General Discussion
AppleInsider › Forums › General › General Discussion › Patent filing suggests Apple is working on 'Street View' mapping technology