Apple's mysterious vans are almost certainly for 3D mapping, not autonomous driving
Apple inadvertently made waves earlier this week with the revelation that the company was behind a fleet of camera-equipped minivans found roaming streets throughout the country. Initial rumors suggested that they may be a step towards a self-driving "iCar," but digging done by AppleInsider suggests that the vehicles are not actually early indications of an autonomous Apple automobile.
One of Apple's sensor-laden vans, spotted in Hawaii by AppleInsider reader matthawaii.
Much-maligned technology analyst Rob Enderle was the catalyst for the rumors, telling San Francisco CBS affiliate KPIX that he believed the blue Dodge Caravan seen by the station had "too many cameras" to be used for mapping. That Apple has not been issued permits for testing autonomous vehicles is inconsequential, Enderle believes, as they could have partnered with a firm that already has the necessary clearances.
Unfortunately for those hoping to see an iCar, the preponderance of evidence -- based on local motor vehicle laws, Apple's hiring efforts, and the camera-equipped cars themselves -- suggests that this is simply not true.
Autonomous vehicle testing on public roads is legal in California, Florida, Michigan, Nevada, and Washington, D.C. It is not authorized in New York, Hawaii, or Wisconsin, though Hawaii does have a legalization measure under consideration in the legislature.
The current legal landscape for autonomous vehicle testing on public roads, via Wikipedia.
Legal issues notwithstanding, sending early-stage autonomous vehicles so far away from Apple's home base in Cupertino also seems suspect. Google's driverless car, under development for nearly a decade, rarely leaves the confines of Mountain View.
This is largely because Google's vehicles, despite being considered at the forefront of automated driving, can't independently process unknown routes. They make decisions by combining data from onboard sensors with ultra-precise maps that are preprocessed on Google's server farms and then loaded onto the car before it sets off.
During development of the Apple Watch, for instance, Apple hired numerous high-level personnel from the fashion and biomedical industries. Among those were former Yves St. Laurent CEO Paul Deneve, fitness guru Jay Blahnik, former Nike design director Ben Shaffer, and former Masimo chief medical officer Michael O'Reilly.
Each of those hires can now be tied to specific features of the Apple Watch: its positioning as a fashion accessory, its focus on fitness tracking, and its optoelectric heart rate monitor.
There has been no comparable migration of automotive talent to Cupertino. Research conducted by AppleInsider has discovered that Apple has brought in at least a dozen former engineers from electric car startup Tesla in the last two years. However, none of those --?nor any recent additions from global automotive leaders Toyota, Nissan, GM, or Ford, according to LinkedIn --?have been senior-level hires. Many, if not most, of these auto industry experts are focused on battery and materials technology, while a small percentage seem to be working on Apple's CarPlay initiative.
In fact, more employees have moved from Apple to Tesla than in the other direction. At least 150 workers have made the jump from Cupertino to work with Elon Musk, Bloomberg notes, but not for a lack of effort on Apple's part.
"Apple tries very hard to recruit from Tesla," Musk said, offering massive signing bonuses and salary bumps. "But so far they've actually recruited very few people."
Apple has made some interesting hires when it comes to mapping. Former Navteq executive Torsten Krenz, who previously led Navteq's worldwide ground truth and digital mapping operations, moved to Apple last August. He works alongside Philip Stanger, the former CEO of indoor navigation firm Wifarer, whom Apple nabbed earlier last year.
Several positions which Apple has yet to fill also point to the ramp-up of a major data collection push.
In the U.K., Apple is hiring a "ground-truthing team lead" to "lead a team of contractors in a data-collection project." It's worth noting that this is not the first time Apple has started a ground truth exercise, which is designed to correlate digital data with real-world observations; the company also hired nearly a dozen "ground truth local experts" in 2013.
Closer to home, Apple is looking for a pair of regional managers for Maps that will be expected to oversee "data collection management" and "leveraging use of automatic detection tools (Probe, Images, etc.)" Taken together, it seems clear that Apple is in the midst of a significant overhaul and expansion of the data that feeds Maps.
They're nearly ideal for data collection, however. Tall, with a roomy interior for equipment and a smooth ride. In fact, Google's first-generation Street View car was a Chevy Astro van.
An educated guess at identifying the sensors on Apple's van. Original image via Claycord.com
The technology atop the vans also points to data collection. Returning to Enderle's assessment of "too many cameras" for a mapping vehicle, the opposite appears true: it doesn't seem to have enough for a self-driving car.
Google uses pairs of cameras mounted around the exterior of its cars for depth perception. This system works much the same way as human eyes, with computers analyzing the parallax created by the slightly-different images.
Looking at Apple's vans, we can see that it seems to have blind spots in the front and rear, positions taken up by the LiDAR modules. While not a big deal when compositing images for mapping, it would be a catastrophic loss of situational awareness for a car that's trying not to run into things.
Google's prototype Street View van.
The balance of the equipment --?including the wheel encoder that can be seen mounted to the side and the two dish-shaped GPS antennas up top --?appears to be part of an inertial navigation system almost identical to those used by Google, Navteq, Microsoft, and other mapping firms on their ground truth vehicles.
Finally, the most glaring discrepancy of all: Apple, one of the most secretive companies in the world, leased these vans under their own name. With a well-known sense of security paranoia that reportedly extends to asking employees to check in to hotels under assumed identities in the run-up to major product launches, the idea that the company would depend on civil servants at the California Department of Motor Vehicles to keep a top-secret robotics project under wraps seems farfetched.
Apple's FlyOver feature, which gathers data from planes flying overhead, has little support for street-level features and runs into trouble with multi-layered landscapes. This is especially noticeable on tightly-packed city streets, where things like trees and awnings can cause distortion in the images.
These problems could be mitigated by combining the FlyOver data with imagery and three-dimensional maps captured by ground vehicles, giving Apple a significant leg up in an area that is sure to become more important as augmented reality developments continue.
An example of street-level distortion in Apple's Flyover
In addition to accuracy improvements and 3D world generation, driving the streets would have the added benefit of reducing Apple's dependence on outside vendors. The company currently combines data from a number of sources, including TomTom, OpenStreetMap, and a variety of regional players, such as Autonavi in China.
While mapping may not be the most glamorous use for Apple's fleet of mystery vans, it's the most pragmatic --?and, almost certainly, the correct one.
One of Apple's sensor-laden vans, spotted in Hawaii by AppleInsider reader matthawaii.
Much-maligned technology analyst Rob Enderle was the catalyst for the rumors, telling San Francisco CBS affiliate KPIX that he believed the blue Dodge Caravan seen by the station had "too many cameras" to be used for mapping. That Apple has not been issued permits for testing autonomous vehicles is inconsequential, Enderle believes, as they could have partnered with a firm that already has the necessary clearances.
Unfortunately for those hoping to see an iCar, the preponderance of evidence -- based on local motor vehicle laws, Apple's hiring efforts, and the camera-equipped cars themselves -- suggests that this is simply not true.
Legality & practicality
Aside from the vehicles spotted in and around San Francisco, a nearly identical van bearing California license plates was seen last September in Brooklyn, New York. AppleInsider readers have also reported sightings in Hawaii; Madison, Wisc.; and Panama City, Panama.Autonomous vehicle testing on public roads is legal in California, Florida, Michigan, Nevada, and Washington, D.C. It is not authorized in New York, Hawaii, or Wisconsin, though Hawaii does have a legalization measure under consideration in the legislature.
The current legal landscape for autonomous vehicle testing on public roads, via Wikipedia.
Legal issues notwithstanding, sending early-stage autonomous vehicles so far away from Apple's home base in Cupertino also seems suspect. Google's driverless car, under development for nearly a decade, rarely leaves the confines of Mountain View.
This is largely because Google's vehicles, despite being considered at the forefront of automated driving, can't independently process unknown routes. They make decisions by combining data from onboard sensors with ultra-precise maps that are preprocessed on Google's server farms and then loaded onto the car before it sets off.
Hiring
Such a widespread rollout of autonomous vehicles would also surely necessitate a not-insignificant increase in headcount for the project. In particular, bringing on engineering leadership with automotive expertise would seem to be a key component for success.During development of the Apple Watch, for instance, Apple hired numerous high-level personnel from the fashion and biomedical industries. Among those were former Yves St. Laurent CEO Paul Deneve, fitness guru Jay Blahnik, former Nike design director Ben Shaffer, and former Masimo chief medical officer Michael O'Reilly.
Each of those hires can now be tied to specific features of the Apple Watch: its positioning as a fashion accessory, its focus on fitness tracking, and its optoelectric heart rate monitor.
More engineers have moved from Apple to Tesla than have moved from other carmakers to Apple.
There has been no comparable migration of automotive talent to Cupertino. Research conducted by AppleInsider has discovered that Apple has brought in at least a dozen former engineers from electric car startup Tesla in the last two years. However, none of those --?nor any recent additions from global automotive leaders Toyota, Nissan, GM, or Ford, according to LinkedIn --?have been senior-level hires. Many, if not most, of these auto industry experts are focused on battery and materials technology, while a small percentage seem to be working on Apple's CarPlay initiative.
In fact, more employees have moved from Apple to Tesla than in the other direction. At least 150 workers have made the jump from Cupertino to work with Elon Musk, Bloomberg notes, but not for a lack of effort on Apple's part.
"Apple tries very hard to recruit from Tesla," Musk said, offering massive signing bonuses and salary bumps. "But so far they've actually recruited very few people."
Apple has made some interesting hires when it comes to mapping. Former Navteq executive Torsten Krenz, who previously led Navteq's worldwide ground truth and digital mapping operations, moved to Apple last August. He works alongside Philip Stanger, the former CEO of indoor navigation firm Wifarer, whom Apple nabbed earlier last year.
Several positions which Apple has yet to fill also point to the ramp-up of a major data collection push.
In the U.K., Apple is hiring a "ground-truthing team lead" to "lead a team of contractors in a data-collection project." It's worth noting that this is not the first time Apple has started a ground truth exercise, which is designed to correlate digital data with real-world observations; the company also hired nearly a dozen "ground truth local experts" in 2013.
Closer to home, Apple is looking for a pair of regional managers for Maps that will be expected to oversee "data collection management" and "leveraging use of automatic detection tools (Probe, Images, etc.)" Taken together, it seems clear that Apple is in the midst of a significant overhaul and expansion of the data that feeds Maps.
The cars
Apple's choice of Dodge Caravans puts yet another nail in the autonomous vehicle coffin. The minivans are big, comparatively unmaneuverable, and old --?three characteristics that a company developing futuristic robotic systems is sure to avoid when selecting a test mule.They're nearly ideal for data collection, however. Tall, with a roomy interior for equipment and a smooth ride. In fact, Google's first-generation Street View car was a Chevy Astro van.
An educated guess at identifying the sensors on Apple's van. Original image via Claycord.com
The technology atop the vans also points to data collection. Returning to Enderle's assessment of "too many cameras" for a mapping vehicle, the opposite appears true: it doesn't seem to have enough for a self-driving car.
Google uses pairs of cameras mounted around the exterior of its cars for depth perception. This system works much the same way as human eyes, with computers analyzing the parallax created by the slightly-different images.
Looking at Apple's vans, we can see that it seems to have blind spots in the front and rear, positions taken up by the LiDAR modules. While not a big deal when compositing images for mapping, it would be a catastrophic loss of situational awareness for a car that's trying not to run into things.
Google's prototype Street View van.
The balance of the equipment --?including the wheel encoder that can be seen mounted to the side and the two dish-shaped GPS antennas up top --?appears to be part of an inertial navigation system almost identical to those used by Google, Navteq, Microsoft, and other mapping firms on their ground truth vehicles.
Finally, the most glaring discrepancy of all: Apple, one of the most secretive companies in the world, leased these vans under their own name. With a well-known sense of security paranoia that reportedly extends to asking employees to check in to hotels under assumed identities in the run-up to major product launches, the idea that the company would depend on civil servants at the California Department of Motor Vehicles to keep a top-secret robotics project under wraps seems farfetched.
So what then?
All of this adds up to only one conclusion: These vans are meant for mapping, not shuttling kids to school while mom sleeps off the riesling hangover. This doesn't necessarily mean that we'll see Apple Street View in future iOS versions, but it seems a likely possibility.Apple's FlyOver feature, which gathers data from planes flying overhead, has little support for street-level features and runs into trouble with multi-layered landscapes. This is especially noticeable on tightly-packed city streets, where things like trees and awnings can cause distortion in the images.
These problems could be mitigated by combining the FlyOver data with imagery and three-dimensional maps captured by ground vehicles, giving Apple a significant leg up in an area that is sure to become more important as augmented reality developments continue.
An example of street-level distortion in Apple's Flyover
In addition to accuracy improvements and 3D world generation, driving the streets would have the added benefit of reducing Apple's dependence on outside vendors. The company currently combines data from a number of sources, including TomTom, OpenStreetMap, and a variety of regional players, such as Autonavi in China.
While mapping may not be the most glamorous use for Apple's fleet of mystery vans, it's the most pragmatic --?and, almost certainly, the correct one.
Comments
Why didn't you just come out and say APPLE IS DOOMED!?
It would be nice if they could merge these ground level videos seamlessly into FlyOver allowing users to go all the way to the ground and then look back up and sideways in 3D.
Between technologies obtained from C3 Technologies, PrimeSense and Apple know how, they can do wonders with Apple Map.
Expect huge updates for Apple Maps by WWDC June 8-12.
Quote:
Apple should have set up a shell company to collect this data 5 years ago. Starting now?
Why didn't you just come out and say APPLE IS DOOMED!?
He's going to wait 5 years...
(and I think the Google van looks like the Monitor)
I can't disagree but hopefully technology has improved in those 5 years in a way that will make the wait worth it. Many times first isn't always best ... I can mention many example this one always comes to mind first, NTSC vs PAL in the old days of TV.
Thank you for pointing out "Driver" in the photo. That was very helpful. (sarcasm)
Perhaps it was to underscore that these aren't autonomous vehicles.
Thank you for pointing out "Driver" in the photo. That was very helpful. (sarcasm)
Perhaps it was to underscore that these do not appear to be autonomous vehicles.
Not that I mind him being quoted mind you. It's pretty amusing to see him make a complete fool of himself time and time again.
Oh, these vans are made for mappin'
And that's just what they'll do
One of these days these vans are gonna drive right over you.
These things wouldn't happen if you ran Apple. Send your resume to the board of directors.
Kind of like those scanner-bots in the movie [I]Prometheus[/I] - send them out, let them criss-cross over a range of terrain then return to home.
Maybe not cheaper at first, but you're going to be doing this repeatedly over time.
Then again people would freak out at the idea of "iBots" flying over streets collecting data.
Just a matter of time.
nice.
Or consumer VideoTape format:
BetaMax - Better Video and Sound quality
VHS - Two-hour Capacity
... Good enough wins
Someone tell me why Enderle is both maligned and oft-quoted? The guy has a notorious history of being wrong when it comes to Apple yet is often the go-to guy for quotes about Apple.
Not that I mind him being quoted mind you. It's pretty amusing to see him make a complete fool of himself time and time again.
Enderle has both the high distinction of being almost always wrong and being a loud mouth. But he knows about every journalist in tech, is the goto outside advisor or analyst for many big firms that he manages to keep his credibility despite lack of vision simply by being quoted before being proved wrong.
Gruber summarized it quite aptly more than 10 years ago : "Rob Enderle: Putting the ‘Anal’ in ‘Analyst’ "
Why didn't you just come out and say APPLE IS DOOMED!?
No need to go hyperbolic. Indeed it's a good question why it took them so long. Probably they had more important problems to sort and were looking for a better solution than just displaying pictures in a regular pattern like Google does. I'm impatient to see what the result will be, street imagery is one important thing missing from maps now. Add a better search, transit directions and point of interest search and it will be perfect.
Drones may be cheaper but are they authorized to go down streets at a low altitude? Plus vans aren't noticeable per se.
Not to mention the weight and size they would need to be to carry the mapping equipment to produce useable data