Apple research paper details LiDAR-based 3D object recognition for autonomous vehicle navi...

Posted:
in General Discussion edited November 2017
Apple researchers are pushing forward with efforts to bring autonomous vehicle systems to public roads, and last week published an academic paper outlining a method of detecting objects in 3D point clouds using trainable neural networks. While still in its early stages, the technology could mature to improve accuracy in LiDAR navigation solutions.




Like other recent scholarly articles published by Apple engineers, the latest entry, "VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection" by AI researcher Yin Zhou and machine learning specialist Oncel Tuzel, was made public (PDF link) through the arXiv archive of scientific papers.

In its article, Apple notes accurate detection of objects in 3D point clouds, like those generated by LiDAR arrays, is a sticking point in a number of burgeoning real-world applications. From autonomous cars to robotic vacuums, machines that navigate the world around them without the assistance of human operators need to detect critical object with speed and precision.

Compared to 2D image-based detection, LiDAR technology proves to be a more reliable alternative as it provides depth information to better localize objects in space, Apple says. However, LiDAR point clouds, generated by emitting laser pulses and logging the time it takes for the light to return after bouncing off a solid surface, are sparse and have highly variable point density, thus causing a host of problems.

Current state-of-the-art techniques designed to manage data interpretation involve manually creating feature representations for said point clouds. Some methods project point clouds into a bird's eye perspective view, while others transform the data into 3D voxel grids and encode each voxel with certain features. Manually crafting feature representations introduce an "information bottleneck" that restricts such systems from efficiently leveraging 3D shape information, according to Apple.

Instead, Zhou and Tuzel propose the implementation of a trainable deep architecture for point cloud based 3D detection. The framework, called VoxelNet, uses voxel feature encoding (VFE) layers to learn complex features for characterizing 3D shapes. In particular, the technique breaks down the point cloud into 3D voxels, encodes the voxels via stacked VFE layers and renders a volumetric representation.

In tests, Apple's methodology showed promise, outperforming current LiDAR based detection algorithms and image-based approaches "by a large margin." This is according to evaluations run through the KITTI 3D object detection benchmark, which Apple used to assess its process. VoxelNet was trained to detect three basic objects -- car, pedestrian and cyclist -- in a variety of tests.

Aside from theoretical research, Apple is currently evaluating a self-driving vehicle testbed on the streets of Cupertino, Calif. The company's efforts in autonomous technology began under the "Project Titan" initiative, which sought to build a branded self-driving car from the ground up. After significant investment and multiple employee reassignments, Titan hit a number of snags and was ultimately put on ice in late 2016, though remnants of the initiative, like supporting software and hardware, remain active.

A report in August claimed Apple is looking to parlay the technology into an autonomous shuttle that will ferry employees between its Silicon Valley campuses.

While Apple's research paper focuses heavily on autonomous vehicle navigation, the tech described can also be applied to augmented reality systems that use depth mapping hardware to detect real-world objects. The new iPhone X sports equipment similar to LiDAR arrays in its front-facing TrueDepth camera, which incorporates a miniaturized dot projector for accurate depth mapping operations. If TrueDepth's range is extended, and mounted on the rear of a portable device, it could potentially be paired with advanced software to power an entirely new consumer AR experience.

Comments

  • Reply 1 of 15
    jd_in_sbjd_in_sb Posts: 1,483member
    Those images remind me of The Terminator
    SoliSpamSandwichwatto_cobra
  • Reply 2 of 15
    foggyhillfoggyhill Posts: 4,767member
    Funny how the rumor it was "put on ice" is seemingly notworthy even though it was pretty sketchy and there is very little info on the actual scope of project Titan.
    Note that Apple spent 1B on didi investment after this thing was "put on ice".... Wonder what thati is all about huh.

    Doubtful that this thing is in any way put on ice, they just shifted their focus on public transport (and automated fleets) (especially in asia) for now cause it will be an easier target and you can gather a hell of a lot more data there than anywhere else. Building Apple Transport fleet would be a much more interesting endeavor than building doing a self-driving car sold to individuals.

    BTW, analysts just throw things at wall and hope the tech press will be a PR agent for their turd throwing feats. The whole "news" about Apple's foray in cars fit in that mold.

    Many of the people that were hired in their buildup for Titan, especially people like mechanical engineers, are STILL AT APPLE. What are they doing there?
    repressthistmaywatto_cobracornchip
  • Reply 3 of 15
    radarthekatradarthekat Posts: 3,012moderator
    I’m just happy the article doesn’t include that Motorweek iCar design render.  Whew!
    gregoriusmrepressthistmaybikertwinSpamSandwichmacxpressschlackcornchipwatto_cobra
  • Reply 4 of 15
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    watto_cobra
  • Reply 5 of 15
    Rayz2016Rayz2016 Posts: 4,556member
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    Your dots are the ones zero feet away from the LIDAR?  

    Or am I missing something. 


  • Reply 6 of 15
    radarthekatradarthekat Posts: 3,012moderator
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    I’d guess each unit could have its own modulation pattern.
    schlack
  • Reply 7 of 15
    Rayz2016 said:
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    Your dots are the ones zero feet away from the LIDAR?  

    Or am I missing something. 


    The dots are projected on the road ahead.. Elapsed time between light emission and recapture determines distance..  So what if 50 cars are projecting dots?
    edited November 2017
  • Reply 8 of 15
    Rayz2016Rayz2016 Posts: 4,556member
    Rayz2016 said:
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    Your dots are the ones zero feet away from the LIDAR?  

    Or am I missing something. 


    The dots are projected on the road ahead.. Elapsed time between light emission and recapture determines distance..  So what if 50 cars are projecting dots?
    Yes, I am missing something. 

    That’s a good question. 


  • Reply 9 of 15
    tmaytmay Posts: 3,567member
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    That's a relevant issue.

    My guess is that there will be a peer to peer/mesh/ad hoc networking capability for AV's in the near future, and that cars in LIDAR range with each other would just negotiate for a narrow time slot to make the scan; it's likely not that bandwidth intensive. For now, we consider only a single vehicle's AV capability, not the swarm's, and that is not the sufficient for the future.

    At the same time, vehicles would need to coordinate with each other to manage traffic, and to build the "packets" of vehicles that platoon to near and distant locations. Autonomous vehicle will have to be anything but that once they are in traffic, and let's just call that part SkyNet. At some point in the future, "classic" vehicles with human drivers, will have to be retrofitted with transponders giving relevant data about the vehicle's and driver's capability to the swarm. Good drivers would have a higher score based on ML that would allow them to interact closer with the swarm, or individual AV's. We shouldn't forget all of the ethical issues that have to be solved once we loose AV's on the world in large numbers.

    Then, sometime in the future, there would be elimination of classic vehicles entirely on major arteries in urban areas, followed by restriction to limited areas of use. 

    Sadly, the odds on me still being alive when that happens is pretty good, but by then, my age will have me happy to have the assist from AV's.
    edited November 2017 patchythepirate
  • Reply 10 of 15
    brucemcbrucemc Posts: 1,525member
    jd_in_sb said:
    Those images remind me of The Terminator
    No doubt that is how it will end...
    cornchip
  • Reply 11 of 15
    brucemcbrucemc Posts: 1,525member
    And Tesla is trying to get to full autonomy without Lidar (and by the end of this year based on Musk's last comments on the topic, I believe)?  Don't think it will happen.  To get to the level of autonomy / safety / reliability required, I think Lidar is a must.

    I do believe fully self-driving cars will come, but the timeframe for widespread use is longer than most expect.  I use technology all the time, every day, and none of it is perfected to the level that I would expect self-driving to require.  
    tmaycornchipwatto_cobra
  • Reply 12 of 15
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    I’d guess each unit could have its own modulation pattern.
    Yup this is how it will work. This is how its done for RADAR, which a ton of cars on the road are already using. Each LIDAR system will attend to its own reflected light.
    edited November 2017 watto_cobracornchip
  • Reply 13 of 15
    tmay said:
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    That's a relevant issue.

    My guess is that there will be a peer to peer/mesh/ad hoc networking capability for AV's in the near future, and that cars in LIDAR range with each other would just negotiate for a narrow time slot to make the scan; it's likely not that bandwidth intensive. For now, we consider only a single vehicle's AV capability, not the swarm's, and that is not the sufficient for the future.

    At the same time, vehicles would need to coordinate with each other to manage traffic, and to build the "packets" of vehicles that platoon to near and distant locations. Autonomous vehicle will have to be anything but that once they are in traffic, and let's just call that part SkyNet. At some point in the future, "classic" vehicles with human drivers, will have to be retrofitted with transponders giving relevant data about the vehicle's and driver's capability to the swarm. Good drivers would have a higher score based on ML that would allow them to interact closer with the swarm, or individual AV's. We shouldn't forget all of the ethical issues that have to be solved once we loose AV's on the world in large numbers.

    Then, sometime in the future, there would be elimination of classic vehicles entirely on major arteries in urban areas, followed by restriction to limited areas of use. 

    Sadly, the odds on me still being alive when that happens is pretty good, but by then, my age will have me happy to have the assist from AV's.
    If I had my druthers, it would be insurance premiums which would 'drive' non-autonomous cars into museums, not the laws. The effectiveness of competing systems should be the only concern.
  • Reply 14 of 15
    tmay said:
    Am I missing something, but if all vehicles are equipped with LIDAR, how does each vehicle distinguish its own dots from the others'?
    Can only work if you're the only one on the road, it seems to me..
    That's a relevant issue.

    My guess is that there will be a peer to peer/mesh/ad hoc networking capability for AV's in the near future, and that cars in LIDAR range with each other would just negotiate for a narrow time slot to make the scan; it's likely not that bandwidth intensive. For now, we consider only a single vehicle's AV capability, not the swarm's, and that is not the sufficient for the future.

    At the same time, vehicles would need to coordinate with each other to manage traffic, and to build the "packets" of vehicles that platoon to near and distant locations. Autonomous vehicle will have to be anything but that once they are in traffic, and let's just call that part SkyNet. At some point in the future, "classic" vehicles with human drivers, will have to be retrofitted with transponders giving relevant data about the vehicle's and driver's capability to the swarm. Good drivers would have a higher score based on ML that would allow them to interact closer with the swarm, or individual AV's. We shouldn't forget all of the ethical issues that have to be solved once we loose AV's on the world in large numbers.

    Then, sometime in the future, there would be elimination of classic vehicles entirely on major arteries in urban areas, followed by restriction to limited areas of use. 

    Sadly, the odds on me still being alive when that happens is pretty good, but by then, my age will have me happy to have the assist from AV's.
    If I had my druthers, it would be insurance premiums which would 'drive' non-autonomous cars into museums, not the laws. The effectiveness of competing systems should be the only concern.
    Which would be sad for classics. I for one still dream of having a 1956 Chevy Bel-Air. 
  • Reply 15 of 15
    cornchipcornchip Posts: 1,252member
    foggyhill said:

    Building Apple Transport fleet would be a much more interesting endeavor than building doing a self-driving car sold to individuals. 

    —————————————-

    Many of the people that were hired in their buildup for Titan, especially people like mechanical engineers, are STILL AT APPLE. What are they doing there?

    It does seem like something Apple would do, and I’m sure they’re considering it, but I’d guess their biggest concern about taking “Titan” in that direction is the fact that what they were working on would then be out in the open. Something Apple isn’t generally keen on. 

    —————————————-

    Good point. Maybe the analysts have a good answer for that… analysts? 🏏🏏🏏


Sign In or Register to comment.