Apple greatly reduced autonomous vehicle testing in 2019, fleet of 23 cars drove only 7,50...
Apple's autonomous vehicle testing program saw a significant drawdown in 2019, with its fleet driving 72,201 miles less than it did in 2018. The company also appears to have changed its disengagement reporting protocol, information commonly used to gauge system efficacy.
Apple's self-driving car testbed in California.
According to numbers submitted by Apple to California's Department of Motor Vehicles, 23 of the company's 69 registered self-driving testbeds were active between December 2018 and November 2019. Collectively, the fleet of modified Lexus RX450h SUVs logged 7,544 miles during the reporting period, down from 79,754 miles in 2018.
Apple reported 64 disengagements across the 7,544 miles driven, which equates to 8.48 disengagements per 1,000 miles. This compares to a whopping 69,510 disengagements, or 871.65 disengagements per 1,000 miles, recorded in 2018. At the time, Apple's system was reported, on a disengagements-by-mile basis, as the worst-performing test platform in California. By contrast, then-front runner Waymo managed 0.09 disengagements per 1,000 miles over the same evaluation period.
While the improvement seems impressive, it should be noted that disengagement reporting is not a hard science. Typically, disengagements are defined as instances in which a self-driving vehicle's autonomous systems are unable to process current conditions, forcing it to pass control back to the human driver. Human interventions also count as disengagements.
As explained by Apple in a letter to California's DMV last year, the company in 2018 took a "conservative" approach to disengagements that required "drivers to proactively take manual control of the vehicle any time the system encounters a scenario beyond our currently proven abilities." Further, the company said its "software self-monitors and returns control back to the driver when it encounters any errors or issues during operation."
The letter implies Apple's built-in threshold for failure is, or at least was, much less tolerant than competing systems, a thesis was borne out in total disengagement figures recorded during the period. From December 2017 through November 2018, the 47 other firms testing automated vehicles on California roads clocked a cumulative 3,040 disengagements over nearly 2 million logged miles.
That said, Apple appears to have narrowed its reporting methodology in 2019.
Of the 64 disengagements, 52 were initiated by the autonomous vehicle system, while human pilots overrode AV decisions 12 times. The most common reason for disengagement was a vague "controls discrepancy" issue.
According to numbers compiled by CNET, Baidu led the pack in 2019 with 0.06 disengagements per 1,000 miles, followed by Waymo and Cruise, both of which managed 0.08 disengagements per 1,000 miles. Toyota reported the highest rate of disengagements with 1,620 per 1,000 miles, though the Japanese carmaker fielded six cars that traveled 1,817 miles.
Apple's self-driving car initiative began under the "Project Titan" banner, with the program initially tasked with creating a branded car from whole cloth. Apple scaled back operations in late 2016 after hitting a number of snags and the Titan team has since refocused attention to autonomous vehicle subsystems. Whether an "Apple Car" will one day make a public debut is unknown.
Apple's self-driving car testbed in California.
According to numbers submitted by Apple to California's Department of Motor Vehicles, 23 of the company's 69 registered self-driving testbeds were active between December 2018 and November 2019. Collectively, the fleet of modified Lexus RX450h SUVs logged 7,544 miles during the reporting period, down from 79,754 miles in 2018.
Apple reported 64 disengagements across the 7,544 miles driven, which equates to 8.48 disengagements per 1,000 miles. This compares to a whopping 69,510 disengagements, or 871.65 disengagements per 1,000 miles, recorded in 2018. At the time, Apple's system was reported, on a disengagements-by-mile basis, as the worst-performing test platform in California. By contrast, then-front runner Waymo managed 0.09 disengagements per 1,000 miles over the same evaluation period.
While the improvement seems impressive, it should be noted that disengagement reporting is not a hard science. Typically, disengagements are defined as instances in which a self-driving vehicle's autonomous systems are unable to process current conditions, forcing it to pass control back to the human driver. Human interventions also count as disengagements.
As explained by Apple in a letter to California's DMV last year, the company in 2018 took a "conservative" approach to disengagements that required "drivers to proactively take manual control of the vehicle any time the system encounters a scenario beyond our currently proven abilities." Further, the company said its "software self-monitors and returns control back to the driver when it encounters any errors or issues during operation."
The letter implies Apple's built-in threshold for failure is, or at least was, much less tolerant than competing systems, a thesis was borne out in total disengagement figures recorded during the period. From December 2017 through November 2018, the 47 other firms testing automated vehicles on California roads clocked a cumulative 3,040 disengagements over nearly 2 million logged miles.
That said, Apple appears to have narrowed its reporting methodology in 2019.
Of the 64 disengagements, 52 were initiated by the autonomous vehicle system, while human pilots overrode AV decisions 12 times. The most common reason for disengagement was a vague "controls discrepancy" issue.
According to numbers compiled by CNET, Baidu led the pack in 2019 with 0.06 disengagements per 1,000 miles, followed by Waymo and Cruise, both of which managed 0.08 disengagements per 1,000 miles. Toyota reported the highest rate of disengagements with 1,620 per 1,000 miles, though the Japanese carmaker fielded six cars that traveled 1,817 miles.
Apple's self-driving car initiative began under the "Project Titan" banner, with the program initially tasked with creating a branded car from whole cloth. Apple scaled back operations in late 2016 after hitting a number of snags and the Titan team has since refocused attention to autonomous vehicle subsystems. Whether an "Apple Car" will one day make a public debut is unknown.
Comments
It's also difficult to read into reduced road time, it could mean anything, such as time spent in R&D, time spent testing new software on private tracks before moving into public spaces.
Since Apple now have roads in continental USA 3D mapped as part of their Maps overhaul, it seems probable that this data can also be used in some virtual testing scenarios for the car project.
The evidence is there but the caveats and changes since then are also explained in the article.
Until two different cars are put to exactly the same road conditions at the same time, it will be difficult to truly compare how autonomous they really are and how they perform in other areas.
The main thing to take into consideration is that it seems each company is allowed to determine what counts as a "disengagement". So trying to compare one company's numbers with another's, is pointless. And as someone else pointed out, no one knows the differences in driving conditions during those logged miles.
Having such a sharp drop in miles driven makes me think that larger number was just random driving to collect as much data as they could. And then last year they drove and tested (gathered data) for very specific situations.
Wasn't Apple gonna go out of business when Jobs died? Why is Apple bigger now and why did some of the most successful products in Apple history release post-Jobs?
Are we gonna hang onto the "riding on Jobs momentum" theory forever? It's gonna be 10 years for crying out loud!
Please correct this as it is unsubstantiated rumor and speculation. Apple has never, nor has anyone confirmed this guess/speculation about what Apple is up to with Project Titan. Indeed, this particular guess is generally thought of as one the least likely avenues Apple would have any reason to go down regarding Project Titan.
Your claim than one man was responsible for success is at odds with reality. It’s also at odds with what that man said himself — Jobs had said at events before that credit for their success is not his alone, that it takes an entire team, teams of people, to realize these goals. By your logic Apple was DOOMed after Jobs’ death, but we have seen conclusively that this was not the case.
Moreover, Apple did zero autonomous vehicle testing between December 2018 and May 2019. None of their cars logged a single mile during this six-month timeframe which points to a possible suspension of the entire program.
And when they did resume in June 2019, only a fraction of their registered vehicles resumed very light testing.
I concur with the last lines of the article that Apple has focused on subsystems and is interested mostly in improving safety and accuracy in a system users will be comfortable with because its based on something they already know -- how iOS devices work -- that can be licensed to manufacturers as the hugely successful CarPlay is today.