Flaw in distracted driving study: no evaluation of Apple's "Eyes Free" Siri mode
A widely publicized study that reported that voice assistants like Apple's Siri aren't any safer than manually texting while driving failed to test Siri as it was actually designed to be used in its distracted driving experiment.
Source: Apple
The study, conducted by Texas A&M Transportation Institute, noted that drivers on a closed course holding a smartphone in one hand while interacting with Siri (or Vlingo, a similar service for Android smartphones) took nearly twice as long to respond to an external testing event.
Specifically, the study said researchers installed a green LED light on the subjects' vehicle dashboard, "and drivers were instructed to press a button every time they saw it come on. Their response times served as a measure of their distraction levels."
The study's author Christine Yager subsequently concluded that voice-to-text services "do not increase driver safety compared to manual texting."
In response to the experiment, a report by Xconomy noted that the study was flawed because the way Siri was tested is not how it is intended to be used.
Cheyer added "Of course your driving performance is going to be degraded if you?re reading screens and pushing buttons," and explained that Siri's designers took special efforts to create a driving-appropriate mode that was not only handsfree, but also "Eyes Free" as Apple has branded it.
When Siri is used with a Bluetooth headset or other speakers as a handsfree phone, it automatically enters into a safer mode that limits interactions to be voice only. "It assumes you are ?eyes-busy? and responds differently," Cheyer said.
"My goal is not to knock this particular study," the report cited Cheyer as saying. "I?m just dismayed that the message being communicated by news media -- that 'Siri is just as risky as texting' -- is misleading."
Responding to the criticism of her study, Yager said "we tested the applications in a way that is consistent with how many drivers typically use them," and added, "we examined the product information contained in the packaging for the iPhone 4S, and were not able to find information related to the directed mode use of the device."
Media coverage of the distracted driving study didn't distinguish between the test's evaluation of manually using Siri as voice-driven way to text and drive, and using Siri as it is intended to be used, both handsfree and "Eyes Free" while driving.
Advancing alongside a federal push to call attention to the problem of distracted driving, Apple's Eyes Free design for Siri is intended not only to make accessing directions or other information safer while driving, but also to facilitate easy integration for car manufacturers.
Since Apple announced the push, a series of car manufacturers have signed on to enable one button access to Siri in a way that makes the voice assistant available without distracting the driver with a visual, hands-occupying interface.
Siri's Eyes Free is the culmination of a decade of Apple's car integration efforts that began with the iPod. Last year, Apple enhanced Siri with even tighter integration to Maps, putting Apple into more direct competition with Google in both mapping and voice based search.
Google efforts to leverage its early lead in maps and voice-based services to duplicate Apple's Siri features are complicated by the fact that the company currently relies on popup advertising banners to support its technology. A Siri-like service will be hard to monetize with ads when users want distraction free, audible information.
Source: Apple
The study, conducted by Texas A&M Transportation Institute, noted that drivers on a closed course holding a smartphone in one hand while interacting with Siri (or Vlingo, a similar service for Android smartphones) took nearly twice as long to respond to an external testing event.
Specifically, the study said researchers installed a green LED light on the subjects' vehicle dashboard, "and drivers were instructed to press a button every time they saw it come on. Their response times served as a measure of their distraction levels."
The study's author Christine Yager subsequently concluded that voice-to-text services "do not increase driver safety compared to manual texting."
In response to the experiment, a report by Xconomy noted that the study was flawed because the way Siri was tested is not how it is intended to be used.
Holding Siri is not handsfree
"I don?t think that there is any evidence that shows that if Siri and other systems are used properly in Eyes Free mode, they are ?just as risky as texting," stated Adam Cheyer, a cofounder of the project Apple acquired in 2010 to release Siri as a feature of the iPhone 4S. Cheyer left Apple last year.Cheyer added "Of course your driving performance is going to be degraded if you?re reading screens and pushing buttons," and explained that Siri's designers took special efforts to create a driving-appropriate mode that was not only handsfree, but also "Eyes Free" as Apple has branded it.
When Siri is used with a Bluetooth headset or other speakers as a handsfree phone, it automatically enters into a safer mode that limits interactions to be voice only. "It assumes you are ?eyes-busy? and responds differently," Cheyer said.
"My goal is not to knock this particular study," the report cited Cheyer as saying. "I?m just dismayed that the message being communicated by news media -- that 'Siri is just as risky as texting' -- is misleading."
Responding to the criticism of her study, Yager said "we tested the applications in a way that is consistent with how many drivers typically use them," and added, "we examined the product information contained in the packaging for the iPhone 4S, and were not able to find information related to the directed mode use of the device."
Media coverage of the distracted driving study didn't distinguish between the test's evaluation of manually using Siri as voice-driven way to text and drive, and using Siri as it is intended to be used, both handsfree and "Eyes Free" while driving.
Eyes Free key to Apple's automotive strategy for iOS
Six months ago, AppleInsider noted how automotive integration has become a driving feature of Apple's mobile iOS platform.Advancing alongside a federal push to call attention to the problem of distracted driving, Apple's Eyes Free design for Siri is intended not only to make accessing directions or other information safer while driving, but also to facilitate easy integration for car manufacturers.
Since Apple announced the push, a series of car manufacturers have signed on to enable one button access to Siri in a way that makes the voice assistant available without distracting the driver with a visual, hands-occupying interface.
Siri's Eyes Free is the culmination of a decade of Apple's car integration efforts that began with the iPod. Last year, Apple enhanced Siri with even tighter integration to Maps, putting Apple into more direct competition with Google in both mapping and voice based search.
Google efforts to leverage its early lead in maps and voice-based services to duplicate Apple's Siri features are complicated by the fact that the company currently relies on popup advertising banners to support its technology. A Siri-like service will be hard to monetize with ads when users want distraction free, audible information.
Comments
Under many jurisdictions, you can't even hold a phone while driving.
This study is flawed in that it's limited to the handheld scenario.
I smell a "study" that reached its conclusion first and then went in search of ways of making the data match, instead of being empirical.
I think his point is valid. If you only testing Siri by having the driver hold the phone with one hand whilst driving instead of having their hands on the steering wheel. There is also a natural proclivity to look at the display of the device you're holding, especially if you know it's displaying results on screen. Now I don't know if the local Siri software is intelligent enough to know that if you ask, "how many ounces in a liter?" to not display the Wolfram Alpha results on the iDevice screen but to speak a simplified result over the speaker system, but that is beside the point in regards to these tests.
Now I agree that any distractions can be a danger. After all, by definition a distraction is "a thing that prevents someone from giving full attention to something else" but I don't agree with any assertion that Siri integrated into a car and Siri via an iDevice directly in one's hand should be seen the same way. I'd say changing the radio or adjusting the climate control from the dash controls are more of a distraction than pressing the Siri button from the steering wheel, yet we still give drivers legal access to those functions in automobiles.
That assumes people aren't looking at their iPhone's display. Since it's the most common way to engage an iPhone it probably takes some additional cognitive effort to actively not look at the iPhone in your hand that you're actively using.
Quote:
Originally Posted by ktappe
I don't believe the study for a different reason: If you are not looking at the phone, what difference could it make whether the phone is in your hand or on the dashboard? And before you answer "Then your hands are not at 10 and 2", the old "10 and 2" hand position style of driving was recently discredited too.
I smell a "study" that reached its conclusion first and then went in search of ways of making the data match, instead of being empirical.
Which hand do you use to press the button on the dash?
The one on the steering wheel or the one holding the device?
It's like that in LA but it stops no one. The traffic cams are all turned off it seems cause no one is ever actually caught running red lights, eating etc with a hand off the wheel. Perhaps if those cams were turned back on and folks are ticketed crap would slow down
It's like that in LA but it stops no one. The traffic cams are all turned off it seems cause no one is ever actually caught running red lights, eating etc with a hand off the wheel. Perhaps if those cams were turned back on and folks are ticketed crap would slow down
Quote:
Originally Posted by ktappe
I don't believe the study for a different reason: If you are not looking at the phone, what difference could it make whether the phone is in your hand or on the dashboard? And before you answer "Then your hands are not at 10 and 2", the old "10 and 2" hand position style of driving was recently discredited too.
I smell a "study" that reached its conclusion first and then went in search of ways of making the data match, instead of being empirical.
I believe there are flaws with the study, but in answer to your question; it has nothing to do with whether your eyes are looking at what's in your hand or not.
I can drive with an ice cream cone in one hand without ever looking at it. And it's absolutely a distraction.
Quote:
Originally Posted by SolipsismX
I think his point is valid. If you only testing Siri by having the driver hold the phone with one hand whilst driving instead of having their hands on the steering wheel. There is also a natural proclivity to look at the display of the device you're holding, especially if you know it's displaying results on screen. Now I don't know if the local Siri software is intelligent enough to know that if you ask, "how many ounces in a liter?" to not display the Wolfram Alpha results on the iDevice screen but to speak a simplified result over the speaker system, but that is beside the point in regards to these tests.
Now I agree that any distractions can be a danger. After all, by definition a distraction is "a thing that prevents someone from giving full attention to something else" but I don't agree with any assertion that Siri integrated into a car and Siri via an iDevice directly in one's hand should be seen the same way. I'd say changing the radio or adjusting the climate control from the dash controls are more of a distraction than pressing the Siri button from the steering wheel, yet we still give drivers legal access to those functions in automobiles.
I concede, it's debatable how much distraction a driver can safely handle, but fewer distractions and especially fewer distractions that require the driver's attention away from the road would be better. I look forward to the day Google's driving tech is widely adopted and faces a multitude of competitors to speed refinement via free market competition.
Quote:
Originally Posted by SolipsismX
I think his point is valid. If you only testing Siri by having the driver hold the phone with one hand whilst driving instead of having their hands on the steering wheel.
Not only were they holding the device the were instructed to press another button (On the dash? It didn't say where the green light button was). I wonder which hand they used for that activity.
I wonder if laws will eventually change to allow open container in driver-less vehicles.
I didn't even pick up on that until your post despite having read hill60's comment before yours.
Quote:
Originally Posted by SpamSandwich
I concede, it's debatable how much distraction a driver can safely handle, but fewer distractions and especially fewer distractions that require the driver's attention away from the road would be better.
Even when someone is holding the device to their ear while driving it is often easy to tell they are distracted. You see them tailgating or driving slowly or erratically. I can spot them a mile away and sometimes and say to myself, 'there is another person talking on the phone while driving'... Hands free is probably a bit better but a distraction with the conversation itself is often enough to compromise safety. Somehow having a conversation with someone who is actually in the car is not as distracting.
Quote:
Originally Posted by SpamSandwich
Any distractions while driving are a danger. Dilger, you're stretching more than usual.
It's entirely valid to say "holding your iPhone in your hand to use Siri to do tasks while driving is not significantly safer than texting while driving."
It's pretty bad science to set up an "experiment" where the desired conclusion is just being reaffirmed in a pointless "test" in order to announce that "Siri is as dangerous as texting while driving!!!"
The problem is not glancing at a phone on occasion. Drivers routinely glance at the dashboard indicators (like the speedometer!), radio and climate controls without causing a big uptick in accidents. The problem is focusing on a hand held device, particularly one you have to navigate with the full attention of one hand and most of your concentration.
Holding a phone in one hand and blindly dictating a question into it (such as "give me directions to Pizza Hut") is clearly vastly safer than trying to manually type in the same thing while looking at the screen. But its even safer to keep your hands on the wheel, push a button, and speak with your eyes on the road, and then follow voice navigation directions entirely hands free and eyes free.
This was pretty clearly worthless research intended to create a stupid headline that muddies, rather than clarifying, reality.
I wish cars had a feature that detected what the current speed limit was and either played a tone or turned on a light on the speedometer or something to notify the driver. Some people speed accidentally and this would be helpful. Of course those who choose to speed would probably find it annoying, but nevertheless it would be very simple technologically either by using RFID or just GPS and a smart map that knew the speed limit of all streets. For example the Nissan R already has a similar feature where the automobile will not go faster than 180 MPH? unless it is on an approved Nissan racetrack of which there are only a few and it knows the GPS coordinates of those tracks.
i think all Garmin and Tom Tom GPS have that feature built-in.
Quote:
Originally Posted by SolipsismX
I wonder if laws will eventually change to allow open container in driver-less vehicles.
I believe that eventually could be possible, though the current laws say a driver must be in the vehicle and be able to take over the controls if necessary.
Many years ago, just playing with the radio while driving was considered distracting and dangerous.
Anyone that thinks that they can text, talk on a phone handless or not is deluding themselves if they think some of their reflexes are not affected.
Cars a dangerous and using one merits ones undivided attention for the safety of others.
So how can we take your study seriously, when you are not even smart enough to conduct the study properly?