Apple's safe automotive telematics patent mates touchscreen with tactile controls
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent for a touchscreen-based telematics system that incorporates embedded controls, such as knobs, sliders, and switches, with a tactile display to give driver's feedback so they can keep their eyes on the road.
Source: USPTO
Apple secured U.S. Patent No. 8,482,535 for "Programmable tactile touch screen displays and man-machine interfaces for improved vehicle instrumentation and telematics," which lays out the groundwork for a touchscreen display that allows users to easily and safely operate certain vehicle functions. By employing a screen "feel," or tactile feedback, drivers can more effectively change car settings like temperature control without having to look away from the road.
The patent is a continuation of a property filed for in 2011 and subsequently granted in 2006, which itself is a continuation and extension of a very basic multitouch invention from 1992 credited to Canadian inventor Timothy R. Pryor. While the over 20-year-old filing was later abandoned, the original invention is patented under the title "Method for providing human input to a computer," a property assigned to Apple in 2012.
In Tuesday's patent, the multitouch system described by Pryor is adapted to automotive uses, or more specifically telematics and communication via a cellular phone.
Illustration of screen.
The invention deals mainly with the center stack, or area of a car's dashboard where navigation, entertainment, and environment controls are commonly placed. Because the system is programmable, it can not only replace current instrumentation, but also take over other car functions like window wiper control.
Tactile center stack with image projector (110) and camera sensor (117).
Numerous embodiments are covered in the patent literature, including a heads-up display (HUD) embedded in the armrest, screens that respond to laser pointers as modes of input, and even cameras that detect a driver's head position. Most of the examples, however, revolve around a touchscreen onto which images are projected, thus making the system highly programmable and flexible.
The screen itself can have raised ridges, indents or other tactile properties to facilitate operation without diverting attention away from the road. Notches, or grooves, and raised ridges are perhaps the most feasible in a real-world system. Some car makers already offer such solutions in their branded infotainment centers.
Tactile overlays.
Also mentioned are interchangeable transparent members, like knobs and sliders, that can be placed into a screen overlay for tactile feedback. Lights behind the screen illuminate the controls when needed, and go dark when not in use.
In many cases, an electro-optical sensing system is used to manage input. As the patent is quite old, it notes a projector-type setup can be used to display graphical assets next to or on top of the interchangeable sliders and knobs. Examples include "High" or "Low" text, or red and blue color changes for climate control settings.
Illustration of slider controls.
Because display technologies have advanced substantially, the proposed system would no longer require a projection interface, and would likely use an LCD or other modern display tech with capacitive properties.
In some embodiments, the system can be paired with voice and gesture control for enhanced capabilities. Apple is already deploying such a method in Siri Eyes Free, and looks to take that further in iOS 7 with "iOS in the Car."
Interestingly, the patent notes that pairing with cellular communications can feed programs, or what we call "apps" today, such as stock tickers or other information sourced from the Internet.
Most recently, hidden code in Apple's iOS 7 beta revealed iOS in the Car will likely interface via Wi-Fi communications using the AirPlay protocol.
Apple's telematics patent was first filed for in 2009.
Source: USPTO
Apple secured U.S. Patent No. 8,482,535 for "Programmable tactile touch screen displays and man-machine interfaces for improved vehicle instrumentation and telematics," which lays out the groundwork for a touchscreen display that allows users to easily and safely operate certain vehicle functions. By employing a screen "feel," or tactile feedback, drivers can more effectively change car settings like temperature control without having to look away from the road.
The patent is a continuation of a property filed for in 2011 and subsequently granted in 2006, which itself is a continuation and extension of a very basic multitouch invention from 1992 credited to Canadian inventor Timothy R. Pryor. While the over 20-year-old filing was later abandoned, the original invention is patented under the title "Method for providing human input to a computer," a property assigned to Apple in 2012.
In Tuesday's patent, the multitouch system described by Pryor is adapted to automotive uses, or more specifically telematics and communication via a cellular phone.
Illustration of screen.
The invention deals mainly with the center stack, or area of a car's dashboard where navigation, entertainment, and environment controls are commonly placed. Because the system is programmable, it can not only replace current instrumentation, but also take over other car functions like window wiper control.
Tactile center stack with image projector (110) and camera sensor (117).
Numerous embodiments are covered in the patent literature, including a heads-up display (HUD) embedded in the armrest, screens that respond to laser pointers as modes of input, and even cameras that detect a driver's head position. Most of the examples, however, revolve around a touchscreen onto which images are projected, thus making the system highly programmable and flexible.
The screen itself can have raised ridges, indents or other tactile properties to facilitate operation without diverting attention away from the road. Notches, or grooves, and raised ridges are perhaps the most feasible in a real-world system. Some car makers already offer such solutions in their branded infotainment centers.
Tactile overlays.
Also mentioned are interchangeable transparent members, like knobs and sliders, that can be placed into a screen overlay for tactile feedback. Lights behind the screen illuminate the controls when needed, and go dark when not in use.
In many cases, an electro-optical sensing system is used to manage input. As the patent is quite old, it notes a projector-type setup can be used to display graphical assets next to or on top of the interchangeable sliders and knobs. Examples include "High" or "Low" text, or red and blue color changes for climate control settings.
Illustration of slider controls.
Because display technologies have advanced substantially, the proposed system would no longer require a projection interface, and would likely use an LCD or other modern display tech with capacitive properties.
In some embodiments, the system can be paired with voice and gesture control for enhanced capabilities. Apple is already deploying such a method in Siri Eyes Free, and looks to take that further in iOS 7 with "iOS in the Car."
Interestingly, the patent notes that pairing with cellular communications can feed programs, or what we call "apps" today, such as stock tickers or other information sourced from the Internet.
Most recently, hidden code in Apple's iOS 7 beta revealed iOS in the Car will likely interface via Wi-Fi communications using the AirPlay protocol.
Apple's telematics patent was first filed for in 2009.
Comments
As with hands-free cell phones, the problem is the distraction of the device itself, not the manipulation needed to work it
It almost seems that at one point over the past 21 years Apple had considered a Microsoft SYNC-like vehicle integration.
I would assume they no longer are pursuing that, but if that were the case, what is the point of this patent?
as for this very un-usual patent, I am confused by the "display (HUD) embedded in the armrest" - how can a "heads up" display be in an armrest???
Actually, the whole "heads up" display annoys me because I once had a car built in 1955 that had that feature, it is nothing new at all!
What's funny, in light of Siri, is how this older application poo-poo's voice control...
SUMMARY OF THE CHALLENGE AND OPPORTUNITY
The automotive dashboard is today a confused array of switches, knobs, dials, gages, and other tactile physical selection or adjustment means and instruments. It is often hard to see and to understand, and can cause undo distraction to the driver. As a device, it is filled with different parts and thus expensive to manufacture both in serial quantity, and in redesign and tooling for new models. Furthermore, it is inflexible, and invariant once manufactured. It cannot be changed in its design by the user, and cannot be easily changed by the manufacturer or the dealer.
In addition, the instrument panel in its current form it is at its limit in so far as its ability to present data which can be acted on. There isn't any more room on the dash to put more devices, and various safety issues such as airbag deployment and passive interior safety preclude many choices. Data which might be desired for action can be from the vehicles own controls and state, its surroundings such as from other vehicles or parked objects, or could include material downloaded from the internet, or navigational data from satellites.
Because of this, Voice recognition techniques are now being researched in earnest in order to allow the user to interact with computer based functions. IBM VIA VOICE and DRAGON SOFTWARE Naturally Speaking products have already reached the general office market, and are reasonably accurate and effective if the surrounding environment is quiet and stable. But in the car, this is not the case, and even specialized limited capability Telematic systems such as Fonix Corporations Automatic Speech Recognition (ASR), L and H, etc, have lots of problems--for example:
Noise--the car is not an ideal environment under any circumstance, never mind with CD Player at full volume;
Different drivers, with different accents, practices;
Limited or no teach in available to perfect the function of the recognition program; and
Passenger disturbance, and conversation interruption (i.e. a background noise source which disturbs the recognition, or the actual intonation of the speech.
These problems result in function which is possibly error prone or time delayed. Having to repeat or worse, having the wrong action, is not desirable in mission critical driving situations so to speak. The Cell phone problem is notorious. In the extreme, a form of rage can take over if the system frustrates the driver. For this reason, voice recognition systems used for actual vehicle functions (as opposed to dialing phones and the like) have in effect, backup controls--defeating the concept of increasing dashboard space, and control comprehension.
In addition there is a sociological problem--talking to ones car in unacceptable to many, even if the voice recognition program works. For some this is always true, for others its true with passengers present.
In addition, the presentation of data by the computer to the driver by text to speech programs is known to have unacceptable intonation or time latency in many circumstances by the average user, and in any case is woefully inadequate for graphical or tabular information, for example. The data to be presented to the user, even in a simple list of choices, has to be vocally spoken by the computer to the driver in sequence, taking a great deal of concentration to avoid missing an important point, and taking minutes to do what can visually be done in a glance.
I would love for my car to have most of the features of IOS completely integrated. This is especially attractive when you consider that most automotive "technology" packages are $1,500-$2,000. Even an iPad mounted to the center console could do so much more.
Did you watch the Keynote?
I also saw my first Siri car commercial the other day.
.
Quote:
Originally Posted by mdcat
As with hands-free cell phones, the problem is the distraction of the device itself, not the manipulation needed to work it
Exactly. My favorite quote on the matter: "Congratulations, your hands are free. Your head's still up your ass."
Oh, you mean exactly where you arm is when driving? I can imagine it.
The steering wheel isn't going away, and putting controls in the middle is the same as thinking a vertical touchscreen on a computer is a good idea.
Ah, this must be a usage of the word "subsequently" with which I was previously unfamiliar.
Forseeable future, and, interestingly enough, I don't believe it's the "self-driving car" concept that will get rid of it! Rather, that it shouldn't be what does, but the car companies are absolute whores about not innovating in any regard whatsoever, so what [I]should[/I] happen probably won't be what does.
A much more natural control mechanism would be a joystick. Stick it down on the center console, you know? Where the gearshift occasionally is. Then turning is just moving your wrist (either side to side or twisting it; whichever).
Then all you'd need up front is... nothing. Put the airbag there, alone. Project [I]directly onto the windshield[/I] the information you need while driving (GPS data, current speed, miles to empty... that's really it; you don't NEED to see anything else that dashboards usually have unless attention is required. Why should I care what my temperature and oil are doing unless they're doing something WRONG?). If something requires my attention, it should then appear on the windshield.
Oh, and Tesla has the right idea about side mirrors.
[B]EDIT:[/B] Had a thought: perhaps it [I]will[/I] be self-driving cars that get rid of the steering wheel... but only because it becomes unwieldy as it's something you "don't use very often". Have self-driving cars be the impetus to switch to a joystick or similar less-obtrusive control.
How about just real knobs, sliders and switches?
What is great about them is that they can be different shapes, sizes, and textures.
You can learn which is which by feel.
With any flat screen, even with tactile controls it is still one dimensional. You can actually hold it.
Even Ford is going back to offering some knobs and buttons because drivers (you know, the people behind the wheel) found that touch screen everything just doesn't work so well.
Amusing that the 'STOCKS' ticker is showing MSFT and IBM rather than AAPL...