Apple's safe automotive telematics patent mates touchscreen with tactile controls

Posted:
in General Discussion edited January 2014
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent for a touchscreen-based telematics system that incorporates embedded controls, such as knobs, sliders, and switches, with a tactile display to give driver's feedback so they can keep their eyes on the road.

Telematics
Source: USPTO


Apple secured U.S. Patent No. 8,482,535 for "Programmable tactile touch screen displays and man-machine interfaces for improved vehicle instrumentation and telematics," which lays out the groundwork for a touchscreen display that allows users to easily and safely operate certain vehicle functions. By employing a screen "feel," or tactile feedback, drivers can more effectively change car settings like temperature control without having to look away from the road.

The patent is a continuation of a property filed for in 2011 and subsequently granted in 2006, which itself is a continuation and extension of a very basic multitouch invention from 1992 credited to Canadian inventor Timothy R. Pryor. While the over 20-year-old filing was later abandoned, the original invention is patented under the title "Method for providing human input to a computer," a property assigned to Apple in 2012.

In Tuesday's patent, the multitouch system described by Pryor is adapted to automotive uses, or more specifically telematics and communication via a cellular phone.

Telematics
Illustration of screen.


The invention deals mainly with the center stack, or area of a car's dashboard where navigation, entertainment, and environment controls are commonly placed. Because the system is programmable, it can not only replace current instrumentation, but also take over other car functions like window wiper control.

Telematics
Tactile center stack with image projector (110) and camera sensor (117).


Numerous embodiments are covered in the patent literature, including a heads-up display (HUD) embedded in the armrest, screens that respond to laser pointers as modes of input, and even cameras that detect a driver's head position. Most of the examples, however, revolve around a touchscreen onto which images are projected, thus making the system highly programmable and flexible.

The screen itself can have raised ridges, indents or other tactile properties to facilitate operation without diverting attention away from the road. Notches, or grooves, and raised ridges are perhaps the most feasible in a real-world system. Some car makers already offer such solutions in their branded infotainment centers.

Telematics
Tactile overlays.


Also mentioned are interchangeable transparent members, like knobs and sliders, that can be placed into a screen overlay for tactile feedback. Lights behind the screen illuminate the controls when needed, and go dark when not in use.

In many cases, an electro-optical sensing system is used to manage input. As the patent is quite old, it notes a projector-type setup can be used to display graphical assets next to or on top of the interchangeable sliders and knobs. Examples include "High" or "Low" text, or red and blue color changes for climate control settings.

Telematics
Illustration of slider controls.


Because display technologies have advanced substantially, the proposed system would no longer require a projection interface, and would likely use an LCD or other modern display tech with capacitive properties.

In some embodiments, the system can be paired with voice and gesture control for enhanced capabilities. Apple is already deploying such a method in Siri Eyes Free, and looks to take that further in iOS 7 with "iOS in the Car."

Interestingly, the patent notes that pairing with cellular communications can feed programs, or what we call "apps" today, such as stock tickers or other information sourced from the Internet.

Telematics


Most recently, hidden code in Apple's iOS 7 beta revealed iOS in the Car will likely interface via Wi-Fi communications using the AirPlay protocol.

Apple's telematics patent was first filed for in 2009.
«1

Comments

  • Reply 1 of 21
    mdcatmdcat Posts: 79member


    As with hands-free cell phones, the problem is the distraction of the device itself, not the manipulation needed to work it

  • Reply 2 of 21
    blackbookblackbook Posts: 1,361member
    Very strange patent.

    It almost seems that at one point over the past 21 years Apple had considered a Microsoft SYNC-like vehicle integration.

    I would assume they no longer are pursuing that, but if that were the case, what is the point of this patent?
  • Reply 3 of 21
    digital_guydigital_guy Posts: 162member
    I don't know why Apple isn't thinking of automotive! This seems like an area ripe for innovation regarding displays and control systems. The next area of growth isn't a clothing accessory or article. It should be automotive display and control systems. The hardware and software available in most automobiles today it seems is an afterthought of the manufacturer. Let's hope that iOS in the car does come to fruition. This is an area that I think Apple needs to pursue aggressively. You heard it here, first; And, I hope somehow that someone from Apple hears this message. iOS for automotive display and control systems, Apple!
  • Reply 4 of 21
    hey digital_guy, there are already vehicles fitted with siri integration and other iOS connectivity, so it's happening (slowly).
    as for this very un-usual patent, I am confused by the "display (HUD) embedded in the armrest" - how can a "heads up" display be in an armrest???

    Actually, the whole "heads up" display annoys me because I once had a car built in 1955 that had that feature, it is nothing new at all!
  • Reply 5 of 21
    All this touch screen junk is just that in a car- junk. It all requires taking your eyes off the road and even more concentration on adjusting the on-screen control. Nothing works easier for humans than actual, physical tactile knobs, buttons, and sliders. I wish people would stop thinking that some touch screen can replace *everything*. Even iPhones have 5 distinct buttons that can't really be easily turned into virtual buttons and still work easily. For the "basics" I believe there must be a minimum number of buttons or controls, for advanced things like navigation/maps/"infotainment", those can have touch control but it should be strictly locked out while the vehicle is in motion.
  • Reply 6 of 21
    thrangthrang Posts: 1,007member


    What's funny, in light of Siri, is how this older application poo-poo's voice control...


     


    SUMMARY OF THE CHALLENGE AND OPPORTUNITY 



    The automotive dashboard is today a confused array of switches, knobs, dials, gages, and other tactile physical selection or adjustment means and instruments. It is often hard to see and to understand, and can cause undo distraction to the driver. As a device, it is filled with different parts and thus expensive to manufacture both in serial quantity, and in redesign and tooling for new models. Furthermore, it is inflexible, and invariant once manufactured. It cannot be changed in its design by the user, and cannot be easily changed by the manufacturer or the dealer. 



    In addition, the instrument panel in its current form it is at its limit in so far as its ability to present data which can be acted on. There isn't any more room on the dash to put more devices, and various safety issues such as airbag deployment and passive interior safety preclude many choices. Data which might be desired for action can be from the vehicles own controls and state, its surroundings such as from other vehicles or parked objects, or could include material downloaded from the internet, or navigational data from satellites. 



    Because of this, Voice recognition techniques are now being researched in earnest in order to allow the user to interact with computer based functions. IBM VIA VOICE and DRAGON SOFTWARE Naturally Speaking products have already reached the general office market, and are reasonably accurate and effective if the surrounding environment is quiet and stable. But in the car, this is not the case, and even specialized limited capability Telematic systems such as Fonix Corporations Automatic Speech Recognition (ASR), L and H, etc, have lots of problems--for example: 



    Noise--the car is not an ideal environment under any circumstance, never mind with CD Player at full volume; 



    Different drivers, with different accents, practices; 



    Limited or no teach in available to perfect the function of the recognition program; and 



    Passenger disturbance, and conversation interruption (i.e. a background noise source which disturbs the recognition, or the actual intonation of the speech. 



    These problems result in function which is possibly error prone or time delayed. Having to repeat or worse, having the wrong action, is not desirable in mission critical driving situations so to speak. The Cell phone problem is notorious. In the extreme, a form of rage can take over if the system frustrates the driver. For this reason, voice recognition systems used for actual vehicle functions (as opposed to dialing phones and the like) have in effect, backup controls--defeating the concept of increasing dashboard space, and control comprehension. 



    In addition there is a sociological problem--talking to ones car in unacceptable to many, even if the voice recognition program works. For some this is always true, for others its true with passengers present. 



    In addition, the presentation of data by the computer to the driver by text to speech programs is known to have unacceptable intonation or time latency in many circumstances by the average user, and in any case is woefully inadequate for graphical or tabular information, for example. The data to be presented to the user, even in a simple list of choices, has to be vocally spoken by the computer to the driver in sequence, taking a great deal of concentration to avoid missing an important point, and taking minutes to do what can visually be done in a glance. 

     

  • Reply 7 of 21
    maccentricmaccentric Posts: 263member
    I hope to buy my next car with iOS in it in a few years. My car is a 2009, but similar to what Tim Cook said about TV, using the electronic controls and GPS is like going back 10 years. They want $99 for a maps update. It would be so much better to have Apple Maps integrated.

    I would love for my car to have most of the features of IOS completely integrated. This is especially attractive when you consider that most automotive "technology" packages are $1,500-$2,000. Even an iPad mounted to the center console could do so much more.
  • Reply 8 of 21
    pendergastpendergast Posts: 1,358member
    I don't know why Apple isn't thinking of automotive! This seems like an area ripe for innovation regarding displays and control systems. The next area of growth isn't a clothing accessory or article. It should be automotive display and control systems. The hardware and software available in most automobiles today it seems is an afterthought of the manufacturer. Let's hope that iOS in the car does come to fruition. This is an area that I think Apple needs to pursue aggressively. You heard it here, first; And, I hope somehow that someone from Apple hears this message. iOS for automotive display and control systems, Apple!

    Did you watch the Keynote?

    I also saw my first Siri car commercial the other day.
  • Reply 9 of 21
    satoricalsatorical Posts: 60member


    .

  • Reply 10 of 21
    satoricalsatorical Posts: 60member

    Quote:

    Originally Posted by mdcat View Post


    As with hands-free cell phones, the problem is the distraction of the device itself, not the manipulation needed to work it



    Exactly. My favorite quote on the matter: "Congratulations, your hands are free. Your head's still up your ass."

  • Reply 11 of 21
    robin huberrobin huber Posts: 3,949member
    The steering wheel has been with us too long. Until it gets out of the way, a truly modern control screen cannot be placed where it should be. Imagine always using your iPad placed two feet to your right. Even worse for right hand drive countries unless you are left handed.
  • Reply 12 of 21
    tallest skiltallest skil Posts: 43,388member
    [quote name="Robin Huber" url="/t/158404/apples-safe-automotive-telematics-patent-mates-touchscreen-with-tactile-controls#post_2359326"]Imagine always using your iPad placed two feet to your right.[/QUOTE]

    Oh, you mean exactly where you arm is when driving? I can imagine it.

    The steering wheel isn't going away, and putting controls in the middle is the same as thinking a vertical touchscreen on a computer is a good idea.
  • Reply 13 of 21
    "The patent is a continuation of a property filed for in 2011 and subsequently granted in 2006, which itself is a continuation...".

    Ah, this must be a usage of the word "subsequently" with which I was previously unfamiliar.
  • Reply 14 of 21
    nagrommenagromme Posts: 2,834member
    I give AppleInsider full credit on the headline: this is just one of a zillion random patents, and the headline doesn't make it out to be any more than that. AI could have easily said "Apple Patent Points to Future Tactile Screens in Cars" or some such bait.
  • Reply 15 of 21
    robin huberrobin huber Posts: 3,949member
    Oh, you mean exactly where you arm is when driving? I can imagine it.

    The steering wheel isn't going away, and putting controls in the middle is the same as thinking a vertical touchscreen on a computer is a good idea.
    Isn't going away ever, or just not in the foreseeable future? It just seems odd to me that a control that does exactly one thing, control left/right movement of a vehicle, occupies both our hands, is the largest control of all, and takes up nearly all the most important surface directly in front of the operator. As other primary controls have moved to "fly-by-wire" it seems reasonable to me that this one may too. Do you really think this giant ring that needs to be manhandled from lock to lock is the ultimate expression of human interface for this function? If so, I think you lack imagination.
  • Reply 16 of 21
    tallest skiltallest skil Posts: 43,388member
    [quote name="Robin Huber" url="/t/158404/apples-safe-automotive-telematics-patent-mates-touchscreen-with-tactile-controls#post_2359433"]Isn't going away ever, or just not in the foreseeable future?[/QUOTE]

    Forseeable future, and, interestingly enough, I don't believe it's the "self-driving car" concept that will get rid of it! Rather, that it shouldn't be what does, but the car companies are absolute whores about not innovating in any regard whatsoever, so what [I]should[/I] happen probably won't be what does.

    A much more natural control mechanism would be a joystick. Stick it down on the center console, you know? Where the gearshift occasionally is. Then turning is just moving your wrist (either side to side or twisting it; whichever).

    Then all you'd need up front is... nothing. Put the airbag there, alone. Project [I]directly onto the windshield[/I] the information you need while driving (GPS data, current speed, miles to empty... that's really it; you don't NEED to see anything else that dashboards usually have unless attention is required. Why should I care what my temperature and oil are doing unless they're doing something WRONG?). If something requires my attention, it should then appear on the windshield.

    Oh, and Tesla has the right idea about side mirrors.

    [B]EDIT:[/B] Had a thought: perhaps it [I]will[/I] be self-driving cars that get rid of the steering wheel... but only because it becomes unwieldy as it's something you "don't use very often". Have self-driving cars be the impetus to switch to a joystick or similar less-obtrusive control.
  • Reply 17 of 21
    robin huberrobin huber Posts: 3,949member
    I guess we're in agreement. When you consider the pace at which the latest lane-keeping, automatic emergency braking, etc. have come on, plus the rapid rise of electric powered vehicles, maybe the foreseeable horizon line has gotten much closer. Cheers.
  • Reply 18 of 21
    mactacmactac Posts: 316member


    How about just real knobs, sliders and switches?


     


    What is great about them is that they can be different shapes, sizes, and textures.


    You can learn which is which by feel.


     


    With any flat screen, even with tactile controls it is still one dimensional. You can actually hold it.


     


    Even Ford is going back to offering some knobs and buttons because drivers (you know, the people behind the wheel) found that touch screen everything just doesn't work so well.

  • Reply 19 of 21
    robin huberrobin huber Posts: 3,949member
    I think tactile controls are best, too. All I want is a display screen that is big, clear, and consistent with my iOS user experience. Touch interface as backup or for passenger.
  • Reply 20 of 21
    hentaiboyhentaiboy Posts: 1,252member


    Amusing that the 'STOCKS' ticker is showing MSFT and IBM rather than AAPL...


     


    image

Sign In or Register to comment.