MacBook Pro may detect gestures in the air above the Touch Bar in the future

in General Discussion edited July 2020
Apple is looking to build on the Touch Bar by giving it the ability to detect finger movements and hand gestures in the air, rather than requiring users to touch and swipe on the Bar itself.

The Touch Bar could be augmented by the ability to detect finger movement above it.
The Touch Bar could be augmented by the ability to detect finger movement above it.

A new patent application shows that Apple is looking at ways of extending its Touch Bar concept. Users may be able to gesture above the bar with what Apple calls a "sensing strip" on MacBook Pro models.

"Electronic Device with Sensing Strip" describes a notebook that might give new meaning to the name MacBook Air.

"An electronic device may have an elongated sensing strip," says the patent. "Control circuitry may use the sensing strip to gather air gesture input from the fingers or other body part of a user."

"Non-contact gestures (sometimes referred to as three-dimensional gestures or air gestures) may involve this type of lateral swiping motion and/or may involve movement of one or more of the user's finger in the vertical (Z) dimension away," it continues.

So rather than having to look to see the place on a Touch Bar to operate, say, the volume controls, a user may be able to wave their hand over a "sensing strip" instead. Certainly they won't have to be so precise with where they tap their finger.

The intent of this particular patent application is practically the opposite of Apple's recent one on potentially adding screens to MacBook Pro trackpads and keyboards. That one posited that people look away from their screens and down to their keyboards, saying that it is "difficult for the user to maintain visual focus on both the keys being typed and the main display rendering the typed characters."

Cross-section from the patent showing how a Touch Bar-like sensor strip could detect both taps and gestures in the air
Cross-section from the patent showing how a Touch Bar-like sensor strip could detect both taps and gestures in the air

This latest patent disagrees. "It can be challenging to operate electronic devices using certain input devices," it says. "For example, some input devices may operate best when a user is looking directly at the device. This may cause the user to look away from content that is currently being presented on a display and can disrupt a user's work flow."

So this patent is for touch-typists. It aims to give users the ability to focus on the screen, and control aspects of their Mac as easily as they can type. This ability to use "air gestures above the sensing strip" is intended to be a supplement to touch, though rather than a replacement.

"The sensing strip may also... detect touch events associated with direct contact of a user's fingers or other external objects with the sensing strip," continues the patent.

So a strip could retain the same functionality as a Touch Bar, but add this extra ability to recognize and respond to more vague gestures in the air above the device.

"In general, any dynamic finger activity (up and down movement in the Z dimension, lateral movement in the X and/or Y dimensions, etc.) can be used in controlling [the] device 10," says the patent. "As an example, [the] sensing strip... can detect how rapidly a user's finger is moving towards... and can use this information to implement a velocity sensitive virtual piano key in a music application."

The sensor strip patent application has three inventors, Adam T. Garelli, Dinesh C. Mathew, and Senem E. Emgin. Garelli and Mathew were both previously credited as inventors on the a parent regarding redesigning the iMac into a single sheet of glass, plus Garelli and Emgin were both previously credited on a patent for how future iPad keyboards could connect to the touch screen instead of the Smart Connector.


  • Reply 1 of 8
    I am sorry to say this, but that would make it slightly more functional than near useless which it is currently.
  • Reply 2 of 8
    MplsPMplsP Posts: 3,966member
    My biggest complaint with the touch at OB my MBP is that if my finger is a bit too high when I hit a number key it will brush the touchpad and end up inserting an autocorrect word, or whatever happened to be there. The Touch Bar is so sensitive that you don’t even realize you’ve touched it. 

    It has a couple of useful features but mostly it’s just a gimmick that adds cost without true function. 
  • Reply 3 of 8
    gatorguygatorguy Posts: 24,387member
    Has anyone else read that patent application? It is exceedingly broad and will no doubt be trimmed a few times before getting USPTO approval. 

    "The sensing strip may have a one-dimensional array of sensor elements such as capacitive sensor elements, optical sensor elements, ultrasonic sensor elements, or radio-frequency sensor elements... 

    Electronic device 10 may be a computing device such as a laptop computer, a computer monitor containing an embedded computer, a tablet computer, a cellular telephone, a media player, or other handheld or portable electronic device, a smaller device such as a wrist-watch device, a pendant device, a headphone or earpiece device, a device embedded in eyeglasses or other equipment worn on a user's head, or other wearable or miniature device, a television, a computer display that does not contain an embedded computer, a gaming device, a navigation device, an embedded system such as a system in which electronic equipment with a display is mounted in a kiosk or automobile, equipment that implements the functionality of two or more of these devices, or other electronic equipment. Illustrative configurations in which a sensing strip such as an air gesture sensing strip is incorporated into an electronic device such as a laptop computer may sometimes be described herein as an example.

    devices 12 may include buttons, joysticks, scrolling wheels, touch pads, key pads, keyboards, microphones, speakers, tone generators, vibrators and other haptic output devices, sensors with digital image sensors such as visible light cameras and other sensors, light-emitting diodes and other status indicators, data ports, etc.

    devices 12 may include one or more displays such as display 14. Display 14 may be an organic light-emitting diode display, a liquid crystal display, or other display. Display 14 may be a touch screen display that includes a touch sensor for gathering touch input from a user or display 14 may be a touch insensitive display that is not sensitive to touch. A touch sensor for display 14 may be based on an array of capacitive touch sensor electrodes, acoustic touch sensor structures, resistive touch components, force-based touch sensor structures, a light-based touch sensor, or other suitable touch sensor arrangements.

    Input-output devices 12 may also include sensors 18. Sensors 18 may include magnetic sensors, ambient light sensors, force sensors, pressure sensors, microphones, humidity sensors, gas sensors, accelerometers, gyroscopes, and/or other sensors. Sensors 18 may be used in gathering information on the environment surrounding device 10 and may be used in gathering user input..


    edited March 2020 FileMakerFeller
  • Reply 4 of 8
    wood1208wood1208 Posts: 2,920member
    A time will come when Apple will make MAC/Macbook keyboard area to include touch keys, gesture based to imitate iPhone/iPad. This will extend Apple's desire not make MAC/Macbook display touch screen. That would be expensive so Apple should let guard down and make display acquire touch screen feature.
  • Reply 5 of 8
    pscooter63pscooter63 Posts: 1,080member
    I dunno.  Theremins, for example, demand a LOT from the user in terms of consistent spatial accuracy.  

    Seems like someone watched the Iron Man movies, and decided to run with the visual conceit.
  • Reply 6 of 8
    macguimacgui Posts: 2,388member
    This AirTouchBar will be used with the upcoming Apple Conduct app.

    "Lead your own Philharmonic!"
  • Reply 7 of 8
    gatorguy said:
    Has anyone else read that patent application? It is exceedingly broad and will no doubt be trimmed a few times before getting USPTO approval.
    It boggles the mind that the signal to noise ratio on this application is so woeful. Yet the quality wouldn't be so low if there wasn't an expectation that it would be reviewed as-is. Shameful. Standards need to be raised across the board.
  • Reply 8 of 8
    entropysentropys Posts: 4,221member
    Ahh the Touch Bar, the Segway of keyboard options.
Sign In or Register to comment.