Future Magic Mouse could detect gestures in the air like Apple Vision Pro

AppleInsider

Apple has been researching a new Magic Mouse for the Mac that would detect gestures -- not on its surface, but on the table near it.

The finger does attach to a hand -- it just doesn't have to touch the Magic Mouse

When Apple introduced its Magic Mouse back in 2009, it featured multi-touch gestures. Since then it's been redesigned, and there have been rumors of a further redesign.

Those rumors, though, have concentrated on the recurring issue that the USB-C charging port is on the bottom. There hadn't been any idea that Apple has been looking at anything else, certainly not at adding more gestures to it.

But a newly-granted patent called "Electronic devices with extended input-output capabilities," that is exactly what Apple has been doing. Only, alongside taps and swipes, the whole idea is literally about gestures alongside the mouse.

"A mouse or other electronic pointing device may have an array of sensors on a housing sidewall configured to gather information on the location of a finger of the user in an area that is laterally adjacent to the electronic pointing device," says the patent. "The sensors may be optical sensors based on light-sources such as light-emitting diodes or lasers and light detectors."

"Capacitive sensors may also be used in gathering user finger input in the area laterally adjacent to the electronic pointing device," it continues.

The idea is that this would mean "three-dimensional gestures may be supplied by a user." The gestures could be made by "moving the user's fingers or hand through the air near an electronic device."

So a future Magic Mouse could gain at least a subset of the type of gesture detection that is so key to the use of the Apple Vision Pro. Backing that up, the patent adds that the gestures could be detected using "one or more visible light cameras and/or one or more infrared cameras."

As well as cameras, the patent talks about projectors. The mouse, or other device, "may include projectors... [which] may, for example, project an image... onto [a] surface.

"[The] images projected... may contain virtual keys or other content with visual elements that a user may select with finger input," it continues. "As an example, [it] may project an image that contains multiple selectable regions... (e.g., visual elements such as file folder icons, keys such as alphanumeric keys, menu options, labels with 10 text, graphics, and/or moving image content, etc.)."

So exactly in the way that you can now get QWERTY keyboards that are laser-projected onto any surface, the mouse could present an image on your desk. You could then tap on that image, and the mouse's cameras will register your position and infer which projected control you meant.

That sounds more like a MacBook Pro, for instance, projecting a keyboard or a mouse area for the user to move their fingers through. And there are illustrations that show exactly that -- except they still include a regular keyboard and trackpad.

The mouse could project an image for a user to see and touch -- but so could a MacBook Pro

All patents do try to anticipate future uses and future patents from other firms, and as ever this one does so by listing more than a mouse. Typically Apple will use phrases like "a mouse or other pointing device," but this time it goes very much further.

While it does concentrate on this facility being used in a mouse, it also briefly specifies that the same technology could be used in:

• trackpad, or other pointing device


• laptop computer


• computer monitor containing an embedded computer


• tablet computer


• cellular telephone


• media player


• wrist-watch


• pendant


• headphone or earpiece


• headset

And more than a dozen other examples, including "electronic equipment embedded in tables, chairs, desks, [or] other furniture." So drumming your fingers next to your mouse might kickstart a Shortcut.

Putting a coffee mug down on your desk might turn on your Mac. Spreading your fingers on your table might act as a pinch and zoom on the device.

But Apple does seem to at least have an idea that such a mouse, or desk, would let a user supplement other input, rather than being a complete gesture by itself. The mouse, or chair, would determine "the position of a user's finger or other body part moving through the air," it says.

Then this "additional user input may help extend the input capabilities... and thereby supplement the information gathered using buttons, touch sensors, pointing device movement sensors."

"User input may be used in manipulating visual objects (e.g., icons, etc.), may be used in supplying [the] system... with text," it continues, "may be used in making menu selections, and/or may otherwise be used in operating the equipment of [the] system."

So you could maybe click on a "Bid" button in an auction, and then wave your hand to say how high you're willing to go.

Or perhaps more practically, a flick of the wrist near the mouse could be enough to move the cursor around the Mac's screen. Just as now you can click even only roughly near a button and it will be detected, perhaps a slapping motion could tell the Mac to click the highlighted tick box.

As ever, though, the patent is chiefly concerned with how something might work. While it gives an unusually comprehensive list of possible devices, it doesn't give examples of why a user might want to do any of this.

The mouse could sense nearby movement, but so could an iMac, so could a trackpad

And then as ever with all patents, the fact that it has been granted does not guarantee that the idea will make it into a future product.

The three credited inventors on the patent include the extremely prolific Paul X. Wang. His very many previous patents and patent applications, include one for replacing the old Touch Bar with an Apple Pencil holder.



Read on AppleInsider

applezulu

As long as they keep the charging port on the bottom, I'm good with any possible upgrades. Or they could replace the port with inductive MagSafe charging, which naturally would be on the bottom. I wonder if people would still go nuts if you could just set it on an iPhone or watch MagSafe charger, as you can with an AirPods case? I mean, it would no longer be on its side or upside down, as with the current plug-in port, but it would still be insistently wireless. 

netrox

No, I would rather see it gone. It's ugly. It's distracting. I always tend to try to plug my USB device into MagSafe only to realize it's not the right port. 

cpenzone

Please God no.

fastasleep

"So you could maybe click on a "Bid" button in an auction, and then wave your hand to say how high you're willing to go."

lol wut

marvin

AppleInsider said:

Apple has been researching a new Magic Mouse for the Mac that would detect gestures -- not on its surface, but on the table near it.

So exactly in the way that you can now get QWERTY keyboards that are laser-projected onto any surface, the mouse could present an image on your desk. You could then tap on that image, and the mouse's cameras will register your position and infer which projected control you meant.

That sounds more like a MacBook Pro, for instance, projecting a keyboard or a mouse area for the user to move their fingers through. And there are illustrations that show exactly that -- except they still include a regular keyboard and trackpad.

There's a product that does something like this that's been around for a few years:



https://www.amazon.com/Leap-Motion-Controller-Gesture-Control/dp/B00E3CP9UM
https://leap2.ultraleap.com/products/leap-motion-controller-2/

It costs around $200 so would only be worthwhile if they can keep the cost down.

If they can integrate it into a more expensive Mac and it adds $50 to the price, it offers some interesting capability. It would be possible to draw without a touch surface and the Mac can be controlled like Apple Vision OS by pinching and dragging things around.

mac'em x

"...the recurring issue that the USB-C charging port is on the bottom..."

"Recurring"? What, it's on the bottom, and then one day, awesome, it's not on the bottom, but then, dammit, now it's back on the bottom again...

Is anyone at AppleInsider familiar with English?

edited May 7

corin_oz

I’d like to see not only the charging method change but also the design of the mouse overhauled. Though I do like its gesture functionality, I find it annoyingly over sensitive plus it might be nice to have a little more to hold on to. 

mplsp

AppleZulu said:

As long as they keep the charging port on the bottom, I'm good with any possible upgrades. 

for God's sake, why would anyone want such an absolutely f*cked up design? 

It's absolutely amazing that the easiest improvement Apple could make to the Magic Mouse is the one that they are too pig-headed to make.

randominternetperson

AppleInsider said:

The finger does attach to a hand -- it just doesn't have to touch the Magic Mouse

Are we sure?

Clearly Apple is hinting at a future of detachable digits.

tiredskills

The gesture detection would have to improve.  It's spotty on the AVP.