Future Apple hardware could work together to tell you about sounds in other rooms

Posted:
in Future Apple Hardware

If you're reading this, you've probably got a lot of Apple devices in your house. Apple is working on technology to get them all to listen to your house, and tell you at your desk when something is going on in another room.

Cross-section of a house at dusk with visible rooms and furnishings against a cloudy sky backdrop.
Apple wants homes to be filled with Apple devices that talk to each other -- and listen, too. (Source: Apple)



If your house is not already full of Apple devices, then Apple would very much like to change that -- and it's looking at one way of making it very appealing. For Apple has been researching how any one of your devices could detect and recognize any other one -- and what those other devices are hearing.

So if your baby cries in the bedroom while you're making up formula in the kitchen, perhaps the HomePod mini on your bedside table could send an alert to your Apple Watch. Or maybe the iPad in your kitchen could recognize that your kettle has boiled and so ping your iPhone to say get down here and make the tea.

The newly-granted patent, "Home sound localization and identification," builds on the existing ability of Apple devices to recognize the sound of smoke alarms. It builds on sound recognition and adds in location awareness, so that any one device can tell any other device where a sound is coming from.

The location may not sound especially useful as you should know where your baby is, and you'll remember where you put the kettle on a couple of minutes ago. But such a system could, for instance, recognize a front door bell and send an alert to someone who is hard of hearing.

This has the potential to work simply because of how, as Apple says in its patent, "the availability of these devices has allowed them to become ubiquitous in our lives and homes." Apple argues that the devices "can collect and access information about their surroundings and environment," too.

However, it also maintains that "current devices may not be able to utilize this computing power and information to provide solutions to many problems and questions a user may experience in day-to-day life." Consequently, Apple claims that "a need exists to use the capabilities and ubiquity of electronic devices in new ways to solve heretofore unaddressed everyday problems."

When a sound fits certain characteristics, such as type or volume, then the device can act. "In use, one or more of the devices can detect a sound and transmit one or more signals corresponding to the detected sound to the processor," says Apple.

"The processor can be configured to determine a physical location of origin of the sound based at least in part on the received signals," it continues, "as well as additional information that can be collected or received by the system, such as the locations of users or objects in the space, the time of day, and/or calendar events."

So if you have a party in your calendar, for instance, devices might set themselves so that only louder sounds get recognized.

Schematic drawing of a person inside a house, interacting with various home automation devices labeled with numbers.
Detail from the patent showing one example of a device notifying a user of a sound in another room



All of this means that your Apple devices will be listening all the time, and as with every recent Apple patent, the text includes reference to the need for privacy. Given that they are listening, the devices could be listening out only for specific sounds, such as alarms, kettles boiling, and so on, and also ignore general background noises.

But, unusually, the patent does give room for "targeted content" to be delivered by devices. The suggestion is that this could include advertising, but the patent focuses instead on health benefits.

"For instance, health and fitness data may be used to provide insights into a user's general wellness, or may be used as positive feedback to individuals using technology to pursue wellness goals," it says. "The present disclosure contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices."

Where the patent is not clear is in how this health and fitness information fits with the rest of this idea. There's a suggestion that it just means giving "positive feedback to individuals using technology to pursue wellness goals."

So perhaps that's just that at certain times of day some or all of your devices nag you. Or conceivably it could be that if your Apple TV 4K doesn't hear you cursing enough while playing Apple Fitness+ that it tells your partner that you're slacking.

This patent is credited to Paul G. Puskarich and Fiona P. O'Leary. Puskarich has previously worked on patents regarding what's called focused listening.



Read on AppleInsider

Comments

  • Reply 1 of 10
    omasouomasou Posts: 572member
    "Have you checked the children?"...."It's Coming From Inside The House"
    mattinozwatto_cobraJaphey
  • Reply 2 of 10
    Would rather they develop the clunky “Intercom” feature on HomePods along these lines—incorporating other devices. 
    byronlwilliamlondonwatto_cobra
  • Reply 3 of 10
    omasouomasou Posts: 572member
    edited February 27
  • Reply 4 of 10
    dewmedewme Posts: 5,356member
    I’m totally into ambient/ubiquitous computing as a personal assistant and automation hub, but a number of the examples given in this article sound very similar to what Amazon has been delivering for a few years already, e.g. Alexa Care Hub, smoke & CO alarm detection, glass breakage detection, unusual sounds detection, etc. 

    This part of the article scares/annoys crap out of me:

    But, unusually, the patent does give room for "targeted content" to be delivered by devices.

    In other words, get ready for ubiquitous advertising to infringe on yet another aspect of our lives. Apple listening devices will be able to detect the acoustic signature of your nearly empty toilet paper roll spinning and push out an ad to you asking if you want to re-order another 12-pack, along with maybe some Gas-X and fiber supplements. 

    Lucky us.
  • Reply 5 of 10
    I see a cat in the patent diagram...  "The cat scratching expensive soft furniture sound detection" alert some might find useful
    Sound training could be an issue, unless you can rent a trained cat to scratch on command.
    williamlondonForumPostwatto_cobra
  • Reply 6 of 10
    omasou said:
    Thanks for the useful info, will give it a try!
    williamlondonwatto_cobra
  • Reply 7 of 10
    Apple devices are already always listening. That's how Siri works. The only difference is that a trigger word won't be needed to initiate further action.

    - Charlie pours the last of the milk into his cereal
    - "Sounds like you are out of milk. There's a $2.00 off deal at GroceryX. Shall I arrange a delivery?"
    watto_cobra
  • Reply 8 of 10
    mattinozmattinoz Posts: 2,313member
    mjpbuy said:
    I see a cat in the patent diagram...  "The cat scratching expensive soft furniture sound detection" alert some might find useful
    Sound training could be an issue, unless you can rent a trained cat to scratch on command.
    "RoboVac... dispatched"
    watto_cobra
  • Reply 9 of 10
    Honestly, the existing intercom feature has got me thinking about putting one in almost every room.
  • Reply 10 of 10
    mattinozmattinoz Posts: 2,313member


    When a sound fits certain characteristics, such as type or volume, then the device can act. "In use, one or more of the devices can detect a sound and transmit one or more signals corresponding to the detected sound to the processor," says Apple.

    "The processor can be configured to determine a physical location of origin of the sound based at least in part on the received signals," it continues, "as well as additional information that can be collected or received by the system, such as the locations of users or objects in the space, the time of day, and/or calendar events."

    Read on AppleInsider

    This is the the sort of ML Task Analogue processors are excelling at with very low power demands to boot. It seems a natural fit for many of Apple's current ML tasks that they would be looking at such a reactive low-power draw processor type to be added to the other special cores they use. 
Sign In or Register to comment.