Future HomeKit could track you through your house and predict your needs

Posted:
in General Discussion
A future version of HomeKit may keep track of where the people in the house are, and learn user habits to figure out when to automatically take actions, without you having to ask Siri.

HomeKit


The temperature and humidity sensors in the new HomePod, and now enabled in the HomePod mini, are designed to be used as part of automated systems. Right now, you can create a Shortcut that says if the indoor temperature falls below a certain point, you want your heater turned on.

That does require a heater you can control remotely, or perhaps just a smart outlet, plus the HomePod. But it also requires you to set up that Shortcut, and "Using In-home Location Awareness," a newly-granted patent, suggests that Apple wants to move away from that.

Rather than you having to create a Shortcut or to set up a HomeKit automation, Apple wants it all to just happen for you. By itself.

"Users often perform the same or repeated actions with accessory devices while in a particular location," says the patent. "For example, every time a user comes home from work, they may close the garage door when they are in the kitchen."

"Therefore, certain activities with respect to devices in a home may be performed regularly and repeatedly (e.g., daily, several times throughout a day) while the user is in a certain location," it continues. "This can be a time consuming and tedious task for a user since these tasks are performed regularly or several times throughout the day."

"Thus... it is desired for the home application on the mobile device to be able to determine a location of the user," says Apple, "and suggest a corresponding accessory device that a user may want to control or automatically operate a corresponding accessory device based on the location of the mobile device of the user."

Detail from the patent showing a range of connected devices
Detail from the patent showing a range of connected devices


Apple doesn't want to remove the user from the decision to turn on a heater, or a light, but rather to remove any effort.

"[When] a user is at a determined location according to detected sensor information, one or more associated accessory devices can be suggested to a user, thereby enhancing a user's experience," says the patent. "Also, rooms and accessory devices associated with the room can be suggested to a user."

"In addition, scenes and accessory devices associated with the scenes can be suggested to the user," it continues. "The suggestions regarding accessory devices and scenes can be learned or can be rule based."

The key word there is learned. Currently an iOS widget can offer you different information at different times of the day, and it learns that, for instance, you check your calendar first thing, but you like news headlines at the end of the day.

That's exactly what Apple wants with your smart home devices. They're smart, you use them all the time, they should just work.

"It would be beneficial if accessory devices and/or scenes can be automatically suggested to a user based on their current location," says the patent, "or if an accessory device is automatically controlled based on a current location of the mobile device based on the user's history of activity with an accessory device."

The patent runs to more than 17,000 words, and while naturally a large part of that is to do with privacy, the majority is about precisely detecting a user's presence.

As is typical for a patent trying to cover as broad an idea as possible, it's a little vague on the specifics. But if Machine Learning can already provide services such as presenting a calendar in the morning, it is this issue of precision location finding that is what's new and key.

Geolocation is not yet precise enough

Currently, HomeKit is able to have an automation that, for instance, stops music playing when everyone has left the house. That's chiefly done by determining whether iPhones are connected to the home's Wi-Fi network, though.

Or there are motion sensors which can pick up when you enter a room. But the problem there is that, normally, they can only detect motion such as entering or leaving, they don't actually sense a presence.

Aqara has begun selling what it calls a "human presence" sensor, which is able to detect minute body movements, but its device is not yet commonly available.

Rather than look to any one specific new technology, Apple's patent suggests combining information from many sources, and then making calculations.

"For example, the mobile device can include sensors to measure the distance that a user is walking or stepping as they turn on lights in, for example, a room or a house," says Apple. "Motion detectors and motion sensors in, for example, the accessory devices, can also be used to aggregate data regarding a user's movement in a room or area."

The patent doesn't directly refer to the U1 ultra wideband chip, but if it's in a user's devices, it will clearly be used.

So if you enter a room and also head toward your desk lamp instead of the overhead light, HomeKit could know to turn on only that lamp.

Apple is adding the U1 processor to more devices, plus HomeKit can already get information from sources such as motion sensors. Plus, Apple's support of Matter should mean such plans could extend to many more smart home devices.

Read on AppleInsider

Comments

  • Reply 1 of 13
    lkrupplkrupp Posts: 10,557member
    And I just found out from a TidBits article that Siri now supports rebooting your iPhone/iPad. “Hey Siri, Reboot” It works
    watto_cobraAlex1N
  • Reply 2 of 13
    This is one of those things that makes sense as research but would likely be impractical/annoying in real world use. 
    boxcatcherwilliamlondonFileMakerFellerwatto_cobraAlex1N
  • Reply 3 of 13
    Rather than trying to install a vast array of sensors across an entire house and attempt to infer where someone is + automate what they want, I have to think it’ll end up being a lot easier to just use “Apple Glass”.

    *Walk into a room wearing Apple Glass*

    *Look at a light* Turn on that light

    *Light turns on*

    Basically: why not use the one sophisticated sensor a person is wearing + explicit intent vs. tens of fixed sensors + inferred intent?
    edited February 2023 watto_cobra
  • Reply 4 of 13
    Yeah, we don’t need this.

    At least I certainly don’t want to be tracked throughout my home just for the possibility that my home automation might be a wee bit helpful but in all likelihood it would just be annoying or wrong.
    williamlondonFileMakerFellerdecoderringwatto_cobraAlex1N
  • Reply 5 of 13
    I was thinking about investing in Homekit but if there was one thing that would turn me away from it forever, that would be it in a nutshell. If Apple implements this then you can count me out forever investing in any home automation system which is kinda ironic as my Masters degree is in Control Engineering.

    I hate all forms of tracking and adverts but as I'm a grumpy old boomer, I don't count do I?
    williamlondonFileMakerFellerdecoderringboboliciousmuthuk_vanalingamwatto_cobraAlex1N
  • Reply 6 of 13
    After IOS 16.2 update wrecked HomeKit, I can't even use it to control lights that have worked for years. Thank God for Alexa still working. I would appreciate a fix for HomeKit before adventures into new frontiers.
    itinj24williamlondonjd143cnwatto_cobraAlex1N
  • Reply 7 of 13
    dewmedewme Posts: 5,423member
    My expectations with respect to ambient computing aren’t aligned around Siri ordering more toilet paper, you know, just in case the meal in progress had some serious aftershocks. Geez. 

    I’d be happy with some assistance around energy usage based on approaching weather. 
    FileMakerFellerwatto_cobraAlex1N
  • Reply 8 of 13
    Hmm, it's just HomeKit making suggestions, just more suggestions.

    I actually have presence detection in my home right now (Not motion detection) and it is truly great that lights follow you as you go through the house. Also things like if the last person leaves a room whilst tv is running, to pause it, and then restart it when they return.

    Wish I could get music to follow me.

    Siobhan
    watto_cobraAlex1N
  • Reply 9 of 13
    Much as I would love to have Siri be as competent as Jarvis in being my virtual assistant, I’d rather be able to ask Siri first instead of having the automation anticipate my needs. In no way, would I ever let automation control the door to my home, or control a heating device in my home. If I tell Siri to turn the heat on, that’s one thing, for it to just assume that I want it turned on because the temperature drops below 65° at my house is another. I am an elderly lady, so having these devices carry out my needs is very attractive, and I have smart plugs to get Siri to tell them to do that, but I definitely, don’t want to do it on an anticipatory basis. 
    FileMakerFellerdecoderringwatto_cobraAlex1N
  • Reply 10 of 13
    Hmm, it's just HomeKit making suggestions, just more suggestions.

    I actually have presence detection in my home right now (Not motion detection) and it is truly great that lights follow you as you go through the house. Also things like if the last person leaves a room whilst tv is running, to pause it, and then restart it when they return.

    Wish I could get music to follow me.

    Siobhan
    Even in my advancing years, I can remember enough of the music I like to have access to it wherever I go. I'm surprised more people don't cultivate this skill.
    beowulfschmidtwatto_cobraAlex1N
  • Reply 11 of 13
    I was thinking about investing in Homekit but if there was one thing that would turn me away from it forever, that would be it in a nutshell. If Apple implements this then you can count me out forever investing in any home automation system which is kinda ironic as my Masters degree is in Control Engineering.

    I hate all forms of tracking and adverts but as I'm a grumpy old boomer, I don't count do I?
    For your consideration - 12 and counting: gizmodo.com/apple-iphone-privacy-analytics-12-lawsuits-statement-1850077715

    "Open up your iPhone’s settings, tap “Privacy & Security,” and scroll down to “Analytics & Improvements.” There you’ll find a setting label “Share iPhone Analytics.” Toggle it on and off all you want. Tests from Mysk found that Apple collects device analytics data, no matter how you adjust the control"

    Alex1N
  • Reply 12 of 13
     Tests from Mysk found that Apple collects device analytics data, no matter how you adjust the control"
    That level of tracking does not bother me for the following :-
    I don't do social media
    I don't use email on my phone.
    I don't use Safari that much either.
    When I'm driving, my phone is usually in Airplane mode.
    Most of the apps I use are related to charging my EV. (different charging networks) or you know... speaking to someone on the phone.
    The opportunity to fire ads at me is limited and I never respond to the few that I see anyway.
    My phone is not the device that runs my life, it is a tool that is used when needed.
     
    Alex1N
Sign In or Register to comment.