Apple smart home invention uses pattern recognition to solve multi-user dwelling issues

Posted:
in General Discussion edited May 2016
An Apple patent application revealed Thursday outlines plans for an automated smart home system that tracks individual users through their connected devices, detects common patterns and compiles events for triggering thermostats, lights, door locks and more.




Apple's application for "Aggregating user routines in an automated environment," as published by the U.S. Patent and Trademark Office, details a method by which multiple smart home devices are automatically and independently controlled based on user habits.

The system tracks user equipment, in this case iPhone and Apple Watch, through a smart home environment, identifies aggregate patterns and triggers accessories accordingly. For example, a user might arrive home every weekday at 4 p.m., turn on the house lights, set the thermostat to 72 degrees and switch on a TV set. Apple's solution would be capable of recognizing the weekday pattern and execute those actions automatically based on time and user location, among other measurements.

Like other products on the market, Apple's smart home invention relies on a single local device for hardware coordination, aptly dubbed the coordinator. In some embodiments, Apple TV acts as a hub, facilitating communication between user devices and a static network of smart home accessories. Apple proposes adding a control logic to the coordinator's list of duties.

To achieve full platform automation, that is to predict intent without intervention, the coordinator must first determine user routines. In some embodiments, the system gathers and analyzes user device metrics -- location, activity, device state -- over time to detect patterns. Typical location acquisition methods are mentioned, including GPS, though the collection and application of user motion and device state data is more nuanced.


Source: USPTO


The activity determining subsystem accumulates data generated by onboard sensors, including accelerometers, gyroscopes and other motion sensors, to detect how a user device is moving. By extrapolating this data, the system can decipher any number of user activities like running, driving and sleeping. Being a wearable, Apple Watch provides even more detail, including advanced biometrics like heart rate readings.

Device states also provide useful information to pattern determination algorithms. A device that is powered off, set to "do not disturb" or connected to a charger might indicate that a user is sleeping, for example.

Once location and activity information is collected, the coordinator can aggregate data from multiple users, parsing out patterns and relating them with smart home accessory status. Users who frequent an automated environment location might establish routines, such as coming home after work every weekday after visiting the gym. Once home, a user might unlock the front door, turn on the lights, set the thermostat to 72 degrees and turn on a TV set.

Next, a set of automation rules is established, each specifying a control action to be taken by an accessory once a condition is met. For example, house lights might be set to turn on at a certain time or when a user arrives home. These rulesets are set either manually or suggested by the coordinator device after it "learns" a user's behavior.

As for multiple user support, the system is able to monitor data in real time and make decisions based on observed activities and patterns. Automating a home for one person is difficult, and the problem is compounded when accommodating for two or more.

Apple proposes a real-time determination technique that studies accessory interactions for each member of the household and combines that information to draw more complex conclusions. Importantly, aggregating data across users can result in more intelligent automation in shared environments.

For example, by combining shared device data, the coordinator is able to compare and contrast user routines for two people as they apply to a given room. In this way, Apple's system can identify interactions that a first user does not perform when they are alone, but does perform when another user is present, and vice versa.

Apple goes on to detail more complex examples, use cases and invention variations.




Apple dabbles in smart home solutions with HomeKit, an iOS framework that in theory makes iPhone a control hub for the connected home. In practice, however, HomeKit is a far cry from the system described in today's invention. Control macros are overly simplified and offer little flexibility, there is no central "hub" app, pattern matching algorithms are non existent and integration with third-party manufacturers is lacking.

Rumors of a first-party HomeKit control app have persisted for months, but whether Apple plans to build technology described in today's patent application into such a release remains unclear.

Apple's smart home automation patent application was first filed for in September 2015 and credits Lukas M. Marti, Kevin P. McLaughlin, Arun G. Mathias and Ronald Keryuan Huang as its inventors.

Comments

  • Reply 1 of 8
    boboliciousbobolicious Posts: 1,167member
    ...how creepy is that...?
  • Reply 2 of 8
    ...how creepy is that...?
    What's creepy about it? I honestly don't understand. 
  • Reply 3 of 8
    mobiusmobius Posts: 380member
    ...how creepy is that...?
    What's creepy about it? I honestly don't understand. 
    Well, my knee-jerk thought was that it's pretty damn creepy. But having thought about it further, it would really only be creepy if it was a human that is tracking you. The fact is it's an algorithm. The is no human looking directly at your activities. Whether or not someone could hack in and spy on you is another matter of course.
    cornchip
  • Reply 4 of 8
    maestro64maestro64 Posts: 5,043member

    If Apple solves this problem then Home Automation will work. I have been doing home automation for a long time and this is the single issue which most people including myself do not like about home automation. How do you deal with multiply people who use a house and they all have their own preferences. In my house I set all the rules about when things turn on and off. If like the person who have the TV remote get to tell everyone what they will watch unless you have a TV for each person. If the system know who is in the house and what they are doing and set things accordingly they is may work.

    I made this point about the NEST thermostat, how does it handle multiply people in the house. and if the one person who has the app on their phone leaves what does NEST do. or better yet if everyone has the app, will you have remote wars going on with different people trying to set the temperatures and such. These ideas work when you are the only one in the house, put more in the house and the problem gets complicated.

  • Reply 5 of 8
    wonkothesanewonkothesane Posts: 1,738member
    Most welcome if implemented. For example it would be great if my Philips Hue only turns off lights when the last person left home and vv, for starters. I know you can do it already, but it involves setting up a server and some scripting. Apple has spoiled me in the past, I prefer things that just work :wink: 
    cornchip
  • Reply 6 of 8
    volcanvolcan Posts: 1,799member
    I think there really needs to be a HomeKit app. Why there isn't one I don't understand. I can envision that rather than let some automated routine aggregator take control, one should be able to open a HomeKit app and create your own scenes with drag and drop appliances, set the date and time, but also be able to execute the action by voice or cancel the action. I don't have anything against automation, I just want more control of what happens because there can be unwanted consequences when everything is triggered automatically.

    I know there are supposed to be customizable scenes but I've not read anywhere how it is actually accomplished.
    edited May 2016 patchythepirate
  • Reply 7 of 8
    vmarksvmarks Posts: 762editor
    volcan said:
    I think there really needs to be a HomeKit app. Why there isn't one I don't understand. I can envision that rather than let some automated routine aggregator take control, one should be able to open a HomeKit app and create your own scenes with drag and drop appliances, set the date and time, but also be able to execute the action by voice or cancel the action. I don't have anything against automation, I just want more control of what happens because there can be unwanted consequences when everything is triggered automatically.

    I know there are supposed to be customizable scenes but I've not read anywhere how it is actually accomplished.
    There is a HomeKit app. In fact, there are several. Insteon+, iDevices, Elgato Eve, and more all aggregate all the homekit devices. Use the one that works the way you like - they collect all the homekit devices into one app view.

    Everything you want to do, you can do in any of those HomeKit aggregator apps.
    volcan
  • Reply 8 of 8
    volcanvolcan Posts: 1,799member
    vmarks said:
    There is a HomeKit app. In fact, there are several. Insteon+, iDevices, Elgato Eve, and more all aggregate all the homekit devices. Use the one that works the way you like - they collect all the homekit devices into one app view.

    Everything you want to do, you can do in any of those HomeKit aggregator apps.
    Thanks for the message. Quick question. Do HomeKit devices usually require their own app to do the initial set up? When HomeKit was first announced I was hoping that Apple would provide the hub for all devices instead of having proprietary apps for each one.
Sign In or Register to comment.