Apple smart home invention uses pattern recognition to solve multi-user dwelling issues
An Apple patent application revealed Thursday outlines plans for an automated smart home system that tracks individual users through their connected devices, detects common patterns and compiles events for triggering thermostats, lights, door locks and more.
Apple's application for "Aggregating user routines in an automated environment," as published by the U.S. Patent and Trademark Office, details a method by which multiple smart home devices are automatically and independently controlled based on user habits.
The system tracks user equipment, in this case iPhone and Apple Watch, through a smart home environment, identifies aggregate patterns and triggers accessories accordingly. For example, a user might arrive home every weekday at 4 p.m., turn on the house lights, set the thermostat to 72 degrees and switch on a TV set. Apple's solution would be capable of recognizing the weekday pattern and execute those actions automatically based on time and user location, among other measurements.
Like other products on the market, Apple's smart home invention relies on a single local device for hardware coordination, aptly dubbed the coordinator. In some embodiments, Apple TV acts as a hub, facilitating communication between user devices and a static network of smart home accessories. Apple proposes adding a control logic to the coordinator's list of duties.
To achieve full platform automation, that is to predict intent without intervention, the coordinator must first determine user routines. In some embodiments, the system gathers and analyzes user device metrics -- location, activity, device state -- over time to detect patterns. Typical location acquisition methods are mentioned, including GPS, though the collection and application of user motion and device state data is more nuanced.
Source: USPTO
The activity determining subsystem accumulates data generated by onboard sensors, including accelerometers, gyroscopes and other motion sensors, to detect how a user device is moving. By extrapolating this data, the system can decipher any number of user activities like running, driving and sleeping. Being a wearable, Apple Watch provides even more detail, including advanced biometrics like heart rate readings.
Device states also provide useful information to pattern determination algorithms. A device that is powered off, set to "do not disturb" or connected to a charger might indicate that a user is sleeping, for example.
Once location and activity information is collected, the coordinator can aggregate data from multiple users, parsing out patterns and relating them with smart home accessory status. Users who frequent an automated environment location might establish routines, such as coming home after work every weekday after visiting the gym. Once home, a user might unlock the front door, turn on the lights, set the thermostat to 72 degrees and turn on a TV set.
Next, a set of automation rules is established, each specifying a control action to be taken by an accessory once a condition is met. For example, house lights might be set to turn on at a certain time or when a user arrives home. These rulesets are set either manually or suggested by the coordinator device after it "learns" a user's behavior.
As for multiple user support, the system is able to monitor data in real time and make decisions based on observed activities and patterns. Automating a home for one person is difficult, and the problem is compounded when accommodating for two or more.
Apple proposes a real-time determination technique that studies accessory interactions for each member of the household and combines that information to draw more complex conclusions. Importantly, aggregating data across users can result in more intelligent automation in shared environments.
For example, by combining shared device data, the coordinator is able to compare and contrast user routines for two people as they apply to a given room. In this way, Apple's system can identify interactions that a first user does not perform when they are alone, but does perform when another user is present, and vice versa.
Apple goes on to detail more complex examples, use cases and invention variations.
Apple dabbles in smart home solutions with HomeKit, an iOS framework that in theory makes iPhone a control hub for the connected home. In practice, however, HomeKit is a far cry from the system described in today's invention. Control macros are overly simplified and offer little flexibility, there is no central "hub" app, pattern matching algorithms are non existent and integration with third-party manufacturers is lacking.
Rumors of a first-party HomeKit control app have persisted for months, but whether Apple plans to build technology described in today's patent application into such a release remains unclear.
Apple's smart home automation patent application was first filed for in September 2015 and credits Lukas M. Marti, Kevin P. McLaughlin, Arun G. Mathias and Ronald Keryuan Huang as its inventors.
Apple's application for "Aggregating user routines in an automated environment," as published by the U.S. Patent and Trademark Office, details a method by which multiple smart home devices are automatically and independently controlled based on user habits.
The system tracks user equipment, in this case iPhone and Apple Watch, through a smart home environment, identifies aggregate patterns and triggers accessories accordingly. For example, a user might arrive home every weekday at 4 p.m., turn on the house lights, set the thermostat to 72 degrees and switch on a TV set. Apple's solution would be capable of recognizing the weekday pattern and execute those actions automatically based on time and user location, among other measurements.
Like other products on the market, Apple's smart home invention relies on a single local device for hardware coordination, aptly dubbed the coordinator. In some embodiments, Apple TV acts as a hub, facilitating communication between user devices and a static network of smart home accessories. Apple proposes adding a control logic to the coordinator's list of duties.
To achieve full platform automation, that is to predict intent without intervention, the coordinator must first determine user routines. In some embodiments, the system gathers and analyzes user device metrics -- location, activity, device state -- over time to detect patterns. Typical location acquisition methods are mentioned, including GPS, though the collection and application of user motion and device state data is more nuanced.
Source: USPTO
The activity determining subsystem accumulates data generated by onboard sensors, including accelerometers, gyroscopes and other motion sensors, to detect how a user device is moving. By extrapolating this data, the system can decipher any number of user activities like running, driving and sleeping. Being a wearable, Apple Watch provides even more detail, including advanced biometrics like heart rate readings.
Device states also provide useful information to pattern determination algorithms. A device that is powered off, set to "do not disturb" or connected to a charger might indicate that a user is sleeping, for example.
Once location and activity information is collected, the coordinator can aggregate data from multiple users, parsing out patterns and relating them with smart home accessory status. Users who frequent an automated environment location might establish routines, such as coming home after work every weekday after visiting the gym. Once home, a user might unlock the front door, turn on the lights, set the thermostat to 72 degrees and turn on a TV set.
Next, a set of automation rules is established, each specifying a control action to be taken by an accessory once a condition is met. For example, house lights might be set to turn on at a certain time or when a user arrives home. These rulesets are set either manually or suggested by the coordinator device after it "learns" a user's behavior.
As for multiple user support, the system is able to monitor data in real time and make decisions based on observed activities and patterns. Automating a home for one person is difficult, and the problem is compounded when accommodating for two or more.
Apple proposes a real-time determination technique that studies accessory interactions for each member of the household and combines that information to draw more complex conclusions. Importantly, aggregating data across users can result in more intelligent automation in shared environments.
For example, by combining shared device data, the coordinator is able to compare and contrast user routines for two people as they apply to a given room. In this way, Apple's system can identify interactions that a first user does not perform when they are alone, but does perform when another user is present, and vice versa.
Apple goes on to detail more complex examples, use cases and invention variations.
Apple dabbles in smart home solutions with HomeKit, an iOS framework that in theory makes iPhone a control hub for the connected home. In practice, however, HomeKit is a far cry from the system described in today's invention. Control macros are overly simplified and offer little flexibility, there is no central "hub" app, pattern matching algorithms are non existent and integration with third-party manufacturers is lacking.
Rumors of a first-party HomeKit control app have persisted for months, but whether Apple plans to build technology described in today's patent application into such a release remains unclear.
Apple's smart home automation patent application was first filed for in September 2015 and credits Lukas M. Marti, Kevin P. McLaughlin, Arun G. Mathias and Ronald Keryuan Huang as its inventors.
Comments
If Apple solves this problem then Home Automation will work. I have been doing home automation for a long time and this is the single issue which most people including myself do not like about home automation. How do you deal with multiply people who use a house and they all have their own preferences. In my house I set all the rules about when things turn on and off. If like the person who have the TV remote get to tell everyone what they will watch unless you have a TV for each person. If the system know who is in the house and what they are doing and set things accordingly they is may work.
I made this point about the NEST thermostat, how does it handle multiply people in the house. and if the one person who has the app on their phone leaves what does NEST do. or better yet if everyone has the app, will you have remote wars going on with different people trying to set the temperatures and such. These ideas work when you are the only one in the house, put more in the house and the problem gets complicated.
I know there are supposed to be customizable scenes but I've not read anywhere how it is actually accomplished.
Everything you want to do, you can do in any of those HomeKit aggregator apps.