'Apple Car' may control how doors move, offer motion sickness-free VR
The long-rumored Apple Car may have doors that prevent passengers from being hit by another vehicle as they exit the car, while an in-car VR system could have added features to minimize the chance of motion sickness.
A 2017 partnership between Ubisoft and Renault experimented with in-car VR entertainment.
The Apple Car has been the subject of numerous patent applications from Apple over the years, with more filings surfacing revealing it is still working on new elements for the self-driving vehicle. Applications published on Thursday by the US Patent and Trademark Office reveal concepts relating to passenger safety and in-car virtual reality systems.
Though useful, doors aren't obviously providing any level of intelligence-derived safety features, as it has to be dragged in place manually and interacted with by the user. However, in the patent application for "Dynamic Element Protection," Apple suggests some form of intelligent assistance could be at play.
The filing has Apple describing how a car door could be controlled to deal with "dynamic elements located within proximity of the vehicle." This is taken to mean hazards that may arise like other vehicles or pedestrians, though generally any moving item nearby could be considered a hazard.
Apple Car may control the doors, preventing them from opening if the car is sinking.
In short, it is a system that manages whether a door or window will open or stay shut, as well as to manage the automatic opening of said doors and windows. Depending on the situation, the system may prevent the entryways from opening until it is safe enough to do so.
Using the car's built-in sensors, the system monitors nearby objects and other elements in motion, and then acts depending on that data. For a parked car, it may elect to keep the door closed as another vehicle passes by, or restrict the amount the door swings open by to prevent contact with another vehicle.
While this is a simple example, the filing goes into a few other areas that are more complex, such as if a car has been parked and is occupied with a passenger or a pet, but without the driver. In such a situation, Apple suggests the air conditioning could be enabled to keep the climate habitable for extended periods.
In the case of an emergency with the driver no longer in the vehicle, the car could automatically lock doors based on the response of other occupants or notifying the driver directly, and if necessary perform an emergency response if specific conditions are detected.
On the more dramatic side, if a car is partially or fully submerged, the vehicle goes into an entire routine of closing windows, locking doors, unlocking safety restraints, deploying floatation devices, and generating a distress signal. It then provides air to the vehicle's interior if needed, and then asks if passengers are ready to leave the vehicle, with a positive response opening both windows and doors.
The filing lists its inventors as Tie-Qi Chen, Benjamin B. Lyon, Byron B. Han, and Ahmad Al-Dahle. It was originally filed on September 22, 2016.
Apple files numerous patent applications on a weekly basis, but while the existence of a filing indicates areas of interest for Apple's research and development efforts, it doesn't guarantee the idea will appear in a future product or service.
The filing seems to work nicely with another patent from September 2019, one where force-feedback hinges would adjust how a door moves while it is being swung, including taking into account gravity and how the vehicle is positioned on unstable ground.
The first, an "Augmented reality display system," discusses an AR display that is created on one or more transparent surfaces, such as the windscreen. The display would in theory be able to show important information to the driver and passengers, such as speed, as well as the location of road hazards both in view and unseen, such as a car blocked from view by a building.
An in-screen AR view could offer speed notifications on the windscreen of Apple Car.
The same system would be able to provide extra details, such as superimposing enhanced graphics over road elements, like a brighter road sign in a different language, to make it easier for the driver to see and understand. For driving a route, the same system could highlight the intended path on the road for the driver to follow.
This is not the first time Apple has proposed some form of AR view for drivers or passengers, as it has done so a few times in the past. This has included one version superimposing a digital view of the road on the windscreen, lining up with the driver's eyeline.
Apple has also explored ways to embed display systems into layers of glass, with the image able to be seen by those inside the vehicle but not outside. One August patent explains about how the layers of glass can be combined with liquid crystal or OLED panels and other layers to create a display sandwich, one that would be ideal for assisting with driving.
The filing was invented by Kjell F. Bronder, Scott M. Herz, and Karlin Y. Bark, and was filed on September 22, 2016.
The other related patent application, the innocently-named "Immersive virtual display," is less about creating a superior virtual headset and more making it so that the passenger using it doesn't suffer from motion sickness.
One of the issues when creating a VR display is having to make the virtual view match the expectations of the passenger, in terms of camera motion and angle, among other elements, with a mismatch in this area leading to motion sickness. In a car, there is more motion that the user may be aware of, but in a VR headset, it is only felt, as the real-world view is blocked off.
Apple plans to fight in-car VR motion sickness with moving backgrounds.
To combat the effects of motion sickness, Apple suggests that the virtual world could be reactive, specifically to the car's own motions. Taking into account the movements of the car and its onboard sensors, the virtual view could change, such as altering direction or speed, to match what the passenger ultimately feels.
The background could be made to be completely independent of whatever task or activity the user is doing, such as talking around a virtual meeting cable with a background of a moving desert scene. The system could also provide more accommodations to a passenger if it knows ahead of time that they are prone to motion sickness.
The filing lists its inventors as Mark B. Rober, Sawyer I. Cohen, Daniel Kurz, Tobias Holl, Benjamin B. Lyon, Peter Georg Meier, Jeffrey M. Riepling, and Holly Gerhard. It was filed on April 29, 2020.
Apple's previous filings on in-car VR and motion sickness prevention largely involved the use of inertial measurement devices on the headset and connected mobile devices, rather than relying on readings from the vehicle's sensors.
Apple isn't the only one trying to crack in-car VR, as a partnership between Ubisoft and Renault in 2017 aimed to make it an in-car entertainment option for self-driving vehicles.
A 2017 partnership between Ubisoft and Renault experimented with in-car VR entertainment.
The Apple Car has been the subject of numerous patent applications from Apple over the years, with more filings surfacing revealing it is still working on new elements for the self-driving vehicle. Applications published on Thursday by the US Patent and Trademark Office reveal concepts relating to passenger safety and in-car virtual reality systems.
Protective Car Doors
Typically, a car door provides safety for the user by simply existing. It helps keep the vehicle rigid, holds out the elements, minimizes the felt impact during a crash, and can even keep passengers safe from attackers when the door is locked.Though useful, doors aren't obviously providing any level of intelligence-derived safety features, as it has to be dragged in place manually and interacted with by the user. However, in the patent application for "Dynamic Element Protection," Apple suggests some form of intelligent assistance could be at play.
The filing has Apple describing how a car door could be controlled to deal with "dynamic elements located within proximity of the vehicle." This is taken to mean hazards that may arise like other vehicles or pedestrians, though generally any moving item nearby could be considered a hazard.
Apple Car may control the doors, preventing them from opening if the car is sinking.
In short, it is a system that manages whether a door or window will open or stay shut, as well as to manage the automatic opening of said doors and windows. Depending on the situation, the system may prevent the entryways from opening until it is safe enough to do so.
Using the car's built-in sensors, the system monitors nearby objects and other elements in motion, and then acts depending on that data. For a parked car, it may elect to keep the door closed as another vehicle passes by, or restrict the amount the door swings open by to prevent contact with another vehicle.
While this is a simple example, the filing goes into a few other areas that are more complex, such as if a car has been parked and is occupied with a passenger or a pet, but without the driver. In such a situation, Apple suggests the air conditioning could be enabled to keep the climate habitable for extended periods.
In the case of an emergency with the driver no longer in the vehicle, the car could automatically lock doors based on the response of other occupants or notifying the driver directly, and if necessary perform an emergency response if specific conditions are detected.
On the more dramatic side, if a car is partially or fully submerged, the vehicle goes into an entire routine of closing windows, locking doors, unlocking safety restraints, deploying floatation devices, and generating a distress signal. It then provides air to the vehicle's interior if needed, and then asks if passengers are ready to leave the vehicle, with a positive response opening both windows and doors.
The filing lists its inventors as Tie-Qi Chen, Benjamin B. Lyon, Byron B. Han, and Ahmad Al-Dahle. It was originally filed on September 22, 2016.
Apple files numerous patent applications on a weekly basis, but while the existence of a filing indicates areas of interest for Apple's research and development efforts, it doesn't guarantee the idea will appear in a future product or service.
The filing seems to work nicely with another patent from September 2019, one where force-feedback hinges would adjust how a door moves while it is being swung, including taking into account gravity and how the vehicle is positioned on unstable ground.
AR and VR Displays
Two patent applications relate to the use of virtual reality or augmented reality in vehicles.The first, an "Augmented reality display system," discusses an AR display that is created on one or more transparent surfaces, such as the windscreen. The display would in theory be able to show important information to the driver and passengers, such as speed, as well as the location of road hazards both in view and unseen, such as a car blocked from view by a building.
An in-screen AR view could offer speed notifications on the windscreen of Apple Car.
The same system would be able to provide extra details, such as superimposing enhanced graphics over road elements, like a brighter road sign in a different language, to make it easier for the driver to see and understand. For driving a route, the same system could highlight the intended path on the road for the driver to follow.
This is not the first time Apple has proposed some form of AR view for drivers or passengers, as it has done so a few times in the past. This has included one version superimposing a digital view of the road on the windscreen, lining up with the driver's eyeline.
Apple has also explored ways to embed display systems into layers of glass, with the image able to be seen by those inside the vehicle but not outside. One August patent explains about how the layers of glass can be combined with liquid crystal or OLED panels and other layers to create a display sandwich, one that would be ideal for assisting with driving.
The filing was invented by Kjell F. Bronder, Scott M. Herz, and Karlin Y. Bark, and was filed on September 22, 2016.
The other related patent application, the innocently-named "Immersive virtual display," is less about creating a superior virtual headset and more making it so that the passenger using it doesn't suffer from motion sickness.
One of the issues when creating a VR display is having to make the virtual view match the expectations of the passenger, in terms of camera motion and angle, among other elements, with a mismatch in this area leading to motion sickness. In a car, there is more motion that the user may be aware of, but in a VR headset, it is only felt, as the real-world view is blocked off.
Apple plans to fight in-car VR motion sickness with moving backgrounds.
To combat the effects of motion sickness, Apple suggests that the virtual world could be reactive, specifically to the car's own motions. Taking into account the movements of the car and its onboard sensors, the virtual view could change, such as altering direction or speed, to match what the passenger ultimately feels.
The background could be made to be completely independent of whatever task or activity the user is doing, such as talking around a virtual meeting cable with a background of a moving desert scene. The system could also provide more accommodations to a passenger if it knows ahead of time that they are prone to motion sickness.
The filing lists its inventors as Mark B. Rober, Sawyer I. Cohen, Daniel Kurz, Tobias Holl, Benjamin B. Lyon, Peter Georg Meier, Jeffrey M. Riepling, and Holly Gerhard. It was filed on April 29, 2020.
Apple's previous filings on in-car VR and motion sickness prevention largely involved the use of inertial measurement devices on the headset and connected mobile devices, rather than relying on readings from the vehicle's sensors.
Apple isn't the only one trying to crack in-car VR, as a partnership between Ubisoft and Renault in 2017 aimed to make it an in-car entertainment option for self-driving vehicles.
Comments
Of all the VR scenarios one is likely to encounter while driving it's odd that they picked the one involving a car going into the water as an example. Perhaps if the driver wasn't wearing a VR headset, possibly playing video games (none from Epic of course), fiddling with his headset, and being completely oblivious to his surroundings while driving he wouldn't have ended up in the water in the first place. So now we know the Apple Car will have an air generation system. I wonder if it will have ballast tanks and a periscope with surveillance video piped into the VR headset?
The upside is that the motion sickness feature may help prevent sea sickness as the Apple car slowly drifts out to sea with the tide.
"Hey Siri - play me some jaunty sea shanties..."
Gen Y: "What's a sedan, mommy?"
Gen Z: "What's a steering wheel, grandpa?"
Lol “drivers”