'iPhone 8' could herald start of Apple's augmented reality ambitions
Computer simulations, and the means to visualize them, could be in the palm of your hand in new and potentially revolutionary ways later this year, if rumors about Apple's forthcoming "iPhone 8" bear fruit.
Apple CEO Tim Cook has made an assortment of remarks, heralding augmented reality and virtual reality as potential cornerstones of the company's future. Combined with rumors of a new dual-lens forward facing camera system and advanced facial recognition software in the "iPhone 8," it's possible that Apple's true interest in the space could be revealed later this year.
While virtual reality promises users a more immersive experience, by sinking a user in the environment with video and audio cues, augmented reality instead offers a hybrid and more social approach, merging a virtual world with the one around us.
Augmented reality is a clear interest for Apple. Rumors about the hardware in the "iPhone 8" coupled with Apple CEO Tim Cook's remarks about the technology suggest that the company is going to dive deep into augmented reality very soon.
Early implementations employed conventional displays in conjunction with mechanical haptics to provide full feedback of the environment to the user.
Because VR is an immersive and largely solitary experience that requires a headset, it's an unlikely fit for Apple. Instead, the company is likely to embrace an offshoot of VR, augmented reality, in a way that can allow users to interact with the world --?and others --?around them, via the iPhone.
Another possible use case is capture of virtual creatures with the flick of a finger, after spotting them in the park on a daily walk such as in Niantic's "Pokemon Go."
Done properly, augmented reality could be readily adopted by the iPhone crowd -- assuming it can be made "friction-free." As it turns out, that's an area where Apple historically has excelled.
Tim Cook on augmented and virtual reality
"The smartphone is for everyone, we don't have to think the iPhone is about a certain demographic, or country or vertical market: it's for everyone. I think AR is that big, it's huge," Cook said in an interview in February. "I get excited because of the things that could be done that could improve a lot of lives. And be entertaining."
"I view AR like I view the silicon here in my iPhone, it's not a product per se, it's a core technology. But there are things to discover before that technology is good enough for the mainstream," Cook concluded. "I do think there can be a lot of things that really help people out in daily life, real-life things, that's why I get so excited about it."
Apple's tangible moves in the AR and VR sector
Apple has made a number of strategic hires and acquisitions over the last five years to further its AR ambitions.
Motion capture specialist Faceshiftwas acquired in 2015, with machine learning and computer vision startup Perceptio grabbed earlier that year.
German AR firm Metaio and Flyby Media also go hand-in-hand with in-house development of transparent displays, iPhone-powered VR rigs, AR maps and other related technologies described in recent patent filings.
In January, Apple was reassigned IP from Metaio for an AR device with advanced point of interest labelling. Specifically, a pair of patents detail a mobile AR system -- or smartphone -- capable of detecting its surroundings and displaying generated virtual information to users in real time.
The "iPhone 8" and augmented reality
While the "iPhone 8" may not be sold to customers on AR prospects alone, it appears to be setting the table for wider adoption of the technology.
According to recent rumors, the sensor expected to be used in this year's "iPhone 8" is intended for a practical face scanning utility, and not solely or specifically for augmented reality application.
However, the technology described for the sensor appears to be a micro-miniaturized version of a LIDAR mapper or rangefinder.
LIDAR is a proven technology, with resolution and accuracy dependent on a combination of factors, not the least of which is the integration of the laser and the sensor -- which Apple's acquisition of Primesense in 2013 appears intended for.
To properly overly the environment requires accuracy -- which a rear-mounted LIDAR installation supplementing existing range finding equipment associated with the iPhone camera would give. It also requires heavy-duty processing power, which the existing A10 Fusion processor also has -- and any successor would as well.
According to analyst Ming-Chi Kuo of KGI Securities, the new camera system expected to arrive in Apple's "iPhone 8" will enable 3D sensing and modeling in applications, which could be used for a range of capabilities including replacing the head of a character in a 3D game, or taking a 3D selfie.
Beyond proprietary camera software, Apple has other AR- and VR-related edges over its competition as well. For example, the iPhone only has a few resolutions, where Android devices have literally hundreds.
The "iPhone 8" is expected to be thinner and lighter than even the iPhone 7. This could allow for less user fatigue, whether installed in a headset, or held by the user for an AR environment overlay.
Apple as iterator, not innovator
First off the starting blocks doesn't guarantee victory -- just ask Eiger Labs or Creative Media how their corporate fortunes are doing in the wake of the iPod, how Microsoft's tablet initiative fared after a 2002 debut, or how Samsung's smart watches are selling.
Apple wasn't the first to the computer mouse, the digital media player, an internet-connected phone, or the tablet form factor. It waited until the technology and time was right, perfected each form factor -- making it truly "friction free" to adopt.
And, like most of the rest of the developments its known for, Apple isn't going to be first to AR or VR. Google Glass rose and died in less than two years, and is beyond a niche in the company's Mountain View headquarters, if it isn't completely dead -- all because the product didn't work well.
Virtual reality is tied up in litigation between Zenimax and Facebook, after the latter made a multi-billion dollar bet on the technology.
Apple stands at a point with the "iPhone 8," controlling the current specs and future development of a near-ubiquitous device sold in the millions per quarter, where it can call the shots in AR for now, and the next decade at least.
Apple CEO Tim Cook has made an assortment of remarks, heralding augmented reality and virtual reality as potential cornerstones of the company's future. Combined with rumors of a new dual-lens forward facing camera system and advanced facial recognition software in the "iPhone 8," it's possible that Apple's true interest in the space could be revealed later this year.
While virtual reality promises users a more immersive experience, by sinking a user in the environment with video and audio cues, augmented reality instead offers a hybrid and more social approach, merging a virtual world with the one around us.
Augmented reality is a clear interest for Apple. Rumors about the hardware in the "iPhone 8" coupled with Apple CEO Tim Cook's remarks about the technology suggest that the company is going to dive deep into augmented reality very soon.
Virtual reality
For about three decades, the term "virtual reality" has been used as a catch-all for any sort of real-world simulation or modification. Initial implementations were used by the entertainment industry for relatively compact rides in amusement parks, with additional user by the military for flight simulator training.Early implementations employed conventional displays in conjunction with mechanical haptics to provide full feedback of the environment to the user.
Because VR is an immersive and largely solitary experience that requires a headset, it's an unlikely fit for Apple. Instead, the company is likely to embrace an offshoot of VR, augmented reality, in a way that can allow users to interact with the world --?and others --?around them, via the iPhone.
Augmented reality
AR is less hardware- and software-intensive than VR, using the combination of a camera and display such as those found in the iPhone to manipulate and generate overlays over the surrounding real-life environment. Augmented reality can be used to guide a user on a street, or highlight businesses as a user travels through a town.Another possible use case is capture of virtual creatures with the flick of a finger, after spotting them in the park on a daily walk such as in Niantic's "Pokemon Go."
Done properly, augmented reality could be readily adopted by the iPhone crowd -- assuming it can be made "friction-free." As it turns out, that's an area where Apple historically has excelled.
Tim Cook on augmented and virtual reality
"The smartphone is for everyone, we don't have to think the iPhone is about a certain demographic, or country or vertical market: it's for everyone. I think AR is that big, it's huge," Cook said in an interview in February. "I get excited because of the things that could be done that could improve a lot of lives. And be entertaining."
"I view AR like I view the silicon here in my iPhone, it's not a product per se, it's a core technology. But there are things to discover before that technology is good enough for the mainstream," Cook concluded. "I do think there can be a lot of things that really help people out in daily life, real-life things, that's why I get so excited about it."
"I think AR is that big, it's huge" - Tim Cook
Apple's tangible moves in the AR and VR sector
Apple has made a number of strategic hires and acquisitions over the last five years to further its AR ambitions.
Motion capture specialist Faceshiftwas acquired in 2015, with machine learning and computer vision startup Perceptio grabbed earlier that year.
German AR firm Metaio and Flyby Media also go hand-in-hand with in-house development of transparent displays, iPhone-powered VR rigs, AR maps and other related technologies described in recent patent filings.
In January, Apple was reassigned IP from Metaio for an AR device with advanced point of interest labelling. Specifically, a pair of patents detail a mobile AR system -- or smartphone -- capable of detecting its surroundings and displaying generated virtual information to users in real time.
The "iPhone 8" and augmented reality
While the "iPhone 8" may not be sold to customers on AR prospects alone, it appears to be setting the table for wider adoption of the technology.
According to recent rumors, the sensor expected to be used in this year's "iPhone 8" is intended for a practical face scanning utility, and not solely or specifically for augmented reality application.
However, the technology described for the sensor appears to be a micro-miniaturized version of a LIDAR mapper or rangefinder.
LIDAR is a proven technology, with resolution and accuracy dependent on a combination of factors, not the least of which is the integration of the laser and the sensor -- which Apple's acquisition of Primesense in 2013 appears intended for.
To properly overly the environment requires accuracy -- which a rear-mounted LIDAR installation supplementing existing range finding equipment associated with the iPhone camera would give. It also requires heavy-duty processing power, which the existing A10 Fusion processor also has -- and any successor would as well.
According to analyst Ming-Chi Kuo of KGI Securities, the new camera system expected to arrive in Apple's "iPhone 8" will enable 3D sensing and modeling in applications, which could be used for a range of capabilities including replacing the head of a character in a 3D game, or taking a 3D selfie.
Beyond proprietary camera software, Apple has other AR- and VR-related edges over its competition as well. For example, the iPhone only has a few resolutions, where Android devices have literally hundreds.
The "iPhone 8" is expected to be thinner and lighter than even the iPhone 7. This could allow for less user fatigue, whether installed in a headset, or held by the user for an AR environment overlay.
Apple as iterator, not innovator
First off the starting blocks doesn't guarantee victory -- just ask Eiger Labs or Creative Media how their corporate fortunes are doing in the wake of the iPod, how Microsoft's tablet initiative fared after a 2002 debut, or how Samsung's smart watches are selling.
Apple wasn't the first to the computer mouse, the digital media player, an internet-connected phone, or the tablet form factor. It waited until the technology and time was right, perfected each form factor -- making it truly "friction free" to adopt.
And, like most of the rest of the developments its known for, Apple isn't going to be first to AR or VR. Google Glass rose and died in less than two years, and is beyond a niche in the company's Mountain View headquarters, if it isn't completely dead -- all because the product didn't work well.
Virtual reality is tied up in litigation between Zenimax and Facebook, after the latter made a multi-billion dollar bet on the technology.
Apple stands at a point with the "iPhone 8," controlling the current specs and future development of a near-ubiquitous device sold in the millions per quarter, where it can call the shots in AR for now, and the next decade at least.
Comments
I hope not.
No company should ever expect to repeatedly win on "groundbreaking". There just isn't that much groundbreaking to do. All the technology we have now is in desperate need of maturation and even just continued work to make it actually useful, instead of sometimes useable.
People are way too forgiving of broken technology. Mostly because they can't tell the difference between user error and bad design. That's the religion of the computer industry: blame the user.
Then the geeks come along and say "a good craftsman doesn't blame his tools". Some of these people wouldn't know a quality tool or artisanal craft if it punched them in the crotch. They're too busy marveling at gimmicks and gadgets, complaining about a lack of change for the sake of change, and possibly relying on the need to "fix" users' problems for their income (while mocking their customers as being ignorant or stupid because of not having the same specialized knowledge as geeks).
I loathe this industry. It's sad that Apple was the model of doing it right for a stretch of years between 2007 and 2013, because now they're the model of doing it the same as every other soulless and visionless Wall Street corporation. No visionary is likely to save Apple a second time around, once the Wall Street irrationality turns firmly against them and iPhones are considered "just another product that other companies do better AND cheaper".
With Google Glass the viewers entired field of vision is available for augmentation. Now imagine an alternative system where only the portion of the entired field of vision is available for Augmented Reality ... the portion blocked out by the AR iPad or AR iPhone. This would not freak anybody out.
So that is why I disagree with Tjwolf.
Now the article says ...
"To properly overly the environment requires accuracy —which a rear-mounted LIDAR installation supplementing existing range finding equipment associated with the iPhone camera would give. It also requires heavy-duty processing power, which the existing A10 Fusion processor also has —and any successor would as well."
I think a LIDAR installed on the FRONT of the AR devive is necessary ... NOT a rear-mounted LIDAR.
On my last three posts I have been going on about this. I reproduce two of them below.
============
I think the new size iPad will introduce Apple's new Augmented Reality functionality. This functionality has been heralded already by Tim Cook.
Now the way I think the Augmented Reality functionality will work is to have facial tracking capability (LIDAR) on the front of the device. This will allow the machine to display a "seamless" version of reality on the screen (actually the bezel will be the "seam", that is why they are reducing it as much as possible) as it can work out exactly where the viewer's eyes are all the time.
===========
The way I think the Augmented Reality functionality will work is to have facial tracking capability on the front of the device. This will allow the machine to display a "seamless" version of reality on the screen (actually the bezel will be the "seam", that is why they are reducing it as much as possible) as it can work out exactly where the viewer's eyes are all the time.
I think the iPhone or iPad in AR mode will be more or less invisible ... you can only see the bezel. Of course an "invisible" iPhone is not much use. But quite simple to superimpose ... text, for example ... on the screen. This would enable people to check their iMessages while walking for example ... obviously not the main point of this functionality, just an unintended consequency (which I don't approve of 100% ... by the way).
A more difficult thing to do would be to have text on the screen giving information about some building/car/breakfast-serial/person visible on the screen. This would be full Augmented Reality. It would have similar possibilities to Google Glass. Only the part of your field of view covered by the screen will be available for AR though. Other people will not be freaked out* by it as they were by Google Glass.
============
Now, how many of you cover the front-facing camera on your smartphone? Oooh, I see far fewer hands. Why is that??
Is it because you have more control over your phone? No, it couldn't be that, it's quite the opposite.
Is it because you use it more often? Maybe. But that still doesn't explain the difference in attitude.
I keep my phone's front-facing camera covered with a small piece of black electrical tape. It blends in so well that most people don't even know it's there. Why do I mention this? Because WTF are we to do if Apple actually does embed the camera under the fucking screen?! For me personally, it will mean that the newest iPhone I'll ever use will be a 7 or 7s, and I'd really rather not be stuck there forever.
I really wish people would take a minute to consider why they cover the camera on their laptops, but not their phones. Whether you cover or not, I see LOTS of people at coffee shops, libraries and other work hangouts who do.
Unfortunately you are correct that what apple chooses to invest time in has negative consequences to their neglected products. I don't think that this results in bad Apple products just neglected or abandoned products like the Airport Expess. So Apple will still continue to extremely good products and will still have phenomenal financial results. I actually don't think that Google as a company can do Hardware and Retail anywhere as good as Apple for Android to really have better products. Apple is atleast 10 or 20 years ahead of Google with respect to product creation, marketing and support. Despite being well reviewed Google hasn't even managed to get their Pixel phone out to the public. I see many interesting products in the Google world like the Pixel C tablet and keyboard but not the commitment on the part of the company that even approaches Apple's some times aggravating neglect.. "other companies do cheaper" , but not better and cheaper. That's why android wear (android smart watches) have fallen behind the Apple watch- sometime you just can't make a product cheap and truly make it good - that's why the Pebble is dead.
Now a word of warning: the Echo and Alexa is good and cheap. Hopefully Apple isn't ignoring her flank by not coming out with a competing home HUB product with a SIRI that actually works (but don't expect it to be cheap. Apple if they do this need to go expensive and make a product that takes on SONOS at the same time). Unfortunately I'm not holding my breath on this because a great SIRI product is all FUNCTION not FORM and this is why its been neglected by COOK and IVE.
* while it's all nonsense I flag this one because it's the exact same thing people said about Apple and computers decades ago -- yet here they are, 40 years later, the only still-here PC maker, and sucking up all the profit to boot. Oops.
A lot of people evidently believed in witches at one time, too. Doesn't make it any less bullshit. Same thing with taping your laptop or cell phone.
lol, what a sad and predictable response. Laptop cameras have been proven to be hackable, hacked, have been used for nefarious purposes, and obviously only a small fraction of what happens is publicly known. But because you don't care, that somehow makes people who do care a problem? What an ignoramus. You probably think Snowden exposed mythical boogie men as well.
I spend a lot of time working in public spaces; cafés, coffee shops, libraries, and various locations around my city with public wifi every week. An interesting note about my city is that in the more affluent/educated areas, especially where I see smart people, devs, execs, etc., I see more cameras covered. When I venture into poorer, less educated areas, I see less of this. I wont make claims about other cities, so others may or may not see the same thing, but I clearly see it here. I've also noticed the overall amount of covered cameras has steadily (though slowly) increased over time. I guess according to you, people must be getting "stupider". Good luck with that mindset.
It's also amusing that you found it worth your effort to make a smart ass comment about something that it would seem you don't care about. I mean it's easy to understand why someone might be passionate about their privacy, far less easy to understand why someone would be passionate about removing other people's privacy. Makes one wonder...