What makes a good AR or VR headset and why Apple is positioned to dominate the space

Posted:
in General Discussion edited May 2018
If Apple is working on a VR or AR headset, it is certainly aware of the different technologies and concepts already used in the market, including the benefits and pitfalls that can be encountered. AppleInsider provides an overview of the current state of the two technologies, and what Apple needs to consider for creating its own headset design.

oculus rift vr headset


Despite being a relatively small and young industry, the market for virtual reality and augmented reality-based devices already has a wide variety of different designs to choose from. As the price changes, so does the hardware, and the functionality, with more expensive headsets generally offering a higher-quality experience.

Rumors and speculation from analysts point to Apple entering the VR or AR marketplace in the next few years, but aside from some patent filings, it is unknown what exactly to expect from the iPhone producer's fabled hardware. As a premium device producer in multiple markets, looking at the better and more popular headsets on the market could provide some indication of how Apple's version could turn out.

In creating the headset, Apple faces the task of assembling together a wide array of different components and usage concepts, with each having their own positive and negative points.

Display

Arguably one of the most important elements of AR and VR headsets, the screen or screens are stared at by users for minutes or hours at a time. A good-quality display is an essential component, typically one with a high resolution that minimizes the chance of the user noticing individual pixels, breaking the VR effect.

The refresh rate also needs to be high enough so that the screen can update without the user noticing any jumping between frames. A fast refresh rate and minimal lag between the user moving and it changing on the screen is also essential, in part to keep the illusion intact, as well as to minimize the chance of motion sickness for some individuals.

VR headset lens view


For the refresh rate, some early headsets relied on two separate screens to show each eye an individual image, but this was an expensive concept to use. Modern VR headsets like the Oculus Rift instead opt for a single larger screen and lenses for each eye, effectively dividing the screen in half.

The single display route saves money when compared to creating two smaller high-resolution screens, but technology has advanced enough that it makes the dual-screen headset viable again, such as with the Magic Leap One Lightwear AR headset. A dual-screen approach could also result in faster refresh rates, due to updating two smaller screens simultaneously instead of the longer process for one larger display panel.

The single panel approach also led to the creation of cheap headsets that use high-resolution display many people already have: their smartphone. Google Cardboard, Samsung's Gear VR, and numerous others use the concept, and even Apple has seemingly caught on in some patent filings, showing glasses with space to insert an iPhone in front of the user's eyes.

Google Cardboard


It is worth touching on two other display pitfalls that have become less of a problem now, but were noticeable issues in earlier modern headsets.

The "Screen Door" effect was a problem caused by manufacturing the display with pixels that were slightly too far apart, causing a dark grid created by the unlit space between lit pixels to form. These screens looked acceptable for use in other devices, but the close-up nature of VR headsets meant the gaps were more noticeable, giving the effect of looking through the mesh of a screen door.

A close-up shot of a monitor, showing the grid of the
A close-up shot of a monitor, showing the grid of the "Screen Door Effect"


Mura is an issue where pixels have issues with color accuracy, in that neighboring pixels displaying the same color could show different shades, or even the same shade at slightly different brightnesses. This can sometimes give a screen door-like appearance especially when large blocks of the same color are displayed.

An example of
An example of "Mura," where all pixels in a section of a display are set to show the same color, but vary due to the way it was manufactured.


Generally, these two issues have been solved by VR headset producers in their latest releases, but older and cheaper designs will still suffer from these effects.

AR Display Challenges

The main difference between VR and AR hardware is primarily how it provides users with the visual image. While sensors and other areas could largely be the same as VR, the different ways an AR picture can be displayed means that an entirely different headset could be used compared to a VR version.

One way to show an AR scene is to effectively use a VR headset with a live video feed from onboard cameras. The video is sent to the host computer, which then overlays the AR element and sends the composite image back to the headset for viewing.

This system is straightforward and can give what could be the optimal digital-item-in-real-world effect, but with two drawbacks. First, there's the problems associated with VR headsets in general, including lag that can cause nausea, limitations to movement and range, and so on.

There's also the user's expectation of wanting to see the digital extras added to their view of the real world, a live view unimpeded by screens. People think things along the lines of how smart glasses are used in TV shows and movies, namely Google Glass but better.

Microsoft Hololens


While that's desired, such systems do not currently exist in that form that can provide an augmented reality image. AR headsets do exist and can be bought, such as the Microsoft HoloLens, but are a lot bulkier than the movie magic pretends is possible.

The mixed-reality HoloLens uses a transparent panel that users look through to the outside world, underneath a large headband containing all the sensors and computing power the HoloLens needs to map the environment and the user's movements. A projection system then overlays the image onto these panels, superimposing the view over the world.

As the position of the eyes isn't uniform across all users, AR headsets will also need to figure out where the eyes are located, usually with an eye-tracking system. In some implementations, this can mean beaming light into the eyes and monitoring any reflections of light bouncing off the retina.

By knowing where the eyes are, and where the user is looking, this can help line up the virtual object with the live real-world view more accurately.

Sensors and Motion

In order for a headset to be effective, it needs to have information about the world around it. To correctly produce the scene viewed by the user, the host computing device requires data on movements, both for the current image and in anticipation for the next.

At a base level, this consists of using motion sensors, including gyroscopes and accelerometers, to analyze the head's movement with six degrees of freedom. Accelerometers can help detect how far the head is moving in various directions, while gyroscopes are used to measure any tilting movements of the headset.

Depending on the exact setup required to operate the headset, there may also be the need for cameras or other imaging sensors placed externally. This provides a few things for the headset, such as giving users the ability to see the real world around them without taking the headset off in some apps, as demonstrated with the HTC Vive's Chaperone system.

HTC Vive Pro


For augmented reality, these cameras would provide the real-world views that either become the backdrop for the virtual objects to appear on, or can be used to compute where objects can be placed. If multiple cameras are used on the front, this can provide a stereoscopic view of the world and an improved AR effect.

Some VR headsets, especially room-scale versions, can use emitters at strategic locations to provide the headset even more data about its relative position within the space. For example, the "lighthouse" method used by SteamVR requires at least two stations in opposite corners of the space, typically high up so they are viewable everywhere in the room without being obscured, which are picked up by imaging sensors located on the headset.

As with the display, the sensors also need to operate accurately and at speed, both in acquiring data and in sending it to the host for processing. The lag in producing and transmitting the data adds to the overall delay in the VR scene responding to the user's movements, and as already explained, too much of a delay could break the VR illusion or cause motion sickness. Minimal lag is mandated here.

In the cases of Gear VR, Cardboard, and some others that are based around smartphones, the existing onboard sensors are the ones used for VR and AR. There is minimal lag caused through transmitting data between components, leaving only processing power the only possible cause for slow response times.

For AR that uses translucent screens, the external cameras will be vital for computational purposes more than giving the user a decent view. The computer would need to know about the environment, including flat surfaces and empty space in the air that it could feasibly place objects, and to discover points of reference so it can keep said digital items anchored in place.

Sensors for eye tracking would also be extremely useful for AR, not only for determining placement for transparent or translucent displays, but also for applying depth of field and other effects on the digital object, depending on what the user is focusing on.

Scale

Generally speaking, there are two levels of VR scale that needs to be considered when producing a VR-capable headset.

The first is room-scale, which relies upon beacons in an environment to help the headset understand its location. For home use, this uses the aforementioned "lighthouse" system in most cases, with two beacons in opposite corners of the space. This can give users enough space to walk around an area, albeit limited to within the limits of the room.

For home users, the range of movement is also restricted by the length of cables running from the VR headset to a host computer, providing video, audio, and data transfers. This may be less of a problem in the future, as manufacturers are working on ways to wirelessly transmit video and data between the headset and host.

This concept can also be expanded to larger spaces in entertainment venues, as with Zero Latency's system, where the walls and ceilings of a large space are covered with beacons. For this, groups of users can wear computers in a backpack along with the headset, allowing for tether-free roaming of an environment.

Zero Latency's large-scale VR system
Zero Latency's large-scale VR system


VR also works in a much smaller scale, such as Google Cardboard, Samsung's Gear VR, and the Oculus Go, which are designed to be used with as little available space as possible. As they do not need to analyze the rest of the world, instead relying on measuring movement, these can easily be used from a chair, but do not offer the ability to walk around a room.

As AR headsets rely so much on monitoring the outside world without needing to use externally-mounted sensors, they could feasibly be used anywhere. In the case of HoloLens-style hardware, there's the possibility of walking around a large environment, as the user's sight isn't impeded by a display.

HoloLens group work AR


The potential for collaborative work also means that scale plays a major part in how group sessions can be conducted, both in VR and AR. Aside from requiring more physical space due to more bodies being involved, it also springs up the issue of the headsets needing to take into account each other's positions.

For tethered lighthouse-based systems, each headset would need to determine its location in space and provide that to the host, which then shares the data with the host computers of other users, while at the same time acquiring up-to-date information on others. This is essential data, as otherwise the users will be blind to the other's positioning, and could physically contact each other.

In the HoloLens-style AR where the real-world view is constantly available, this data is not as urgent as there is no vision-based safety issues, but data does still need to be shared for interactions with virtual objects within apps, for example. Data acquired from sensors regarding the local environment could feasibly be shared with others, depending on the application, but ultimately little data would need to be shared with other participants.

Audio

In many VR and AR applications, it's desirable for the user to hear things. Using built-in headphones, a speaker, or even including some way to connect third-party headphones to the headset are all acceptable ways of providing this, but there are still some things that need to be considered.

Enclosing the user's ear will help immerse them in a VR world, such as gaming, by minimizing distractions. There's also the option of using surround sound headphones with multiple speakers for each ear, though this could easily be simulated in software using standard stereo headphones.

HTC Vive Audio Strap


At the same time, opting for a system that still allows the user to hear the nearby environment would be preferable, both from a collaborative viewpoint for work, and from safety. For AR headsets that may be used in a wide variety of locations, hearing the surrounding area would most likely be essential, and if it's to be used for brief periods, could arguably be worth leaving off the device entirely.

Given the variety of different situations, it is probable that headset producers will err towards using a basic on-device audio solution, but still allowing users to plug in their own if needed.

Comfort

While not a technological issue, the ability to wear a headset for a long period of time can be a major factor in a purchasing decision.

The first question is whether the headset is designed to be worn or held. A headset like the Oculus Rift is meant to be held on the head, allowing the hands to be freed up for other peripherals and accessories, but if it is too heavy, it could press on parts of the user's face or add strain to their neck.

The handheld version would be more appropriate for group or brief usage periods, making it ideal for AR applications around a city when on vacation. At the same time, these systems would tie up one or both of the user's hands while in use, limiting what the user could accomplish in the virtual environment.

Magic Leap Creator One AR Headset


Another weight-related concern is where the host computer resides. For handheld VR and AR, this could be a smartphone slotted into the headset itself, as with Google Cardboard and Samsung's Gear VR, or connecting to a waist-worn unit like the Magic Leap Creator One.

There is also the option of connecting the headset to a host by long cables, a system currently used by the HTC Vive and Oculus Rift, which offloads processing to a nearby computer. This does prevent any extra weight from being added directly to the headset, but at the same time it introduces cables that could pull on the headset while it's being worn, or could get in the way underfoot while in use.

Usability

In an ideal world, a VR or AR headset would require as little in the way of tuition or setup to get going. In our current reality, we're not quite at that point, but we're getting there.

Starting off with room-scale VR setups, there's the need to connect the headset to a host by cables, the placement of the "lighthouses," and the setup of the equipment, which could put off less determined users.

Handheld smartphone-based headsets or viewers are easier, as it could be as simple as sliding the smartphone into place and running an app. An even easier prospect is the Oculus Go, as it is self-contained and does not require a host to operate, doesn't use lighthouses or similar external elements that need to be configured, and is supplied with a simple controller.

Oculus Go


Of course, setting up the hardware is only part of the equation. The setup experience on the software side is another matter entirely, and then there's the applications themselves.

Depending on the complexity of each type of VR or AR headset, there will be different requirements that need to be met by app developers to use the hardware. At the same time, they will also need to tailor the user experience of the app depending on the available resources, including how the user interacts with the virtual space or digital AR items, and their range of motion.

Given the variety of both hardware and software in use today, there is no single way to get started in VR or AR, from either a hardware or software side.


But, where does Apple come in?

All this being said, not every user wants to throw tons of cash at a device when they can get a good experience at a cheaper price. VR enthusiasts will want room-scale setups and the equipment to do practically anything on a digital landscape, but someone's aunt or uncle who wants to see what this VR thing is all about could probably be satisfied with an iPhone shoved into Google Cardboard for ten minutes.

There's other considerations to take into account as well, like a lack of available space at home for room-scale VR. In cases like these, items like the Oculus Go may be a better option.

Headset-based AR is available, but is nowhere near as mature a technology as VR. Apple and third parties have taken large strides towards AR becoming a popular feature of apps, ever since the introduction of ARKit, but outside of Microsoft's HoloLens and its related initiatives, the hardware is nowhere near AR in terms of wide commercial viability.

Is Apple working on a VR or AR headset? It's likely, as patent applications and Apple safety reports do indicate that related technologies are being investigated, but this is by no means a sign that a hardware launch is on the horizon.

What route is Apple likely to follow? Given Apple's nature of creating mass-consumer products, the current room-scale system with a separate host is probably out of the question. An easier to use headset closer to the Oculus Go is more likely, reducing the barrier for entry down to a financial transaction rather than technical knowledge.

Apple also has the benefit of extensive integration knowledge and its control over both the software for the device, the firmware, and the entire manufacturing pipeline. The company's ability to make high performance iPhones despite having nominally lower baseline specifications than its rivals in some areas, in part due to this control, would mean any standalone VR or AR headset would be engineered to an unparalleled level.

Apple AR glasses patent image


In October 2017, Apple CEO Tim Cook noted that AR was an important, but still gestating, technology that could have "exponential" growth, but declined to provide details about anything Apple may be working on. In the interview, he advised "I can tell you that the technology itself doesn't exist to do that in a quality way. The display technology required, as well as putting enough stuff around your face - there's huge challenges with that."

Cook also noted Apple wants to be the best, rather than first to market, and give a great experience to their customers. "But now anything you would see on the market anytime soon would not be something any of us would be satisfied with," suggested Cook. "Nor do I think the vast majority of people would be satisfied."

Given Apple's history of focusing on the user experience, it could be a long wait before consumers get to try out an Apple-produced AR headset. Current estimates from Loup Ventures analyst and Apple observer Gene Munster suggests such hardware could arrive by 2021.

In the meantime, anyone wanting to have an Apple-driven AR experience will have to settle with apps running ARKit on their iPhone or iPad.
fotoformat
«1

Comments

  • Reply 1 of 30
    saltyzipsaltyzip Posts: 187member
    I think VR headsets will go the way of 3D on Television, into room 101, extinct in other words.
    edited May 2018 racerhomie3aylk
  • Reply 2 of 30
    Robots78Robots78 Posts: 15member
    Room-scale VR is already moving away from Lighthouse-style external sensors and towards HoloLens-style "inside-out" systems with the sensors in the headsets. Microsoft's "Mixed Reality" standard uses this technique. I'm really looking forward to more advancements in the AR space. I've spent some decent time with a HoloLens and it's a very exciting glimpse at what the future holds. The included games, in particular, are incredibly cool.
    edited May 2018
  • Reply 3 of 30
    KITAKITA Posts: 153member
    Robots78 said:
    Room-scale VR is already moving away from Lighthouse-style external sensors and towards HoloLens-style "inside-out" systems with the sensors in the headsets. Microsoft's "Mixed Reality" standard uses this technique. I'm really looking forward to more advancements in the AR space. I've spent some decent time with a HoloLens and it's a very exciting glimpse at what the future holds. The included games, in particular, are incredibly cool.
    Some of the HoloLens applications are very impressive.




    edited May 2018
  • Reply 4 of 30
    gatorguygatorguy Posts: 19,701member
    KITA said:
    Robots78 said:
    Room-scale VR is already moving away from Lighthouse-style external sensors and towards HoloLens-style "inside-out" systems with the sensors in the headsets. Microsoft's "Mixed Reality" standard uses this technique. I'm really looking forward to more advancements in the AR space. I've spent some decent time with a HoloLens and it's a very exciting glimpse at what the future holds. The included games, in particular, are incredibly cool.
    Some of the HoloLens applications are very impressive.
    https://www.wareable.com/vr/google-lg-super-high-res-vr-headset-201
    edited May 2018
  • Reply 5 of 30
    maestro64maestro64 Posts: 4,371member
    I am sorry these headset are geek products and there is no mass consumer appeal for something that sits on your head like these. Until people are not so worried about how they look a product like this will not have any use other than gaming or an application like the one in the video above.
    edited May 2018 SpamSandwichaylk
  • Reply 6 of 30
    SpamSandwichSpamSandwich Posts: 30,417member
    KITA said:
    Robots78 said:
    Room-scale VR is already moving away from Lighthouse-style external sensors and towards HoloLens-style "inside-out" systems with the sensors in the headsets. Microsoft's "Mixed Reality" standard uses this technique. I'm really looking forward to more advancements in the AR space. I've spent some decent time with a HoloLens and it's a very exciting glimpse at what the future holds. The included games, in particular, are incredibly cool.
    Some of the HoloLens applications are very impressive.




    Demos like this illustrate that future workers will either be dummies who have no clue what they are doing and must be guided like sheep through every step of a process, or robots will soon replace manual laborers en masse. I think both scenarios will come to pass. 
  • Reply 7 of 30
    fastasleepfastasleep Posts: 2,102member
    saltyzip said:
    I think VR headsets will go the way of 3D on Television, into room 101, extinct in other words.
    $10 says you haven’t used VR before. 
    StrangeDayslolliverwatto_cobra
  • Reply 8 of 30
    claire1claire1 Posts: 484unconfirmed, member
    I cringe when people suggest the headset NEEDS to have an iPhone strapped to your face.

    Apple isn't into that Nerdy Google/MS stuff. They will make sleek glasses with no external hardware input if they enter the space.
    It won't be huge hardware like current models that makes us look like aliens.



    NOPE



    No cyborg fashion.


    Something like this would be the goal:


    SpamSandwichradarthekat
  • Reply 9 of 30
    apricot88apricot88 Posts: 11member
    Apple would make it part of their wearables product offering. 
    watto_cobra
  • Reply 10 of 30
    FolioFolio Posts: 422member
    This might be a case, despite Tim Cook oldish quote above, where Apple could leverage its growing enterprise presence. Wearing Apple geeky AR glasses on the job (where fashion and privacy less an issue) as Apple perfects a consumer launch a few years later. In short functionality will come way before fashion. And methinks people more picky with something you wear on your face, rather than say on a wrist.
  • Reply 11 of 30
    radarthekatradarthekat Posts: 2,898moderator
    KITA said:
    Robots78 said:
    Room-scale VR is already moving away from Lighthouse-style external sensors and towards HoloLens-style "inside-out" systems with the sensors in the headsets. Microsoft's "Mixed Reality" standard uses this technique. I'm really looking forward to more advancements in the AR space. I've spent some decent time with a HoloLens and it's a very exciting glimpse at what the future holds. The included games, in particular, are incredibly cool.
    Some of the HoloLens applications are very impressive.




    So very Microsoft.  If Apple were to have implemented that, in the first video Siri would say, “see that highlighted valve?  Turn it clockwise.” And a visual overlay would show that action in AR.  There’d be no mention of “work item” or some long obscure part number highlighting the pipe.  It’d be like a veteran expert guiding you through the process, explaining in layman’s terms the details and needed insights.  SMH
    edited May 2018 watto_cobra
  • Reply 12 of 30
    radarthekatradarthekat Posts: 2,898moderator
    What I’d like to see come first, from Apple, is a Frank Herbert Dune Mentat capability.  Just a voice in my
    ear.  Okay, fast forward for those that never read the Dune trilogy.  Jarvis, from Ironman, without all the silly head’s up displays.  Just mount a camera somewhere in a wearable and use it to fill in information about my environment that my phone’s GPS can’t provide. So I’m in France, for example, and I say out loud, what’s that sign say, and Siri tells me, using the wearable camera, the iPhone processing and connectivity (or that on my Watch), and my Airpods.  I might not even want a direct reading for the sign, just give me the synopsis along with anything not on the sign I might want to know, like what a French travel companion would give me, in conversational style, “That’s the Musée d'Orsay.  Tickets are about $19.”
  • Reply 13 of 30
    foggyhillfoggyhill Posts: 4,767member
    claire1 said:
    I cringe when people suggest the headset NEEDS to have an iPhone strapped to your face.

    Apple isn't into that Nerdy Google/MS stuff. They will make sleek glasses with no external hardware input if they enter the space.
    It won't be huge hardware like current models that makes us look like aliens.



    NOPE



    No cyborg fashion.


    Something like this would be the goal:


    The problem right now, is even the dork "fashions" are woefull in performance. To get to looks great and has great performance will take a huge R&D effort and few will have the money and tech to go there.  Apple and maybe Samsung are only two I see that could maybe do something like that within 5-8 years (look like this photo with great perf). I'd say 8 years is probably closer than 5. But, it will happen and it will change everything. There will of course be a small airpod in your ears too talking to you; we will be augmented. The watch will still exist, but mostly to work in concert with the glasses. Everyone will run their own local mesh network of wearables talking to each other.
    watto_cobrafastasleep
  • Reply 14 of 30
    SpamSandwichSpamSandwich Posts: 30,417member
    What I’d like to see come first, from Apple, is a Frank Herbert Dune Mentat capability.  Just a voice in my
    ear.  Okay, fast forward for those that never read the Dune trilogy.  Jarvis, from Ironman, without all the silly head’s up displays.  Just mount a camera somewhere in a wearable and use it to fill in information about my environment that my phone’s GPS can’t provide. So I’m in France, for example, and I say out loud, what’s that sign say, and Siri tells me, using the wearable camera, the iPhone processing and connectivity (or that on my Watch), and my Airpods.  I might not even want a direct reading for the sign, just give me the synopsis along with anything not on the sign I might want to know, like what a French travel companion would give me, in conversational style, “That’s the Musée d'Orsay.  Tickets are about $19.”
    I know what you’re saying, but Mentats were human computers in Dune, not humans enhanced with computers. Remember, the Butlerian Jihad resulted in the elimination of computers made to function like the human mind (artificial intelligence).
  • Reply 15 of 30
    mattspacemattspace Posts: 11member
    AppleInsider provides an overview of the current state of the two technologies, and what Apple needs to consider for creating its own headset design. 
    As someone who is now doing a lot of actual content creation work inside VR - as in VR is now the work environment where I get a significant proportion of what I do, done, here's some observations:

    • AR is an open plan communal workspace, with all of the associated distractions, and the need to build a visually clean work environment so you can concentrate on what you're doing. VR is having your own office, which is always clean and tidy and specific to your current needs.
    • Anyone complaining that headsets are "too" bulky, needs to get out of the office and look at the basic safety gear any other industry requires. A VR headset is nothing compared to the gauntlets, welding helmet, leather protective jacket, angle-grinding shield etc I wear whenever I step into my analog studio. Not all computing technology revolves around people in Jony Ive's antiseptic white world.
    • Apple does not have a single computer that is "good" for VR, because...
    • AMD does not have a single GPU that meets the minimum spec for a machine you'd want to work in all day.
    • Nvidia's 1080ti is the entrypoint for a serious VR station, and the top range AMD Vega 64 comes in around 30% below it in terms of powering 3D gaming engines at high resolution (which is what a VR environment is).
    • Metal and Metal-optimisation will not make weaker hardware on the Mac outperform stronger hardware on Windows PCs. The Windows 3d ecosystem is not bogged down by inefficiency that Metal can "cure".
    • Go look up how many non-game VR utility apps there are for macOS on Steam - most of the ones that were a part of the iMac Pro launch, still haven't been released. A year after Apple announced VR at WWDC 2017, there's a grand total of 4 non-game apps, as opposed to 192 for Windows.
    • VR app makers are not going to flock to macOS in the absence of affordable, high performance GPU hardware that can be kept updated with annual upgrades. There's no ecosystem for high-performance 3D on the Mac.
    • People looking for a VR machine, are not going to buy an eGPU solution that costs more, while only having 1/4 the PCI bandwidth of a machine with a motherboard PCI slot.
    • The state of the art in VR, is not going to stop progressing for at least a decade, and until then, when goggle resolution exceeds visual acuity, and 3D hardware is capable of maintaining enormous models without slowing down, quality will never be "enough".
    • In VR there's nothing of the OS for the user to interact with, every VR app is its own universe, providing all the UI elements, file navigation etc. Headset makers, and Steam, provide the launcher apps, so you don't even need the operating system to manage starting and stopping the apps. VR makes the operating system on the computer as irrelevant to the user, as the firmware in the bios. Gravity Sketch on Windows is the same as Gravity Sketch on macOS, it just performs better on Windows.
    • Tracking without lighthouses, no matter how good it is, is unlikely to be as good as lighthouse tracking, and VR's succeed / fail margin is in the final percentages of how good it is. Non lighthouse tracking is probably ok for AR, because AR is always about what you're looking at in your current field of vision. Lighthouses let you track objects that aren't in your field of view, and free you from spending any processing budget on recognising the ambient environment in order to track the wearer's position.


    Most importantly:

    • The experience of creating in VR is more compelling than any user experience of any device Apple makes. Next to Tiltbrush in a Vive, painting apps on the iPad Pro are banal and dull. VR apps are a new artistic & creative medium, capturing the ability to draw in space, which sculptors have been chasing since the dawn of time. The iPad Pro is, to be uncharitable, the ruination of the no-safety-net thrill of paper.

    I've been mac-based since 1994. I'd give up the Mac in a second if I had to choose between it, and a user-upgradable Nvidia-powered VR station. macOS being "better" or "nicer" simply isn't a part of the equation.

    edited May 2018 gatorguyRobots78fastasleep
  • Reply 16 of 30
    bluefire1bluefire1 Posts: 850member
    When it's perfected, Apple will release it.
    watto_cobra
  • Reply 17 of 30
    KITAKITA Posts: 153member
    KITA said:
    Robots78 said:
    Room-scale VR is already moving away from Lighthouse-style external sensors and towards HoloLens-style "inside-out" systems with the sensors in the headsets. Microsoft's "Mixed Reality" standard uses this technique. I'm really looking forward to more advancements in the AR space. I've spent some decent time with a HoloLens and it's a very exciting glimpse at what the future holds. The included games, in particular, are incredibly cool.
    Some of the HoloLens applications are very impressive.




    So very Microsoft.  If Apple were to have implemented that, in the first video Siri would say, “see that highlighted valve?  Turn it clockwise.” And a visual overlay would show that action in AR.  There’d be no mention of “work item” or some long obscure part number highlighting the pipe.  It’d be like a veteran expert guiding you through the process, explaining in layman’s terms the details and needed insights.  SMH
    That was made for HoloLens by another company, not Microsoft.

    As for the rest, well, let your imagination run wild I suppose.
  • Reply 18 of 30
    MacProMacPro Posts: 17,835member
    mattspace said:
    AppleInsider provides an overview of the current state of the two technologies, and what Apple needs to consider for creating its own headset design. 
    As someone who is now doing a lot of actual content creation work inside VR - as in VR is now the work environment where I get a significant proportion of what I do, done, here's some observations:

    • AR is an open plan communal workspace, with all of the associated distractions, and the need to build a visually clean work environment so you can concentrate on what you're doing. VR is having your own office, which is always clean and tidy and specific to your current needs.
    • Anyone complaining that headsets are "too" bulky, needs to get out of the office and look at the basic safety gear any other industry requires. A VR headset is nothing compared to the gauntlets, welding helmet, leather protective jacket, angle-grinding shield etc I wear whenever I step into my analog studio. Not all computing technology revolves around people in Jony Ive's antiseptic white world.
    • Apple does not have a single computer that is "good" for VR, because...
    • AMD does not have a single GPU that meets the minimum spec for a machine you'd want to work in all day.
    • Nvidia's 1080ti is the entrypoint for a serious VR station, and the top range AMD Vega 64 comes in around 30% below it in terms of powering 3D gaming engines at high resolution (which is what a VR environment is).
    • Metal and Metal-optimisation will not make weaker hardware on the Mac outperform stronger hardware on Windows PCs. The Windows 3d ecosystem is not bogged down by inefficiency that Metal can "cure".
    • Go look up how many non-game VR utility apps there are for macOS on Steam - most of the ones that were a part of the iMac Pro launch, still haven't been released. A year after Apple announced VR at WWDC 2017, there's a grand total of 4 non-game apps, as opposed to 192 for Windows.
    • VR app makers are not going to flock to macOS in the absence of affordable, high performance GPU hardware that can be kept updated with annual upgrades. There's no ecosystem for high-performance 3D on the Mac.
    • People looking for a VR machine, are not going to buy an eGPU solution that costs more, while only having 1/4 the PCI bandwidth of a machine with a motherboard PCI slot.
    • The state of the art in VR, is not going to stop progressing for at least a decade, and until then, when goggle resolution exceeds visual acuity, and 3D hardware is capable of maintaining enormous models without slowing down, quality will never be "enough".
    • In VR there's nothing of the OS for the user to interact with, every VR app is its own universe, providing all the UI elements, file navigation etc. Headset makers, and Steam, provide the launcher apps, so you don't even need the operating system to manage starting and stopping the apps. VR makes the operating system on the computer as irrelevant to the user, as the firmware in the bios. Gravity Sketch on Windows is the same as Gravity Sketch on macOS, it just performs better on Windows.
    • Tracking without lighthouses, no matter how good it is, is unlikely to be as good as lighthouse tracking, and VR's succeed / fail margin is in the final percentages of how good it is. Non lighthouse tracking is probably ok for AR, because AR is always about what you're looking at in your current field of vision. Lighthouses let you track objects that aren't in your field of view, and free you from spending any processing budget on recognising the ambient environment in order to track the wearer's position.


    Most importantly:

    • The experience of creating in VR is more compelling than any user experience of any device Apple makes. Next to Tiltbrush in a Vive, painting apps on the iPad Pro are banal and dull. VR apps are a new artistic & creative medium, capturing the ability to draw in space, which sculptors have been chasing since the dawn of time. The iPad Pro is, to be uncharitable, the ruination of the no-safety-net thrill of paper.

    I've been mac-based since 1994. I'd give up the Mac in a second if I had to choose between it, and a user-upgradable Nvidia-powered VR station. macOS being "better" or "nicer" simply isn't a part of the equation.

    Are you by any chance involved with Matterport?  Just curious.
  • Reply 19 of 30
    dasanman69dasanman69 Posts: 12,969member
    bluefire1 said:
    When it's perfected, Apple will release it.
    Nothing is ever perfected. Even the iPhone is improved on yearly. 
  • Reply 20 of 30
    maestro64maestro64 Posts: 4,371member
    foggyhill said:
    claire1 said:
    I cringe when people suggest the headset NEEDS to have an iPhone strapped to your face.

    Apple isn't into that Nerdy Google/MS stuff. They will make sleek glasses with no external hardware input if they enter the space.
    It won't be huge hardware like current models that makes us look like aliens.



    NOPE



    No cyborg fashion.


    Something like this would be the goal:


    The problem right now, is even the dork "fashions" are woefull in performance. To get to looks great and has great performance will take a huge R&D effort and few will have the money and tech to go there.  Apple and maybe Samsung are only two I see that could maybe do something like that within 5-8 years (look like this photo with great perf). I'd say 8 years is probably closer than 5. But, it will happen and it will change everything. There will of course be a small airpod in your ears too talking to you; we will be augmented. The watch will still exist, but mostly to work in concert with the glasses. Everyone will run their own local mesh network of wearables talking to each other.

    You have no ideal the time it takes to develop this kind of technology. Think about the horse power needed to make this work, then think about how small they can shrink the silicon to it does not look like one of the first two examples. We are not 5 yrs away, we are a long time away just on the technology front alone. This does not even take into consideration the human factor, simply people have very hard time dealing with content being overlaid in their field of view. You eyes can not easily focus near and far field if you do it have a side effect of headache. Google glasses failed due to people getting headache.  

    My thoughts on this is we will not see consumer products based around headset based VR/AR devices, however, products like this will most likely spawn product ideas no one is thinking of today. There is ideas like this which go through lots of iterations until someone figure it out and make a product which everyone agree is right. The Ipad is perfect example of this. tablets are not new, touch interfaces were not new, multi-touch as not new, and the list goes on, all the forerunners fail to deliver what people will like and use then Apple put it all together and nailed it.
    watto_cobrafastasleep
Sign In or Register to comment.