Video: A sneak peak at iOS 11 Augmented Reality at the Apple Park Visitor Center
At WWDC17, Apple introduced ARKit, iOS 11's new dev tools for Augmented Reality. It demonstrated interactive 3D games spilling off a table on stage and offered a hands-on look at a simple app that placed virtual objects on a table its iPads could recognize as a 3D plane. This week, it unveiled a practical new real-world app for exploring the Apple Park project.
What wasn't visible until Apple opened up its Visitor Center as a sneak peak reception after the event was the new Augmented Reality experience created for Apple Park, showing off the potential of ARKit in iOS 11.
When it opens to the public, the Visitor Center will primarily serve as an Apple Store, paired with a cafe. It also houses a vast, 11,000 pound model of the Apple Park site (below), which serves as an aluminum foil for the real star of the show: a virtual tour presented in AR using provided iPads. This new feature distinguishes the new site from the company's other retail locations.
Grand Central Terminal
Paris Opera
London Covent Garden
The new store features wonderfully crafted stone stairways (below) that convey the same sort of timeless craftsmanship felt in the stores integrated into historic gems such as New York City's Grand Central Terminal (above top), the former bank at Paris Opera (above middle) or London's spectacular Covent Garden site (above bottom).
Timeless stone stairways...
... and modern walls of glass
The new Apple Park store also feels incredibly modern, with double-high walls of optically-pure glass that--like the breathtaking entrance to the Steve Jobs Theater--act as structural components, not just windows.
Hong Kong IFC
San Francisco's Union Square
The Oculus at World Trade Center
Unlike the cool urban sophistication of glass and steel seen in Hong Kong's Hysan Place and IFC tower (above top), San Francisco's Union Square (above middle) or The Oculus at NYC's World Trade Center (above bottom), the new Apple Park store blends into its orchard-like surroundings by making use of earthy warm wood slats in its ceilings and across its rooftop deck, which offers a view of the vast Spaceship hiding behind a forest of mature trees just across the street.
The earthy, modern AR-enhanced model room.
A view of the Spaceship from the roof
You can flick on the rooftops of buildings to see inside them, where you can see the arrangements of internal walls, desks and even animated people walking around. Because the app is depicting the scene using AR, you can look up, across, inside and around the vast model to explore its mini-world of site details.
There's also a time-of-day slider that renders the AR graphics with lighting and shadows from early morning to high noon and into the evening, when the lights are switched on to get a feel of what it would look like to fly over the site at night.
At dusk there's a red sky, and if you look at the grand cafeteria in the early morning, you can see an animated depiction of its massive window-wall-doors opening up to expose the vast indoor space to the outside, taking advantage of the Bay Area's mild climate (similar to the huge doors of the company's recently opened new San Francisco Union Square store).
In addition to looking under the roof to see the inner workings of the Spaceship, you can also lift the lid on the Fitness Center (which has contrasting stone walls, rather than glass like the rest of the buildings on the site).
You can even lift up the historic barn that's sat on the site for decades--long before Hewlett Packard paved the original orchards to put up a sea of parking lots for its executive briefing center. When you lift the barn, all that's left behind are some bales of hay and what looks like it could be Clarus the dogcow.
You can also lift the lid of the Steve Jobs Theater, which exposes the grand round display area that leads into the theater pit below. Phase 2 buildings are included in the model, but are not yet interactive to look inside (or perhaps that's just to keep the magical mysteries of the new R&D buildings under wraps).
Because the AR experience is dynamically created in software, Apple will be able to continuously update its AR exploration iPads to add new details and quirky Easter eggs to find-- perhaps they'll add a timeline depicting the site's seven years of construction, or a seasonal depiction of Apple Park as its trees lose their fall leaves or sprout out new ones in the spring.
Beyond looking at the structures, you can also use the AR iPad app to see how the site's solar panels and energy distribution work. In energy mode, the vast solar arrays on the Spaceship, the main parking structures and the Phase 2 parking and data center building glow in a sunny gold, and energy distribution is depicted by animated, sparkling tubes that connect the various buildings.
A third airflow mode lets you look at how ventilation systems help to passively cool the buildings. As you walk around the model, you can sense how cool blue air currents blowing in from the Pacific Ocean to the west flow over the site, picking up heat from human activity and the server farms inside, turning into red currents as they pass through.
Beyond playing games that turn your iOS device into a window looking into a fantasy world, ARKit can also enable users to peel back layers of structure and overlay depictions of invisible forces such as electrical energy, magnetic fields and heat onto real world objects.
Directive Games "The Machines"
ARKit can be used to create apps that interactively visualize the activity inside of a volcano on the horizon, see elevators rising inside an office tower, observe the migration patterns of birds around a globe, track the real world movement of subway trains below your feet in a city, or visualize what you'd look like wearing a new sweater or a specific cut of jeans.
Apple's use of iPads to debut its visitor center AR tour also shows how useful it is to have an immediately intuitive multitouch interface for configuring your augmented view: you can simply touch and swipe to select what layers you want to explore, what invisible ideas you want to visualize and what time of day you want to see depicted in your AR environment.
This comes in contrast to the popular narrative that claims "real" AR or "mixed reality" inherently (or ultimately) demands the kind of goggles that Google and Microsoft have shown off, paired with gloves or handheld controllers to interact with elements in the virtual world.
The hardware expense of Glass, Hololens or Oculus, paired with a way to interact with what you see, combines to create the current barrier of expense that has blunted their mainstream adoption.
Additionally, goggle-based VR experiences are notorious for triggering nausea in many users because it divorces your body position and head movements from your visual perception of the world, something that Apple's screen-based, handheld AR doesn't.
While immersive VR goggles have entertaining and practical applications, the new AR worlds Apple has made available in iOS 11 allow the user to contrast regular and augmented views in the same way you might hold up a magnifying glass, seeing a before and after view of the difference in reality and the digital augmentation.
ARKit will also serve a much broader market with an installed base nearing 1 billion devices, enabling today's iOS users to experience AR without spending lots of money on an entirely new platform and new hardware up front. As soon as iOS 11 ships on September 19, the ARKit platform will represent by far the largest installed base of AR devices.
Apple hasn't yet detailed when it will open its new Apple Park visitor center to the public, but it's sure to become a popular destination for millions of people to visit, as its other spectacularly crafted retail locations have already become.
Something old, something new, something that's $108M just for you
The day before Apple's big event in the Steve Jobs Theater, AppleInsider took some photos of the soon to be opened, new Visitor Center created for Apple Park, which sits just across from the main pedestrian security checkpoint entrance leading to the walking paths into the sprawling campus.The new Augmented Reality experience created for Apple Park shows off the potential of ARKit in iOS 11
What wasn't visible until Apple opened up its Visitor Center as a sneak peak reception after the event was the new Augmented Reality experience created for Apple Park, showing off the potential of ARKit in iOS 11.
When it opens to the public, the Visitor Center will primarily serve as an Apple Store, paired with a cafe. It also houses a vast, 11,000 pound model of the Apple Park site (below), which serves as an aluminum foil for the real star of the show: a virtual tour presented in AR using provided iPads. This new feature distinguishes the new site from the company's other retail locations.
Flagship af
The new Apple Park Visitor Center store also shares a lot in common with Apple's other big budget flagship locations. Despite being located in the sleepy suburbs of Cupertino, California (it straddles the city limits of Santa Clara; just beyond its rear wall there are quiet, tree-lined culs-de-sac of single family homes) the new location shares the same kind of spectacular architecture as some the company's most glamorous pinnacles of merchandising.Grand Central Terminal
Paris Opera
London Covent Garden
The new store features wonderfully crafted stone stairways (below) that convey the same sort of timeless craftsmanship felt in the stores integrated into historic gems such as New York City's Grand Central Terminal (above top), the former bank at Paris Opera (above middle) or London's spectacular Covent Garden site (above bottom).
Timeless stone stairways...
... and modern walls of glass
The new Apple Park store also feels incredibly modern, with double-high walls of optically-pure glass that--like the breathtaking entrance to the Steve Jobs Theater--act as structural components, not just windows.
Hong Kong IFC
San Francisco's Union Square
The Oculus at World Trade Center
Unlike the cool urban sophistication of glass and steel seen in Hong Kong's Hysan Place and IFC tower (above top), San Francisco's Union Square (above middle) or The Oculus at NYC's World Trade Center (above bottom), the new Apple Park store blends into its orchard-like surroundings by making use of earthy warm wood slats in its ceilings and across its rooftop deck, which offers a view of the vast Spaceship hiding behind a forest of mature trees just across the street.
The earthy, modern AR-enhanced model room.
A view of the Spaceship from the roof
Al;AR: augmented aluminum with an iPad app
The aluminum site model is impressive in its heft but only offers a simplistic rendering of the buildings and landscape of Apple Park. However, when you hold up one of the AR iPads that workers hand you, a new AR exploring app lets you see detailed, photo-realistically rendered images of the actual buildings, complete with streets that feature animated vehicles driving down the road as if in SimCity.You can flick on the rooftops of buildings to see inside them, where you can see the arrangements of internal walls, desks and even animated people walking around. Because the app is depicting the scene using AR, you can look up, across, inside and around the vast model to explore its mini-world of site details.
There's also a time-of-day slider that renders the AR graphics with lighting and shadows from early morning to high noon and into the evening, when the lights are switched on to get a feel of what it would look like to fly over the site at night.
If you look at the grand cafeteria in the early morning, you can see an animated depiction of its massive window-wall-doors opening up to expose the vast indoor space to the outside
At dusk there's a red sky, and if you look at the grand cafeteria in the early morning, you can see an animated depiction of its massive window-wall-doors opening up to expose the vast indoor space to the outside, taking advantage of the Bay Area's mild climate (similar to the huge doors of the company's recently opened new San Francisco Union Square store).
In addition to looking under the roof to see the inner workings of the Spaceship, you can also lift the lid on the Fitness Center (which has contrasting stone walls, rather than glass like the rest of the buildings on the site).
You can even lift up the historic barn that's sat on the site for decades--long before Hewlett Packard paved the original orchards to put up a sea of parking lots for its executive briefing center. When you lift the barn, all that's left behind are some bales of hay and what looks like it could be Clarus the dogcow.
You can also lift the lid of the Steve Jobs Theater, which exposes the grand round display area that leads into the theater pit below. Phase 2 buildings are included in the model, but are not yet interactive to look inside (or perhaps that's just to keep the magical mysteries of the new R&D buildings under wraps).
A post shared by AppleInsider (@appleinsider_official) on Sep 12, 2017 at 7:38pm PDT
Because the AR experience is dynamically created in software, Apple will be able to continuously update its AR exploration iPads to add new details and quirky Easter eggs to find-- perhaps they'll add a timeline depicting the site's seven years of construction, or a seasonal depiction of Apple Park as its trees lose their fall leaves or sprout out new ones in the spring.
Beyond looking at the structures, you can also use the AR iPad app to see how the site's solar panels and energy distribution work. In energy mode, the vast solar arrays on the Spaceship, the main parking structures and the Phase 2 parking and data center building glow in a sunny gold, and energy distribution is depicted by animated, sparkling tubes that connect the various buildings.
A third airflow mode lets you look at how ventilation systems help to passively cool the buildings. As you walk around the model, you can sense how cool blue air currents blowing in from the Pacific Ocean to the west flow over the site, picking up heat from human activity and the server farms inside, turning into red currents as they pass through.
Apple's un-goggled vision for AR is already here
The AR experience isn't just an interesting way to explore details of the campus that won't open to the public; it also serves as a depiction of how third party developers can use ARKit to create explorable, augmented worlds that are not just fun to fly over but can also serve as interactive, configurable presentations of information that isn't obvious or visible.Beyond playing games that turn your iOS device into a window looking into a fantasy world, ARKit can also enable users to peel back layers of structure and overlay depictions of invisible forces such as electrical energy, magnetic fields and heat onto real world objects.
Directive Games "The Machines"
ARKit can be used to create apps that interactively visualize the activity inside of a volcano on the horizon, see elevators rising inside an office tower, observe the migration patterns of birds around a globe, track the real world movement of subway trains below your feet in a city, or visualize what you'd look like wearing a new sweater or a specific cut of jeans.
Apple's use of iPads to debut its visitor center AR tour also shows how useful it is to have an immediately intuitive multitouch interface for configuring your augmented view: you can simply touch and swipe to select what layers you want to explore, what invisible ideas you want to visualize and what time of day you want to see depicted in your AR environment.
This comes in contrast to the popular narrative that claims "real" AR or "mixed reality" inherently (or ultimately) demands the kind of goggles that Google and Microsoft have shown off, paired with gloves or handheld controllers to interact with elements in the virtual world.
The hardware expense of Glass, Hololens or Oculus, paired with a way to interact with what you see, combines to create the current barrier of expense that has blunted their mainstream adoption.
As soon as iOS 11 ships on September 19, the ARKit platform will represent by far the largest installed base of AR devices
Additionally, goggle-based VR experiences are notorious for triggering nausea in many users because it divorces your body position and head movements from your visual perception of the world, something that Apple's screen-based, handheld AR doesn't.
While immersive VR goggles have entertaining and practical applications, the new AR worlds Apple has made available in iOS 11 allow the user to contrast regular and augmented views in the same way you might hold up a magnifying glass, seeing a before and after view of the difference in reality and the digital augmentation.
ARKit will also serve a much broader market with an installed base nearing 1 billion devices, enabling today's iOS users to experience AR without spending lots of money on an entirely new platform and new hardware up front. As soon as iOS 11 ships on September 19, the ARKit platform will represent by far the largest installed base of AR devices.
Apple hasn't yet detailed when it will open its new Apple Park visitor center to the public, but it's sure to become a popular destination for millions of people to visit, as its other spectacularly crafted retail locations have already become.
Comments
The AR app for star gazing was really impressive. I think this opens up the potential for some real interactive learning.
I am stoked about the possibilities of this making education really interesting for my kids.
Soli owes me an apology... On an earlier AI thread I suggested that Apple could do an AR tour of Apple Park -- Soli said it would be a waste of time and money and too cartoonish... Apparently Apple had different ideas -- A mockup is too cartoonish and an AR rendering is more realistic.
Yes!
Peel back or add layers to show history. Think of the possibilities -- from Thermopylae to Battle of the Bulge. The movie industry could go crazy... FWIW, it appears that the scanning used in creating Apple Maps identifies the topology including river/lake/ocean bottoms, land beneath buildings, natural as well as man-made features.
Then there's the travel and real estate industries.
I suspect that it won't be too long before any major construction design will require a realistic AR model instead of man-made mockup. Think of it: Apple Park II rendered in AR on an iPad.
Then, there's the actual construction project itself... Project Management tasks intertwined with schedules and resources -- assisted with AR to be completed on budget and on time... I'm an old Pert guy (or was it POP) -- I can see lots of uses for AR in Project Management!
Then there's [dis]assembly/repair/parts manuals... Not only can Ikea use AR to let place furniture in your home -- it can show you how to put it together when it arrives (let's see, that's part BX3 goes into part C34...).
User Guides: Welcome to my world! To illustrate, I just bought a Bostich pancake compressor for my shop. I kid-you-not, the User Guide is a 2' x 3' two-sided sheet of paper that unfolds and folds up (if you're lucky) like a map -- a friggin' map... Back in the day, maps were one of the major causes of divorces (right after over/under toilet paper).
Think of how it's going to be in AR on your iPhone or iPad:
- you bring up the UG for the tool
- it shows a virtual image of the tool
- you pinch zoom, rotate the image as you wish (back, front, side, top, bottom, etc.)
- heads-up appear to show what you are looking at
- you zoom into the area of interest and remove anything that blocks your view
- you tap and get instructions/description that you can drill down to whatever level of detail you need
Now, consider this: a similar process can be used to provide a Guided Tour of the Tool -- when you first get it -- if you haven't used it for a while or are tuning it up -- or when you are considering whether to buy it... Are you listening Amazon, Home Depot, Lowes...There is nothing wrong with your idea for an interactive user guide, but how is this helped by AR? It seems to me like all of that is possible without viewing whatever tool or item live through a camera lens.
http://www.startrek.com/watch_video/episode-preview-the-apple
HD/surround: http://www.dailymotion.com/video/x3ivm5k
I think they could have an app that you download and does most of what was shown without the model. Most of the AR tour could be projected on any flat horizontal surface: floor, table, etc.
AFAIK, the current developer release of AR Kit doesn't, yet, support recognition of horizontal surfaces. Until then, I think they need something like the aluminum model to introduce walls for certain parts of the demo, e.g. the part, at about 33 seconds in, showing how airflow interacts with the building. Or, I could be wrong and there's something else going on.
Most [tool] user guides:
What happens is that:
For example:
Several advantages to AR approach:
You are manipulating the info according to your ad hoc needs -- rather than you compensating for info in a predetermined structure and format.
To your last point: "It seems to me like all of that is possible without viewing whatever tool or item live through a camera lens."
Apple maps on iOS 11 has a Flyover Mode -- in addition to the predetermined Flyover Tour. In Flyover mode you manipulate the view by manipulating the iDevice and Multitouching its display, Oddly, it doesn't use the camera lens -- rather the compass, accelerometer and multitouch gestures.
Call me crazy, but I think that within a short time ARKit will be available on macOS and Safari -- substituting the Mac Display for the camera lens and Magic Mouse (or somesuch) for iDevice multitouch gestures.
When you think about it, Apple Maps on the Mac already lets you manipulate virtual 3D objects in a limited ad hoc fashion.
LOL
Then there's this:
Using your DeWalt manual example. Sure, this can be confusing and time consuming. But that's DeWalt's fault for making the manual that way. Isn't it possible for DeWalt to make a digital manual that I could manipulate in a similar fashion to the way we use Maps so that I can zoom in or out, remove a piece of the tool to see what's underneath, etc. without using ARkit? There are already apps in the App Store that let me look at all the different systems in the human body, removing some to see what's beneath and how blood flows and muscles work and on and on, and they've been available long before ARkit was announced and I don't need to point my camera at an actual human for it to work.
I feel like maybe I'm missing the point.
EDIT: I guess this is what I'm trying to say: the "AR" in the article is augmenting a model of Apple Park which could be done almost identically using a 3D model that you could manipulate and 'explode' and not require viewing anything through a camera lens. To me a "real" AR experience would be walking around the actual Apple Park using the AR to "see through" the walls at how air currents flowed and "into" offices. The camera view would show me the actual building and the augmentation would be the extra stuff that gets overlayed.
The www came along and there were already ways to move around data on the internet. Many didn't see the point of it in 1993-1994, yet it led to an explosion of content and usage.
Simplifying leads to expanded markets; that's were Apple has always been and this is were they are now.