Apple buys machine learning AR firm specializing in mixed realities

AppleInsider · October 10, 2018 1:09PM

Apple has reportedly bought a Danish startup called Spektral, which specialized in real-time separation of objects in photos and videos for a "green screen" effect.

The acquisition was actually made late last year, but has flown under the radar until now, Danish publication Brsen revealed on Wednesday. Co-founders Henrik Paltoft and Toke Jansen are said to have been paid 200 million Danish krone, or about $30 million.

Jansen has become a manager of Computational Imaging at Apple, working out of the San Francisco Bay Area.

Spektral's process relied on machine learning and computer vision technology, even handling object separation for video running at 60 frames per second. The company touted this as useful for creating mixed-reality material.

The firm originated as CloudCutout, and first established a toehold through the U.S. school photography industry. In 2016 it launched website for creating images from product photos, soon after developing its real-time system. Prior to its acquisition, Spektral said it was "working fiercely" on bringing real-time cutouts to smartphones.

Apple's plans are uncertain, though it could conceivably incorporate the technology into apps like Camera or Clips, or expand on the use of animoji and memoji.

When contacted for comment, Apple presented the normal refrain about reported acquisitions, saying that "Apple buys smaller technology companies from time to time, and we generally do not discuss our purpose or plans."

Apple is known to be intensely interested in augmented reality. The company has developed a supporting iOS platform, ARKit, and even re-engineered its iPhone cameras to better support it. It has also made a number of related acquisitions, for instance buying Metaio in 2015, and AR lens maker Akonia Holographics earlier this year.

The company's long-term goal is believed to be an AR headset, codenamed "T288," which will run its own operating system currently dubbed "rOS." The gear may even have 8K displays and a separate processor, but isn't expected to ship until 2020 or later.

ihatescreennames · October 10, 2018 1:34PM

“It’s a very difficult problem to solve” and “we are a bunch of smart engineers and data scientists.”

Thanks for the modesty, LOL.

Still, it’s a cool idea. Hopefully Apple can bring this out in a sooner rather than later update.

spice-boy · October 10, 2018 1:44PM

Apple please design a phone that is easier to hold, and has a battery which last 24 hour, I don't care about making a sci-fi movie with mine.

ihatescreennames · October 10, 2018 2:54PM

spice-boy said:

Apple please design a phone that is easier to hold, and has a battery which last 24 hour, I don't care about making a sci-fi movie with mine.

Maybe the next version can have a built-in pop socket! /s

Of course, the issue with that is some people (like me) find the current battery life more than acceptable and others like having more photo and video possibilities. You can’t please everyone all the time.

PickUrPoison · October 10, 2018 2:58PM

spice-boy said:

Apple please design a phone that is easier to hold, and has a battery which last 24 hour, I don't care about making a sci-fi movie with mine.

Get a battery case for your 5-10% edge case requirements; most users end the day with plenty of battery remaining.

Apple won’t—and shouldn’t—be making a thicker, heavier, more expensive iPhone just because a few people play on their phone all day.

dick applebaum · October 10, 2018 3:06PM

Ohhhh...

RR (Real-time Rotoscoping)...

Think of sports replays, where you:

select a player as the foreground
isolate that player from the background (including other players)
highlight the foreground and/or deemphasize the background
re-composite the foreground and background

Boom! You have a highlighted replay video in an instant.

Sure, the networks can do that now with a lot of expensive gear in their broadcast trucks... But you will be able to do the same with that little camera/computer in your pocket -- and the person beside you can do the same with a different selected player.

Could also be used to review plays by referees, trainers, etc.

StrangeDays · October 10, 2018 3:46PM

spice-boy said:

Apple please design a phone that is easier to hold, and has a battery which last 24 hour, I don't care about making a sci-fi movie with mine.

“Apple, please change physics and chemistry.”

Seriously, there’s a reason no company on earth has magic batteries. Until there is a breakthrough in chemistry and something new is discovered, you can’t fault Apple for having the same batteries as everyone else.

dick applebaum · October 10, 2018 4:54PM

StrangeDays said:

spice-boy said:

Apple please design a phone that is easier to hold, and has a battery which last 24 hour, I don't care about making a sci-fi movie with mine.

“Apple, please change physics and chemistry.”

Seriously, there’s a reason no company on earth has magic batteries. Until there is a breakthrough in chemistry and something new is discovered, you can’t fault Apple for having the same batteries as everyone else.

Ya' know...

I've got this feeling on the back of my neck that a breakthrough of something old and obvious is about to happen...

Remember one of these:

https://en.wikipedia.org/wiki/Crookes_radiometer

Obviously, black and white when exposed to light/heat/radiation can generate energy:

Mmm... Black and White extreme opposites -- like yin and yang:

In Chinese philosophy, yin and yang (/jɪn/ and /jɑːŋ, jæŋ/; Chinese: 陰陽 yīnyáng, lit. "dark-bright", "negative-positive") describes how seemingly opposite or contrary forces may actually be complementary, interconnected, and interdependent in the natural world, and how they may give rise to each other as they interrelate to one another. In Chinese cosmology, the universe creates itself out of a primary chaos of material energy, organized into the cycles of Yin and Yang and formed into objects and lives. Yin is the receptive and Yang the active principle, seen in all forms of change and difference such as the annual cycle (winter and summer),

https://en.wikipedia.org/wiki/Yin_and_yang

....Black and White extreme opposites -- like ones and zeros

Now, what is comprised of ones and zeros...

Why all the bits of programs and data stored on your iPhone...

If there were a way to expose these ones and zeros on your iPhone as black and white images to light/heat/radiation...

Then there's this:

Back in August 2014, researchers at Michigan State University created a fully transparent solar concentrator, which could turn any window or sheet of glass (like your smartphone’s screen) into a photovoltaic solar cell. Unlike other “transparent” solar cells that we’ve reported on in the past, this one really is transparent, as you can see in the photos throughout this story. According to Richard Lunt, who led the research at the time, the team was confident the transparent solar panels can be efficiently deployed in a wide range of settings, from “tall buildings with lots of windows or any kind of mobile device that demands high aesthetic quality like a phone or e-reader.”
.
.
.
Scientifically, a transparent solar panel is something of an oxymoron. Solar cells, specifically the photovoltaic kind, make energy by absorbing photons (sunlight) and converting them into electrons (electricity). If a material is transparent, however, by definition it means that all of the light passes through the medium to strike the back of your eye. This is why previous transparent solar cells have actually only been partially transparent — and, to add insult to injury, they usually they cast a colorful shadow too.
The organic salts absorb UV and infrared, and emit infrared — processes that occur outside of the visible spectrum, so that it appears transparent.
To get around this limitation, the Michigan State researchers use a slightly different technique for gathering sunlight. Instead of trying to create a transparent photovoltaic cell (which is nigh impossible), they use a transparent luminescent solar concentrator(TLSC). The TLSC consists of organic salts that absorb specific non-visible wavelengths of ultraviolet and infrared light, which they then luminesce (glow) as another wavelength of infrared light (also non-visible). This emitted infrared light is guided to the edge of plastic, where thin strips of conventional photovoltaic solar cell convert it into electricity. [Research paper: DOI: 10.1002/adom.201400103 – “Near-Infrared Harvesting Transparent Luminescent Solar Concentrators”]
If you look closely, you can see a couple of black strips along the edges of plastic block. Otherwise, though, the active organic material — and thus the bulk of the solar panel — is highly transparent. (Read: Solar singlet fission bends the laws of physics to boost solar power efficiency by 30%.)

https://www.extremetech.com/extreme/188667-a-fully-transparent-solar-cell-that-could-make-every-window-and-screen-a-power-source

If I understand this correctly, you could charge your iPhone by using your iPhone.

elfig2012 · October 10, 2018 8:34PM

And Apple stocks 10 points down from 226 to 216...who understand this?

claire1 · October 10, 2018 10:01PM

StrangeDays said:

spice-boy said:

Apple please design a phone that is easier to hold, and has a battery which last 24 hour, I don't care about making a sci-fi movie with mine.

“Apple, please change physics and chemistry.”

Seriously, there’s a reason no company on earth has magic batteries. Until there is a breakthrough in chemistry and something new is discovered, you can’t fault Apple for having the same batteries as everyone else.

Only Apple gets the blame for industry wide problems. I've heard iKnockoff morons complain that iPhone glass breaks....

fastasleep · October 10, 2018 11:43PM

dick applebaum said:

StrangeDays said:

spice-boy said:

Apple please design a phone that is easier to hold, and has a battery which last 24 hour, I don't care about making a sci-fi movie with mine.

“Apple, please change physics and chemistry.”

Seriously, there’s a reason no company on earth has magic batteries. Until there is a breakthrough in chemistry and something new is discovered, you can’t fault Apple for having the same batteries as everyone else.

If I understand this correctly, you could charge your iPhone by using your iPhone.

I don’t think you do.

PickUrPoison · October 11, 2018 5:57AM

elfig2012 said:

And Apple stocks 10 points down from 226 to 216...who understand this?

Google down 55, amazon down 115, Netflix down 30. A broad market sell off, nasdaq off over 300 points, about 4%. Some say higher interest rates due to spiking inflation are spooking the market. Who knows.

bestkeptsecret · October 11, 2018 6:05AM

dick applebaum said:

Ohhhh...

RR (Real-time Rotoscoping)...

Think of sports replays, where you:
select a player as the foreground
isolate that player from the background (including other players)
highlight the foreground and/or deemphasize the background
re-composite the foreground and background

Boom! You have a highlighted replay video in an instant.

Sure, the networks can do that now with a lot of expensive gear in their broadcast trucks... But you will be able to do the same with that little camera/computer in your pocket -- and the person beside you can do the same with a different selected player.

Could also be used to review plays by referees, trainers, etc.

RR! We can make our own He-Man cartoons then!!

Apple buys machine learning AR firm specializing in mixed realities

Comments