Apple's patent for an LCD display that also takes photos, video

[AppleInsider]

As rumors swirl over the possibility of a 3G iPhone with video conferencing capabilities, it may be of some interest to recall an earlier patent filing from Apple for an 'integrated sensing display' capable of serving as both a display screen and a digital camera.

The filing, which predates the iPhone's release by nearly three years, describes a new breed of LCD display that could simultaneously take photos while continuing its role as the primary display screen of an electronics device or computer monitor.

The idea behind the invention is to wedge thousands of microscopic image sensors between the LCD cells that make up the display, where each sensor would be responsible for capturing a piece of the overall photo. Those pieces would then be stitched together by software to recreate the complete image capture.

According to the filing, made back in June of 2004, the tiny image sensors could also have variable focal lengths in order to zoom and focus the image: "The lens might either be physically moved or have its properties altered. In the latter case, the lens might be made of a material that changes its refractive index in response to electrical stimulus or a magnetic field."

The filing has a newfound relevance given recent claims by Digg founder Kevin Rose that Apple's forthcoming iPhone revision will include a camera mounted on the face of the unit to coincide with a mobile version of iChat suited for handheld video conferencing.

While Rose's track-record on predicting Apple's future directions is akin to a hit or miss affair, the advent of video conference via iPhones is seemingly inevitable given time. Therefore his proposals, as well as those outlined in the patent, could potentially surface slightly further down the road should they not be part of Apple's more immediate plans.

Of particular interest in this regard, is the electronics maker's claims that 'integrated sensing displays' are ideally suited for video conferencing on handhelds, where the available real estate limits the number of components -- such as a front mounted cam in addition to a rear-mounted cam -- that can fit into the devices.

"A panel created from an integrated, embedded macro charge-coupled devices (CCD) would thus be able to both display and record visual information simultaneously, without the use of an external video capture device. Simply put, the same panel could display a video conference while also recording the participant sitting in front of it," the company said in the filing.

"A video panel that has an embedded macro CCD is no longer just a display. It can be used to transmit as well as receive visual information. One use and benefit for such a panel is video conferencing: a user can maintain eye contact with someone on screen because the camera is 'in' the screen. In addition, portable devices, such as portable digital assistants (PDAs) and cell phones, have very limited space for displays and would benefit if additional real estate were not used for a camera."



The patent filing was published by the United States Patent and Trademark Office in April of 2006 alongside three other filings pertaining to wireless iPods, iChat whiteboards, and virtual touch-screen keyboards -- technologies which were all later implemented in shipping Apple products.

[ View this article at AppleInsider.com ]

[Timeline]

Hmmm, VIdeo Conferencing?

[SpinDrift]

I wonder how power hungry that would be? The theory is excellent, but will it be something that we see as soon as the next gen iPhone? I hope so.

[knadles]

Irony abounds. Apple has invented the telescreen.

[I am a human]

Man alive, I love technology!

Things are getting more and more like Star Trek every day.

[Timeline]

Even on Star Trek they never saw communication images on the Tricoder.

[Lukaz]

Is it possible that the current generation of iPhones (and perhaps the Touch too) ALREDY has this thing!?


Gosh!




"iPhone spanks competition to death"

[csimmons]

So the screen IS the front side camera.

That's just sick.

I love it.

[JeffDM]

Originally Posted by knadles

Irony abounds. Apple has invented the telescreen.

There already were other things that did the job, this is just a specific implementation of it, which might do a better job for that task, but it breaks up the screen pattern to do it.

[Olternaut]

Originally Posted by csimmons

So the screen IS the front side camera.

That's just sick.

I love it.

Actually, a lot of star trek devices had this telescreen stuff......they didn't just have the audio communicators. Anybody remember tekwar? Those video communicators they had is EXACTLY what I'm looking for in the fabled apple tablet.

Yup, micro-cams embedded in the screen itself. Technology is moving forward. Of course.....its being dragged, slowly, kicking and screaming. But at least its moving forward.

[Ireland]

I don't care for video conferring on a phone, frankly I rarely use it on my computer. An OLED screen excites me way more that any video conferencing capabilities.

OLED with rich blacks and vivid colors and the addition of better battery life, and then 3G web speeds and a 32GB HD - that's mouth watering!

[Maestro64]

When video conference first came to the computer world back in the late 80's and early 90's this whole idea of embedding the recording source into the playback source was investigated. The reason it died back then was the fact that people did not like the fact they might not know if someone could see their image when they did not want to. The whole 1984 concept that someone could be watching you.

It a good idea since most people will be looking at the screen because the are viewing the other sides video so the other side will see them straight on and see their eyes. This is the biggest complaint about doing video conferencing on the computer most peoples cameras are not positioned correctly.

[Lukaz]

Originally Posted by Ireland

I don't care for video conferring on a phone, frankly I rarely use it on my computer. An OLED screen excites me way more that any video conferencing capabilities.

OLED with rich blacks and vivid colors and the addition of better battery life, and then 3G web speeds and a 32GB HD - that's mouth watering!

What about an OLED with videoconferency?

I'm serius, if my iPod Touch alredy has this thing (Apple patented it before the beginning of the iPhone Project), I'm going to freak out. First the 3-axis acelerometer (I didn't knew it was 3-axis) now this? What, does this thing also have hidden bluetooth in it?

[palegolas]

It looks really easy on paper with small black dots and all, but I don't have a clue of how well this could work out practically.
And also, would all these microsensors capture "parallell light"? Making a photo with a really weird or no perspective?

[KennyWRX]

They supposedly already have a translucent touch screen right... I would imagine they could put a camera behind it and process the image using a "backdrop" kind of effect to eliminate distortion due to the actual screen not being completely transparent.

[mstone]

Originally Posted by palegolas

It looks really easy on paper with small black dots and all, but I don't have a clue of how well this could work out practically.
And also, would all these microsensors capture "parallell light"? Making a photo with a really weird or no perspective?

It works just like the CheezIt commercial on TV. They load up a 400 mm Canon zoom lens in a catapult and fire it 300 meters at a big target with an iPhone fixed in the center and BOOM Camera Screen!

[anantksundaram]

Originally Posted by Ireland

I don't care for video conferring on a phone, frankly I rarely use it on my computer. An OLED screen excites me way more that any video conferencing capabilities.

OLED with rich blacks and vivid colors and the addition of better battery life, and then 3G web speeds and a 32GB HD - that's mouth watering!

Not to rain on the mouth-watering parade (I actually agree with you on all of the above) and not that they are mutually exclusive in any way, but I'd like to see cut-copy-paste and voice-activated dialing before anything else.

Can't believe that it's a year and a quarter now, and Apple still hasn't implemented these two very basic features.

[thrang]

Originally Posted by anantksundaram

Not to rain on the mouth-watering parade (I actually agree with you on all of the above) and not that they are mutually exclusive in any way, but I'd like to see cut-copy-paste and voice-activated dialing before anything else.

Can't believe that it's a year and a quarter now, and Apple still hasn't implemented these two very basic features.

AGREED!

I think Apple's decision NOT to offer voice dialing is marketing-driven. Despite the clear need to have this, remember that as soon as they introduce it, millions of iPhones will suddenly remain hidden in pockets, purses, briefcases, an belt clips. And right now, I think Apple wants those iPhone OUT in the OPEN as frequently as possible to garner as much attention as possible. But in reality, they are sacrificing sales and safety by not including this. Voice dialing is very important for a lot a people who make calls in their car, and forcing them to dial from the slippery brick means it will be either unsafe for them (and us fellow drivers) or it will be unsold (may father and wife, despite admiring my iPhone, won't buy for this reason alone).

[nagromme]

Interesting concept. But a plain old off-the-shelf camera component seems like it does the job just fine.

[solipsism]

This could also be used for 3D image and video creation.

[SpamSandwich]

Just a reminder, Apple did not come up with the idea for this kind of camera-screen. I distinctly remember the introduction of a chip that had these properties years before the Apple patent was announced.

Also, since the camera lens function would be essentially software based, I don't think the quality could even approach that of a lens. Any engineers care to comment?

[Macvault]

I want to see this technology incorporated into transparent OLED display technology so we can have something that looks like this... http://gizmodo.com/gadgets/apple/tra...kup-185119.php

[Maestro64]

Originally Posted by thrang

AGREED!

I think Apple's decision NOT to offer voice dialing is marketing-driven. Despite the clear need to have this, remember that as soon as they introduce it, millions of iPhones will suddenly remain hidden in pockets, purses, briefcases, an belt clips. And right now, I think Apple wants those iPhone OUT in the OPEN as frequently as possible to garner as much attention as possible. But in reality, they are sacrificing sales and safety by not including this. Voice dialing is very important for a lot a people who make calls in their car, and forcing them to dial from the slippery brick means it will be either unsafe for them (and us fellow drivers) or it will be unsold (may father and wife, despite admiring my iPhone, won't buy for this reason alone).

I can not use a phone today without this feature. My wife has an iphone and since I work for one of the competitor phone companies I am not allow to use a Iphone at work,. But that is okay, beside the cool factor the iphone offers it does not presently support all the things I currently use on my smart phone, the big one being voice dialing.

Hands down the iphone is the most dangerous phone to use while driving a car, you can not dial by touch (meaning feeling the keys) and it does not allow voice dialing so you are crazy to even attempt to dial the iphone while driving. You have to look it to dial, no way around this, they do not even offer speed or one button dialing.

I like the fact that I can say a person name in my address book and phone dials it never requirement me to take my hands off the wheel.

[Gotecki]

Originally Posted by Macvault

I want to see this technology incorporated into transparent OLED display technology so we can have something that looks like this... http://gizmodo.com/gadgets/apple/tra...kup-185119.php

Well, I really hope it's not gonna look like that...

I remember that Toshiba invented something like that... in 2003!

http://www.pcworld.com/article/id,11...s/article.html

Patent war? We'll see!

[suhail]

The idea is fantastic, but I don't see how this can work with microscopic cameras and no common lens.

[JeffDM]

Originally Posted by suhail

The idea is fantastic, but I don't see how this can work with microscopic cameras and no common lens.

I don't think that's a major problem, it might be a very simplified version of a telescope array. It looks like each "camera" could have a single pixel, or maybe a 2x2 array, green red green blue. That could have a microlens above it, and maybe a short tunnel structure to eliminate light outside the desired field of view for the sensor.

I think the real problem is that all this will make the picture on the display look grainier because of this array of microcameras breaking up the image.

[8CoreWhore]

Jobs must've been absolutely livid when they decided OLED would have to wait (for whatever reason). It is thinner (because it requires NO back light) and uses less battery. Looking at the OLED technology on wiki: http://en.wikipedia.org/wiki/Oled Can anyone comment for against the idea that OLED is more compatible with the embedded camera??

[Lukaz]

Originally Posted by JeffDM

I think the real problem is that all this will make the picture on the display look grainier because of this array of microcameras breaking up the image.

I don't think so.

[Lukaz]

Originally Posted by 8CoreWhore

Jobs must've been absolutely livid when they decided OLED would have to wait (for whatever reason). It is thinner (because it requires NO back light) and uses less battery. Looking at the OLED technology on wiki: http://en.wikipedia.org/wiki/Oled Can anyone comment for against the idea that OLED is more compatible with the embedded camera??

I think that's because OLED is not that cheap yet.

[melgross]

Originally Posted by palegolas

It looks really easy on paper with small black dots and all, but I don't have a clue of how well this could work out practically.
And also, would all these microsensors capture "parallell light"? Making a photo with a really weird or no perspective?

I assume this would work the way insect eyes do. They also stitch the image together.

I'm just wondering how the quality of the screen would suffer. The display elements are moved further apart, rather than being almost close enough to touch.

The other question is how they would incorporate these tiny lens elements, solid or otherwise.

While the concept is great, more patents on exactly how this would be manufactured would be required before it could actually be made.

They are requiring several totally different manufacturing technologies on the same screen.

[melgross]

Originally Posted by anantksundaram

Not to rain on the mouth-watering parade (I actually agree with you on all of the above) and not that they are mutually exclusive in any way, but I'd like to see cut-copy-paste and voice-activated dialing before anything else.

Can't believe that it's a year and a quarter now, and Apple still hasn't implemented these two very basic features.

That's only because they know you want it.

[melgross]

Originally Posted by SpamSandwich

Just a reminder, Apple did not come up with the idea for this kind of camera-screen. I distinctly remember the introduction of a chip that had these properties years before the Apple patent was announced.

Also, since the camera lens function would be essentially software based, I don't think the quality could even approach that of a lens. Any engineers care to comment?

I don't think software is a problem. All single sensor chip cameras have extensive software manipulation done now. That includes the Foveon chip camera.

The questions revolve about the distance between the sensors, and how many of them there are. I don't see this working for hi rez photos, but that's not likely the intent. 640 x 480 would be great for portable video for conferencig.

[melgross]

Originally Posted by suhail

The idea is fantastic, but I don't see how this can work with microscopic cameras and no common lens.

The way insects do it, by stitching the images together.

[melgross]

Originally Posted by 8CoreWhore

Jobs must've been absolutely livid when they decided OLED would have to wait (for whatever reason). It is thinner (because it requires NO back light) and uses less battery. Looking at the OLED technology on wiki: http://en.wikipedia.org/wiki/Oled Can anyone comment for against the idea that OLED is more compatible with the embedded camera??

Depends on how they expect to get the technologies as a manufacturable product.

It's doable, either way.

[melgross]

Originally Posted by Lukaz

I don't think so.

I'm concerned about that as well.

If the display elements are moved, say, 20% further apart because of the sensing elements and their lenses, that means either of two things.

The first is that there would have to be about 20% fewer display elements per given screen size.

Two would be that for the same screen resolution, the screen would have to be about 20% larger.

[melgross]

Originally Posted by Lukaz

I think that's because OLED is not that cheap yet.

I think it's because the right screen isn't yet available, and possibly cost.

[Shogun]

Originally Posted by anantksundaram

Not to rain on the mouth-watering parade (I actually agree with you on all of the above) and not that they are mutually exclusive in any way, but I'd like to see cut-copy-paste and voice-activated dialing before anything else.

Can't believe that it's a year and a quarter now, and Apple still hasn't implemented these two very basic features.

Not to mention "To-Do's"!!!

[jeffharris]

Originally Posted by Maestro64

Hands down the iphone is the most dangerous phone to use while driving a car…

Hands down, driving and talking on the phone is absolutely one of the most dangerous things you can do. Driving while yapping is similar to driving drunk.

The level of stupidity involved is off the charts.

As a pedestrian and cyclist, I can't tell how many times I've nearly been killed when some idiot decides that blabbing on the phone is more important than whatever happens to going on around them. After all THEY ARE the center of the universe.

Is making that call IMMEDIATELY all THAT important? Why not just stop?

There are laws against it. they should be enforced.

[Ireland]

Originally Posted by anantksundaram

Not to rain on the mouth-watering parade (I actually agree with you on all of the above) and not that they are mutually exclusive in any way, but I'd like to see cut-copy-paste and voice-activated dialing before anything else.

Voice-dialing seems to be a very American thing to do, I guess that's why I never think of that. As for copy & paste. I solved that already, I'm just waiting for Apple to use my damn idea.

[anantksundaram]

Originally Posted by Ireland

Voice-dialing seems to be a very American thing to do, I guess that's why I never think of that. As for copy & paste. I solved that already, I'm just waiting for Apple to use my damn idea.

Ah, the winters where you live are probably mild! You probably don't wear gloves for half the year like I do!

And, I do remember your brilliant copy-paste idea from the past.