Apple granted patent using hinged OLED screen as a dynamic iPad or Mac keyboard

AppleInsider

Apple's new "dual display" patent could potentially allow for a future MacBook or iPad to use a second screen in place of a keyboard, eschewing physical keys.

The United States Patent and Trademark Office has granted Apple a patent titled "dual display equipment with enhanced visibility and suppressed reflections." The documentation for what is patent number 9,904,502 outlines a device that would use a second display as a dynamic keyboard.

Two implementations of this design are described in the patent application, according to Patently Apple. The first utilizes a permanent hinge, while the second allows the screen to be removed and used separately, along the lines of Microsoft's Surface Pro range and other two-in-one computers.

The patent documentation makes it clear that the implementation is not intended as an accessory that would allow two iPads to be paired together, with one serving as the keyboard. Additionally, illustrations associated with the application explicitly state that one screen is an OLED display, while the other is an LCD.

A double-display set-up could provide easy access to a different keyboard layout language, context-sensitive controls, or even a large sketching surface to use in conjunction with something like an Apple Pencil. However, that flexibility would come at the cost of the traditional typing experience offered by a mechanical keyboard.

Apple's patent addresses the fact that using a second display as a keyboard could result in unwanted reflections between one screen and the other. Polarizer layers and wave plates would be used in both displays to prevent any such glare.

Apple has already flirted with the idea of a dynamic keyboard in part, in the form of the OLED Touch Bar that was introduced to the MacBook Pro line in 2016. Presenting the entire keyboard in this manner would seem to be one possible evolution of that idea.

Many of Apple's patented inventions don't make it to market. At present, there are no rumors suggesting that Apple is actively working on this concept with any of its partners or is close to bringing it to market.

seanismorris

FYI: Hardness
Gorilla Glass = 6 to 6.5 Mohs 
Granite (dirt) = 7 Mohs

If you close the display without being clean you have a good chance of scratching it.  Of course not all dirt is granite, but if you close it 100+ times the odds are you’re going to eventually scratch it.  Also, the coating on the “glass” is no where as hard as Gorilla Glass.

Ideality you’d use a screen protector, but that doesn’t seem possible...

If you could get Gorilla Glass to 7.5 Mohs the design would be more practical.  

But, I don’t see it as feasible on < 12.9 inch iPads (folding in the middle)... a separate keyboard still makes more sense.

edited February 2018

rinosaur

Already in the Microsoft pipeline, we’ll see who gets it out to market first.

spamsandwich

rinosaur said:

Already in the Microsoft pipeline, we’ll see who gets it out to market first.

Apple and Microft have a broad patent sharing arrangement to avoid lawsuits, so things like this may already be covered. The only difference would be in the way each company chooses to implement the patented elements in a product.

https://www.theverge.com/2012/8/13/3239977/apple-and-microsoft-cross-license-agreement-includes-anti-cloning

edited February 2018

baka-dubbs

I'm officially old now, the idea of trying to type without tactile keys sounds horrific.  I would understand for a future ipad if they wanted to make it foldable, but I do not want a MacBook that doesn't have a legit keyboard.   Now if they could do a near seemless design with some form of raised keys built into the display(which I don't think is anywhere near feasible atm), then I would be interested.  But for my use case, on a work machine, I want a physical keyboard with raised keys.  I type slower on laptop keyboards with less depth as is.

edited February 2018

raoulduke42

I'm hoping Apple comes up with some secret sauce to utilize some of their "force touch" magic on a virtual keyboard. Maybe use some infra-red interferometer cleverness to sense an imminent "key" touch so as to have one of those crazy haptic vibrations that feel like a click happen as soon as your finger is on the glass. That'd be rad.

shamino

baka-dubbs said:

I'm officially old now, the idea of trying to type without tactile keys sounds horrific.

Haven't you been paying attention?  Only nerds and wimps type.  The future is all Siri all the time.

(closed-smilied for the humor impaired)

spamsandwich

baka-dubbs said:

I'm officially old now, the idea of trying to type without tactile keys sounds horrific.  I would understand for a future ipad if they wanted to make it foldable, but I do not want a MacBook that doesn't have a legit keyboard.   Now if they could do a near seemless design with some form of raised keys built into the display(which I don't think is anywhere near feasible atm), then I would be interested.  But for my use case, on a work machine, I want a physical keyboard with raised keys.  I type slower on laptop keyboards with less depth as is.

Touchscreens as keyboards are more difficult to use over a prolonged period because there’s no natural resting state for fingers and palms. Basically, you touch the keys and you are inputting. If Apple adjusted the sensitivity of a touchscreen keyboard so a person COULD rest their fingers on the keys without triggering an accidental keystroke, then by slightly increasing finger pressure or velocity resume input, that would go a long way toward making such a flat surface keyboard more tolerable over the course of a workday.

anton zuykov

rinosaur said:

Already in the Microsoft pipeline, we’ll see who gets it out to market first.

Just like MS laptops that resemble MacBooks? Or like MS unibody desktop stations that resemble iMacs?

baka-dubbs

SpamSandwich said:

Touchscreens as keyboards are more difficult to use over a prolonged period because there’s no natural resting state for fingers and palms. Basically, you touch the keys and you are inputting. If Apple adjusted the sensitivity of a touchscreen keyboard so a person COULD rest their fingers on the keys without triggering an accidental keystroke, then by slightly increasing finger pressure or velocity resume input, that would go a long way toward making such a flat surface keyboard more tolerable over the course of a workday.

I'm sure typing on a flat surface can be made better, but I would still have trouble orienting my hands when I have no home keys and know way to tell without looking if my hands have shifted.  I learned to type on a word processor though, so I fully acknowledge that I may just be stubborn and set in my ways

rayz2016

rinosaur said:

Already in the Microsoft pipeline, we’ll see who gets it out to market first.

No, we’ll see who’s works best. 

wizard69

Im still thinking Apples long term iPad goal is to emulate the performance of the machine in the knowledge navigator videos. That means a foldable unit with very high performance when it comes to AI interaction.  

That means most of Siri running locally and interactively.    Interestingly Apples A series chips are quickly moving in this direction.   I can imagine the next process shrink introducing substantialky more capable AI processing hardware.  This wil be good enough to run Siri locally allowing for higher integration into the OD and apps.   Soon iOS and Siri will become one.  

Of course there will be idiots out there crying planned obselecence because these features will required hardware upgrades.    In any event we will see AI in every app Apple includes with iPad.   Some of your interactions will become Star Trek like or certainly remind you of Sci-Fi of the past.  With the AI done locally you get fast human like interaction and fast access to off device data.  

This could be less than 5 years away, but no matter the time frame how we use computers will change dramatically.  

anome

I've been saying this is where they were heading for ages now. Whether they bring it to market is still to be determined. Also, I did think they would use a modified Taptic Engine(tm) to give force feedback and present an illusion of keys, and that they've been training us via the glass keyboard on iOS, and the butterfly mechanism in MacBooks to get used to less travel on the keys.

Of course, the real value isn't in a virtual keyboard, but in the other possible interfaces you can do with such a set up. Replace the keyboard with a jog shuttle for editing audio or video, virtual sliders for adjusting levels, a piano keyboard for playing music, etc (it must be early, I know there are a lot more options, but I'm blanking on them).

Now, getting rid of keys altogether is going to upset a lot of people, who will then declare they can never use Apple laptops again. (Desktops will still have the option of attaching a "proper" keyboard.) In order to mollify them slightly, I'd be interested to see Apple do something I think they patented a few years back - essentially a mechanical keyboard that fits over the OLED display, but can be removed (or even replaced) when needed. I suppose if Apple don't do it, someone else will, but then it probably won't fit in place when the laptop is closed, and it becomes cumbersome to use.

analogjack

It's a red herring.

jmgregory1

anome said:

I've been saying this is where they were heading for ages now. Whether they bring it to market is still to be determined. Also, I did think they would use a modified Taptic Engine(tm) to give force feedback and present an illusion of keys, and that they've been training us via the glass keyboard on iOS, and the butterfly mechanism in MacBooks to get used to less travel on the keys.

Of course, the real value isn't in a virtual keyboard, but in the other possible interfaces you can do with such a set up. Replace the keyboard with a jog shuttle for editing audio or video, virtual sliders for adjusting levels, a piano keyboard for playing music, etc (it must be early, I know there are a lot more options, but I'm blanking on them).

Now, getting rid of keys altogether is going to upset a lot of people, who will then declare they can never use Apple laptops again. (Desktops will still have the option of attaching a "proper" keyboard.) In order to mollify them slightly, I'd be interested to see Apple do something I think they patented a few years back - essentially a mechanical keyboard that fits over the OLED display, but can be removed (or even replaced) when needed. I suppose if Apple don't do it, someone else will, but then it probably won't fit in place when the laptop is closed, and it becomes cumbersome to use.

I've thought about this exact thing for years.  Forget the standard keyboard implemented on a touch-screen that replaces a typical keyboard on a laptop or desktop and jump right into completely customizable input options, from typing to drawing or some combination of multiple inputs.  One negative will be the additional power a second screen will add, but that could be mitigated by using an e-ink display.  

It's likely to be the next big thing in computing, because it really does open up how you can interact with the device in a more natural way than Microsoft's vision where you have to touch the screen in front of you - which is a horrible and not sustainable way to work, where your hands and arms need to be held up in the air vs. resting in a natural state on a desktop or your lap.

boodle

It's a Nintendo 3DS. Rad.

spamsandwich

baka-dubbs said:

SpamSandwich said:

Touchscreens as keyboards are more difficult to use over a prolonged period because there’s no natural resting state for fingers and palms. Basically, you touch the keys and you are inputting. If Apple adjusted the sensitivity of a touchscreen keyboard so a person COULD rest their fingers on the keys without triggering an accidental keystroke, then by slightly increasing finger pressure or velocity resume input, that would go a long way toward making such a flat surface keyboard more tolerable over the course of a workday.

I'm sure typing on a flat surface can be made better, but I would still have trouble orienting my hands when I have no home keys and know way to tell without looking if my hands have shifted.  I learned to type on a word processor though, so I fully acknowledge that I may just be stubborn and set in my ways

I learned to type on an IBM Selectric!

spamsandwich

wizard69 said:

Im still thinking Apples long term iPad goal is to emulate the performance of the machine in the knowledge navigator videos. That means a foldable unit with very high performance when it comes to AI interaction.  

That means most of Siri running locally and interactively.    Interestingly Apples A series chips are quickly moving in this direction.   I can imagine the next process shrink introducing substantialky more capable AI processing hardware.  This wil be good enough to run Siri locally allowing for higher integration into the OD and apps.   Soon iOS and Siri will become one.  

Of course there will be idiots out there crying planned obselecence because these features will required hardware upgrades.    In any event we will see AI in every app Apple includes with iPad.   Some of your interactions will become Star Trek like or certainly remind you of Sci-Fi of the past.  With the AI done locally you get fast human like interaction and fast access to off device data.  

This could be less than 5 years away, but no matter the time frame how we use computers will change dramatically.  

I think the end goal is to make the computer disappear and have all interactions be seamless and completely user centric, regardless of physical ability, language or process.

dick applebaum

baka-dubbs said:

SpamSandwich said:

Touchscreens as keyboards are more difficult to use over a prolonged period because there’s no natural resting state for fingers and palms. Basically, you touch the keys and you are inputting. If Apple adjusted the sensitivity of a touchscreen keyboard so a person COULD rest their fingers on the keys without triggering an accidental keystroke, then by slightly increasing finger pressure or velocity resume input, that would go a long way toward making such a flat surface keyboard more tolerable over the course of a workday.

I'm sure typing on a flat surface can be made better, but I would still have trouble orienting my hands when I have no home keys and know way to tell without looking if my hands have shifted.  I learned to type on a word processor though, so I fully acknowledge that I may just be stubborn and set in my ways

As to resting your hands on the keys and orienting your hands -- if Apple does something like this, I suspect that:

1. resting fingers on the keys will not cause typing -- it will take something like a push (harder) or a tap (up, then down).
2. orienting your hands won't be a problem because the keyboard will move -- wherever you rest your fingers (on the home row) the rest of the keyboard will follow.
3. using AI the keyboard could adjust for the user(s) with large or small hands, fat fingers, fingernails, etc. -- and correct for the user(s)  bad typing habits/errors.

Thinking of the etc. in item 3 reminds me of a true story:  In the Las Vegas IBM office, circa 1964 -- the office (one big room) was shared between computer people and typewriter people.  On one particular day the typewriter people were demonstrating their newest product a Selectric Composer to a very attractive and buxom personal assistant to the head of one of the casinos.  

Of course, all the male typewriter and computer people in the office watched the demo.

It started OK, but the lady couldn't get up to speed... it seemed that something(s) was touching the Selectric keyboard and causing typing errors. 

macxpress

rinosaur said:

Already in the Microsoft pipeline, we’ll see who gets it out to market first.

Who cares who gets it out first. Being first doesn't mean its best. I don't know why some people are stuck on this well A company has this out first like it automatically makes it better. Rushing a product to market just to be first rather than taking a little longer and making it work correctly is never the way to go. 

spamsandwich

dick applebaum said:

baka-dubbs said:

SpamSandwich said:

Touchscreens as keyboards are more difficult to use over a prolonged period because there’s no natural resting state for fingers and palms. Basically, you touch the keys and you are inputting. If Apple adjusted the sensitivity of a touchscreen keyboard so a person COULD rest their fingers on the keys without triggering an accidental keystroke, then by slightly increasing finger pressure or velocity resume input, that would go a long way toward making such a flat surface keyboard more tolerable over the course of a workday.

I'm sure typing on a flat surface can be made better, but I would still have trouble orienting my hands when I have no home keys and know way to tell without looking if my hands have shifted.  I learned to type on a word processor though, so I fully acknowledge that I may just be stubborn and set in my ways

As to resting your hands on the keys and orienting your hands -- if Apple does something like this, I suspect that:

1. resting fingers on the keys will not cause typing -- it will take something like a push (harder) or a tap (up, then down).
2. orienting your hands won't be a problem because the keyboard will move -- wherever you rest your fingers (on the home row) the rest of the keyboard will follow.
3. using AI the keyboard could adjust for the user(s) with large or small hands, fat fingers, fingernails, etc. -- and correct for the user(s)  bad typing habits/errors.

Thinking of the etc. in item 3 reminds me of a true story:  In the Las Vegas IBM office, circa 1964 -- the office (one big room) was shared between computer people and typewriter people.  On one particular day the typewriter people were demonstrating their newest product a Selectric Composer to a very attractive and buxom personal assistant to the head of one of the casinos.  

Of course, all the male typewriter and computer people in the office watched the demo.

It started OK, but the lady couldn't get up to speed... it seemed that something(s) was touching the Selectric keyboard and causing typing errors. 

Oh, my. Sounds like a memorable demo.