or Connect
AppleInsider › Forums › Mac Hardware › Future Apple Hardware › Apple looks to improve pen-based input on tablet touchscreens
New Posts  All Forums:Forum Nav:

Apple looks to improve pen-based input on tablet touchscreens

post #1 of 69
Thread Starter 
Though Apple has shunned the stylus in favor of multi-touch on the iPhone and iPod touch, a new patent application suggests the company is investigating superior handwriting and input recognition via pen.

The patent application, revealed this week, describes a system that would comprehend complex "ink information" created from the use of a pen-like stylus. Entitled "Method and Apparatus for Acquiring and Organizing Ink Information in Pen-aware Computer Systems," the document includes multiple references to "tablet" style computers.

The application even includes a reference to the company's previous stylus-driven portable computer, the Apple Newton, when discussing that current handwriting technology is inadequate for many users.

"Even systems that attempt to improve this situation by using each stroke to determine the input field anew, such as the Apple Newton from Apple Computer Inc. of Cupertino, Calif., can suffer from failure modes that make the situation difficult for both end users and for application developers," the document reads. "For example, a word that accidentally spans two input fields even a tiny amount (due, for instance, to a stray ascender, descender, crossbar, or dot) may be broken up into multiple sessions, causing misrecognition and invalid data entries that must be manually corrected."

The application goes on to describe a pen-based system that would include an "ink manager" to serve as the intermediary between the stylus input and applications on the hardware.

"The ink manager interfaces between a pen-based input device, one or more applications (pen-aware or not) and one or more handwriting recognition engines executing on the computer system," the application states. "The ink manager acquires ink information, such as ink strokes, entered at the pen-based input device, and organizes that information into ink phrases."

Included with the application is an illustration of a tablet-style computer. The design is very much similar to what rumors have suggested Apple's highly anticipated, still unannounced tablet device will look like. AppleInsider has been told the 10-inch, multi-touch device will be unveiled in the first quarter of 2010.



The references to digital ink in the latest patent application, filed in July of 2009, are similar to previous patent filings regarding a next-generation multi-touch input surface. Those documents also referenced the use of a stylus, but noted that pen inputs are typically not dynamic enough to address the needs of many users. The previous applications favored the use of fingertips, and described a system that would be able to understand ten individual fingers separately.
post #2 of 69
I guess credit card companies wanted a better signiture from those modded touches that now serve as sales registers.
--SHEFFmachine out
Da Bears!
Reply
--SHEFFmachine out
Da Bears!
Reply
post #3 of 69
This could be huge but I have to wonder about the tech used to detect the "pen". Will the tablet have a capacitive sensor for Touch and another to detect pen position? I just can't see a reasonably sharp pen being picked up by a capacitive sensor. I don't think much of the inferred pen from finger position approach, the virtual pen if you will.

In any event I expect we wil see lots of patent applications from Apple in the coming weeks as it readies the tablet roll out. Hopefully this is an indication of a truely innovative platform.


Dave
post #4 of 69
I just got a Wacom graphics tablet and played with inkwell for a while, and it was terrible. My 11-year-old Newton that still sits on my desk has far better HWR than inkwell/Wacom combination. So yeah, it needs some updating.
PocketMoney for your iPhone
www.catamount.com
Reply
PocketMoney for your iPhone
www.catamount.com
Reply
post #5 of 69
The most important part for me with a pen based input is that the ink on the display is instantly on screen when writing - no catch-up, no jerkyness, etc. That way it feels reasonably natural. Also the ink should be anti-aliased for smoothness and pleasantness to the eye.

Handwriting recognition based upon strokes rather than analysing the resulting bitmap from writing on screen can be quite accurate, given suitable algorithms and maybe a little training (the device would be pre-trained for common handwriting styles, a 'h' is down, up a bit, right and down for example).

The best thing is that drawing little maps, directions, diagrams in notes/maps/ebooks/etc would be easy and natural.
post #6 of 69
Do we really want to go back to this old technology? Pens? Stylus? Aren't our fingers good enough?
post #7 of 69
Fingers are good for some things, pens are good for others. A device that supported both would be nice.

But I do think that on such a dual device finger would be used most of the time, with pen only for specialised uses.
post #8 of 69
This reminds me of Apple investigating superior mice.
Citing unnamed sources with limited but direct knowledge of a rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of a rumoured device - Comedy Insider (Feb 2014)
Reply
post #9 of 69
Quote:
Originally Posted by AppleInsider View Post

Included with the application is an illustration of a tablet-style computer. The design is very much similar to what rumors have suggested Apple's highly anticipated, still unannounced tablet device will look like.

What, as in a rectangle with curved corners? No shit Sherlock!
post #10 of 69
SJ will make sure that this technology will work well if it is used on the tablet.
80 million iPhones by 2012. That's only 15% of the market.

http://www.iphonethailand.net
Reply
80 million iPhones by 2012. That's only 15% of the market.

http://www.iphonethailand.net
Reply
post #11 of 69
Quote:
Originally Posted by iphonedeveloperthailand View Post

SJ will make sure that this technology will work well if it is used on the tablet.

God willing.
post #12 of 69
Quote:
Originally Posted by sheff View Post

I guess credit card companies wanted a better signiture from those modded touches that now serve as sales registers.

I don't think that has anything to do with it given that the WinMobile ones sucked so badly at the same thing.

I have to sign for packages at least once or twice a week, it's always a WinMobile device, and it always has some kind of problem. It either captures nothing at all, or it captures an illegible squiggle. The delivery person always asks for the name and then types it in on the keyboard as he/she is leaving.

It's been that way for years. I don't see how the iPod touch version could be any worse than that.
post #13 of 69
Quote:
Originally Posted by ascii View Post

Fingers are good for some things, pens are good for others. A device that supported both would be nice.

But I do think that on such a dual device finger would be used most of the time, with pen only for specialised uses.

To me this is the last kind of big mystery about the purported tablet from Apple.

Will there be a stylus? It's so *not* Apple to have a stylus, but the fancy patents they have that may allow them to do without one seem far-fetched.

If Apple can pull off a tablet that doesn't need a stylus to do ink input, it will go in the tech hall of fame. If they merely produce a tablet without a stylus and tell us we don't need ink input, there will be disappointment. If on the third hand, they produce a tablet with a gimmicky plastic stylus in a hole on the side it will be a similar letdown for many.

It will be real interesting to see what they do here.
post #14 of 69
Quote:
Originally Posted by wizard69 View Post

This could be huge but I have to wonder about the tech used to detect the "pen". Will the tablet have a capacitive sensor for Touch and another to detect pen position? I just can't see a reasonably sharp pen being picked up by a capacitive sensor. I don't think much of the inferred pen from finger position approach, the virtual pen if you will.

In any event I expect we wil see lots of patent applications from Apple in the coming weeks as it readies the tablet roll out. Hopefully this is an indication of a truely innovative platform.


Dave

I believe that you can approximate a "reasonably sharp pen" with a capacitive screen. The current iPhone recognizes about 1/8" (30 pixel) radius as a touch and rejects anything less as noise.

I assume that this is done in software because that's the way Apple likes to do things.

If so, then Apple could change the radius (sharpness) with software.

That said, you can get acceptable sharpness for handwriting and signatures on all the current iPhone screens using a home-made stylus and an app like Sketches.

Dick

"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #15 of 69
Thanks ai. For sure one of the most interesting articles these past days...man I can't wait for the tablet.

Btw, when you receive a delivery and you ar given this little pad to sign on acceptance dont you just wonder how this crappy distorted signature that ends up there could have been anyones....well, apple to the rescue then.
post #16 of 69
Quote:
Originally Posted by teckstud View Post

Do we really want to go back to this old technology? Pens? Stylus? Aren't our fingers good enough?

Well, I know you eat with your hands and all, but...
post #17 of 69
Quote:
Originally Posted by wizard69 View Post

This could be huge but I have to wonder about the tech used to detect the "pen". Will the tablet have a capacitive sensor for Touch and another to detect pen position? I just can't see a reasonably sharp pen being picked up by a capacitive sensor. I don't think much of the inferred pen from finger position approach, the virtual pen if you will.

In any event I expect we wil see lots of patent applications from Apple in the coming weeks as it readies the tablet roll out. Hopefully this is an indication of a truely innovative platform.


Dave

Yes Dave.

It's called the pogo stylus (and various other 3rd party solutions). And no, it won't be sharp. If anything, it will be "sharper" at least than the standard capacitive stylus but it certainly won't be as sharp as the average pen. It probably won't be sharp as even the old fashioned regular pen based stylus either.
post #18 of 69
Quote:
Originally Posted by teckstud View Post

Do we really want to go back to this old technology? Pens? Stylus? Aren't our fingers good enough?

Do you write with your fingers on paper? This only makes sense for a tablet device, as the larger surface can accommodate such an input method. And using a stylus for writing would be more natural than typing on a full sized touch keyboard, IMO, due to tactile reasons. Apple has already implemented this feature (somewhat) in its custom concierge devices for Apple Store checkouts. Wouldn't be surprised if this plays a major role in its upcoming tablet device.
post #19 of 69
Quote:
Originally Posted by anonymouse View Post

Well, I know you eat with your hands and all, but...

What do you eat with?
post #20 of 69
Quote:
Originally Posted by Olternaut View Post

Yes Dave.

It's called the pogo stylus (and various other 3rd party solutions). And no, it won't be sharp. If anything, it will be "sharper" at least than the standard capacitive stylus but it certainly won't be as sharp as the average pen. It probably won't be sharp as even the old fashioned regular pen based stylus either.

As I posted above, I believe apple can change the sharpness through software.

A better stylus could be designed with a sharper point, say 5-10 pixel radius. Stylus (pen) pressure could be measured and sent by BlueTooth.

The "Tablet" could detect the presence of the stylus and automatically switch into precision, single-touch mode (vs finger multi-touch mode),

*
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #21 of 69
Quote:
Originally Posted by iReality85 View Post

Do you write with your fingers on paper? This only makes sense for a tablet device, as the larger surface can accommodate such an input method. And using a stylus for writing would be more natural than typing on a full sized touch keyboard, IMO, due to tactile reasons. Apple has already implemented this feature (somewhat) in its custom concierge devices for Apple Store checkouts. Wouldn't be surprised if this plays a major role in its upcoming tablet device.

Actually, if you step back and think about it a smaller device is where you would want finer control, wouldn't you? A larger device you could scale or make the sensitivity different based on the application you are in. Why couldn't you use your finger and have a finer control? Think like Photoshop paintbrushes or pens, you can change the point size, I would imagine the same could be done on a tablet.
post #22 of 69
Quote:
Originally Posted by teckstud View Post

Do we really want to go back to this old technology? Pens? Stylus? Aren't our fingers good enough?

You write with your fingers?
My Android phone is the worst phone I've ever owned.
Reply
My Android phone is the worst phone I've ever owned.
Reply
post #23 of 69
Quote:
Originally Posted by Gazoobee View Post

To me this is the last kind of big mystery about the purported tablet from Apple.

Will there be a stylus? It's so *not* Apple to have a stylus, but the fancy patents they have that may allow them to do without one seem far-fetched.

If Apple can pull off a tablet that doesn't need a stylus to do ink input, it will go in the tech hall of fame. If they merely produce a tablet without a stylus and tell us we don't need ink input, there will be disappointment. If on the third hand, they produce a tablet with a gimmicky plastic stylus in a hole on the side it will be a similar letdown for many.

It will be real interesting to see what they do here.

It it includes any kind of writing or drawing functionality I think it will have to, unless you're Tekstud who uses his fingers. There may of course be more than one tablet, for different uses. A media player tablet won't need a stylus, but a full tablet computer probably would.
My Android phone is the worst phone I've ever owned.
Reply
My Android phone is the worst phone I've ever owned.
Reply
post #24 of 69
Quote:
Originally Posted by Dick Applebaum View Post

As I posted above, I believe apple can change the sharpness through software.

A better stylus could be designed with a sharper point, say 5-10 pixel radius. Stylus (pen) pressure could be measured and sent by BlueTooth.

The "Tablet" could detect the presence of the stylus and automatically switch into precision, single-touch mode (vs finger multi-touch mode),

*

Yes, I also expect apple to take advantage of software, in some fashion, when and if they implement a new standard for stylus design when the tablet debuts.
post #25 of 69
Quote:
Originally Posted by steviet02 View Post

Actually, if you step back and think about it a smaller device is where you would want finer control, wouldn't you? A larger device you could scale or make the sensitivity different based on the application you are in. Why couldn't you use your finger and have a finer control? Think like Photoshop paintbrushes or pens, you can change the point size, I would imagine the same could be done on a tablet.

A finger or virtual pen would probably be OK on a larger surface [than the iPhone] like a tablet.

The biggest disadvantage is that your finger(s) would block the point being drawn from view. This could be mitigated by displaying the ink (drawing points) offset from the finger(s)-- similar to the way the loupe offsets the finger when selecting text.

With a more natural implementation (than the current loupe) it would be fairly easy to become proficient writing with a finger or virtual pen.
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #26 of 69
Quote:
Originally Posted by steviet02 View Post

Actually, if you step back and think about it a smaller device is where you would want finer control, wouldn't you? Why couldn't you use your finger and have a finer control?

I don't know how you could get more point-accurate than using a fine-pointed stylus device. Maybe it's just me.


Quote:
Originally Posted by Dick Applebaum View Post

The biggest disadvantage is that your finger(s) would block the point being drawn from view.

This was the point I was hinting at in my post above. Plus, the tablet I'm sure will be high gloss. As if the current touch and iPhone don't attract enough smudges, I can only imagine what people will say when they have to write using their fingers and the entire tablet screen is left with nasty, oily smudges marks. Writing with a pen type device is natural for all of us, not to mention cleaner. Plus, consider what it would be like for women who have any type of long finger nail. They would have to write using their finger in a heavily horizontal position.
post #27 of 69
So I guess now all of those who lambasted pen/stylus input because the iPhone doesn't support it will suddenly have a change of heart?

Hmm? Windows Mobile never abandoned the stylus option due to accuracy it afforded etc.

Good to see Apple finally coming around again

Note: 3rd party (iPhone) capacitive stylus solutions not withstanding.
"Why iPhone"... Hmmm?
Reply
"Why iPhone"... Hmmm?
Reply
post #28 of 69
Quote:
Originally Posted by teckstud View Post

Do we really want to go back to this old technology? Pens? Stylus? Aren't our fingers good enough?

Finger-tip sizes aren't standardized, stylus/pen tips are/can be.

When accuracy is paramount, there simply is no substitute for a pen/stylus.
"Why iPhone"... Hmmm?
Reply
"Why iPhone"... Hmmm?
Reply
post #29 of 69
This is great news! Hopefully Apple will allow PEN input along with FINGER input in upcoming tablets or iphones.

Finger input is fine for dialing the phone, and inputing just a little bit of text, but have you ever tried to type a couple of pages on the iPhone?

Worse yet, have you tried to do some serious graphics work on the iPhone. Awful. Not much better than finger paintings.

Writing with a pen is a natural way to input text. We have been doing it all our life. As is using a pen for graphics.

I hope Apple improves the options for BOTH finger and pen input.

I currently have a TABLET PC which I friggin love for doing graphics. Corel Painter is awesome on it. Too bad it is a PC with WinXP which sux. I would love to see a MacTablet with full OSX and pen input for serious graphics work NOT a 10" iPhone with only finger gesture input.
post #30 of 69
I know it is bad taste to link to a rival site, but Electronista has posted a version of the patent that includes a block diagram of the software and APIs. This may give some additional insight into Apple's plans.

http://www.electronista.com/articles...t.finger.only/

*
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
"...The calm is on the water and part of us would linger by the shore, For ships are safe in harbor, but that's not what ships are for."
- Michael Lille -
Reply
post #31 of 69
Quote:
Originally Posted by wizard69 View Post

This could be huge but I have to wonder about the tech used to detect the "pen". Will the tablet have a capacitive sensor for Touch and another to detect pen position? I just can't see a reasonably sharp pen being picked up by a capacitive sensor.

The one I have had both capacitive and digitizer.

Quote:
Originally Posted by HardyNH View Post

I just got a Wacom graphics tablet and played with inkwell for a while, and it was terrible. My 11-year-old Newton that still sits on my desk has far better HWR than inkwell/Wacom combination. So yeah, it needs some updating.

Yah, Inkwell not so hot for me either.
post #32 of 69
Quote:
Originally Posted by knightlie View Post

You write with your fingers?

Him? Probably. Fingerpainting is about his speed.
post #33 of 69
Quote:
Originally Posted by iReality85 View Post

I don't know how you could get more point-accurate than using a fine-pointed stylus device. Maybe it's just me.

That's my point. If you are on a smaller device you would want much finer control, but we don't have that with the iPhone. Why is it necessary for a tablet?
post #34 of 69
This tablet is looking more-and-more like a device I want to have.
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #35 of 69
Quote:
Originally Posted by steviet02 View Post

That's my point. If you are on a smaller device you would want much finer control, but we don't have that with the iPhone. Why is it necessary for a tablet?

Are you serious? Why would I want to pull out a pen just to use my phone?

The iPhone isnt for handwriting or drawing, its a fraking phone. If I am using a tablet in class I can draw diagrams and complex calculations with ease with a capacitance pen. How exactly would I do that with my fingers? Why would I do that with my phone? Note, you can get a pen for use with the iPhone.

The pen isnt required for the tablet, its an option for doing more precise work that people with tablets often do. I bet you said the virtual keyboard could never be a suitable replacement for a physical keyboard.
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #36 of 69
Quote:
Originally Posted by Ireland View Post

This reminds me of Apple investigating superior mice.

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #37 of 69
I'm glad to see them exploring this. A stylus, as an option, is a great thing. I've been thinking about this device and for me, unless it also includes 2048 levels or better of pressure sensitivity, it's not going to be very useful. With pressure sensitivity and a stylus, you could then paint and touch up photos in Photoshop, which is what this thing would be perfect for. Unfortunately, the iPhone OS is not up to the task, so unless it's running a slimmed version of OS X, I see this as sort of a dud out of the gates. Maybe eventually, as it matures, it will get more powerful and useful. We'll see.
post #38 of 69
Quote:
Originally Posted by solipsism View Post

Are you serious? Why would I want to pull out a pen just to use my phone?

The iPhone isnt for handwriting or drawing, its a fraking phone. If I am using a tablet in class I can draw diagrams and complex calculations with ease with a capacitance pen. How exactly would I do that with my fingers? Why would I do that with my phone? Note, you can get a pen for use with the iPhone.

The pen isnt required for the tablet, its an option for doing more precise work that people with tablets often do. I bet you said the virtual keyboard could never be a suitable replacement for a physical keyboard.

Yeah right, cause these tablets will be priced for students, riiiighhgggt.... BTW, isn't this a media tablet not a document based tablet? thats all I've been hearing about this device lately.
post #39 of 69
Quote:
Originally Posted by solipsism View Post

This tablet is looking more-and-more like a device I want to have.

Now that's the understatement of the decade. What Apple device isn't?
post #40 of 69
Quote:
Originally Posted by Olternaut View Post

Yes Dave.

It's called the pogo stylus (and various other 3rd party solutions). And no, it won't be sharp. If anything, it will be "sharper" at least than the standard capacitive stylus but it certainly won't be as sharp as the average pen. It probably won't be sharp as even the old fashioned regular pen based stylus either.

If I'm going to be using a pen to supplement touch input it had better be able to zero in tighter than my finger otherwise I might as well use my finger. The whole point, for me anyways, of a stylus is to get fine input control you can't get with a finger.


Dave
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Future Apple Hardware
AppleInsider › Forums › Mac Hardware › Future Apple Hardware › Apple looks to improve pen-based input on tablet touchscreens