Lets brainstorm the ultimate input device...

[peteman]

The mac is 20 years old and we (most of us anyways) are still using QWERTY keyboards and mice, despite their tendancy to cause carpel tunnel syndrome and their inefficiencies. What improvements could be made?

I'll start with an idea for the pointing device:

If Mac OS X had image recognition software that could run in real time and analyze video of the users eyes taken from an iSight, it might be possible to figure out where they were looking on the screen. If this could be done to near pixel level accuracy, then wallah, throw away your mouse, put both hands on the keyboard and work more efficiently. When you need to click on something, just look at it and blink, or press the foot pedal, or a special "click" key on the keyboard or whatever - but no need to interupt your work by taking your hands off the keyboard.

On a bit of a technical note, I imagine one way to acheive this (on single monitor systems anyways) is to display a colored dot in each corner of the screen and then look at where the reflection of those four dots are in the image of the pupils of the viewer. Find the center of the pupil relative to the dots, compare the coordinates to an interpolated calibration set of measurements and you've found where they are looking. Now accuracy, speed, processor load, and dealing with multiple monitors or multiple viewers could be issues. But wouldn't this beat point and click?

[Kurt]

How about a Firewire port installed directly in the back of the neck and connected to the brain. That way we wouldn't have to move or even have a monitor. Everything could be fed directly into the brain and our thoughts would control the computer.:lol: :lol:

[Ebby]

Kurt, we think alike. Only mine was the other way around. Instead of using a mouse and keyboard for FPS games, have a direst neural connection. No one could beat me! Muhahaha! :devil: There are very simple prototypes out there. Basically they are slow and limited, but I believe the technology will be perfected within 20 years. Then you can sign me up.

[Ebby]

messed up again! :mad:

[KeilwerthReborn]

Originally posted by peteman
Wouldn't this beat point and click?

No.

[reynard]

A firewire port on your neck to connect your brain to the computer??? Are you serious?? Firewire your brain to a computer--thats beyond ridiculous. You would need a fiber optic connection at least. Well, there is firewire 1600 and 3200 coming out. They may provide enough bandwidth for some people's thoughts.

Peteman, in case you were serious about your ultimate "optical mouse", do you have the eye discipline to not look at anything but the pixel of choice? Its like when a buxom woman walks into a room. Dont look at her boobs, dont look at her boob, dont look at her boobs...doh!

[shetline]

Ultimate input device?

Two words: anal joystick.

[ast3r3x]

Originally posted by shetline
Ultimate input device?

Two words: anal joystick.

I can see it being advertised...clicking also strengthens your PC muscle! ;)

They are testing devices on monkey that you can control the cursor with your brain, once this is refined and any hazards work out this will be awesome...although I'd guess you need surgery to have it work and that would suck.

[peteman]

Originally posted by reynard

Peteman, in case you were serious about your ultimate "optical mouse", do you have the eye discipline to not look at anything but the pixel of choice? Its like when a buxom woman walks into a room. Dont look at her boobs, dont look at her boob, dont look at her boobs...doh!

Hmm, put it this way: Do you ever click on something that you are not looking at? Sure when you are scrolling or doing repetitive clicks maybe, but for general mousing I bet you'd have a hard time navigating a GUI with a mouse or any other input device while your focusing on the buxom woman who just entered the room.

[Scott]

Originally posted by peteman
...

If Mac OS X had image recognition software that could run in real time and analyze video of the users eyes taken from an iSight, it might be possible to figure out where they were looking on the screen. If this could be done to near pixel level accuracy, then wallah, throw away your mouse, put both hands on the keyboard and work more efficiently. When you need to click on something, just look at it and blink, or press the foot pedal, or a special "click" key on the keyboard or whatever - but no need to interupt your work by taking your hands off the keyboard.

...

They already have technology to do that. I think it uses a laser. But where you're looking is not where you want you mouse to go. People blink all the time too so that mouse would click all the time.

[Ebby]

Originally posted by Scott
They already have technology to do that. I think it uses a laser. But where you're looking is not where you want you mouse to go. People blink all the time too so that mouse would click all the time. Last I heard that technology was in cameras to focus on the subject the photographer is looking at, but I never saw it used with a computer before.

[tfworld]

Anyone ever read the Reality Dysfunction, Neutonium Alchemist, and Naked God books? They have a computer built into their brains! It works with wireless everything. Really cool.

If not that then maybe like on Andromeda where you can jack your brain into the AI?

[Kickaha]

How about a system that watches where you point your finger instead?

We have just such a system in my research group, implemented initially by yours truly.

Oh, and it's MacOS X only. The Windows graphics architecture won't let you accomplish the critical part. ;)

A standard FireWire camera, a simple little app, and a brightly colored thimble is all that's needed, no calibration necessary, anyone can use it.

I gave a demo at UIST'03 in November in Vancouver, and people couldn't believe how simple it was to use... and that was on my 15" 1.25GHz PowerBook G4.

On a dual G5, it hardly affects the rest of the system at all.

[Kurt]

Originally posted by reynard
A firewire port on your neck to connect your brain to the computer??? Are you serious?? Firewire your brain to a computer--thats beyond ridiculous. You would need a fiber optic connection at least. Well, there is firewire 1600 and 3200 coming out. They may provide enough bandwidth for some people's thoughts.



reynard,
Don't you understand sarcasm? That's what the :lol: meant at the end.

[Krassy]

Originally posted by Kurt
reynard,
Don't you understand sarcasm? That's what the :lol: meant at the end.

speech reecognition in a superb form.
i just want to lean back and have a good time... i could use my hands and eyes for other stuff... but: the keyboard and mouse has to stay for the simple reason that when i'm on the phone and want to make some notes i don't like to make a phone-conference with my computer and customer *lol*

edit: i thought about this: there has to be an application to automatically make notes from phonecalls - for a start this can write down and save what is said with a speech recognition that can find out which voice is yours and which is the one of the other person.

also - the applications have to be changed... instead of clicking somewhere the input fields need an identifier which can be said as a placement command. hmmm or we need a combination of a pointing/grabing device and speech recognition to replace the keyboard...

whatever - i like the pointing device in minority report... we also need bigger screens... do we say: dual-3d-projector or something

[myahmac]

For a brain hookup, you wouldnt have to use such a huge connection!! To connect with the correct input nodes of the brain, it would not require more than USB1.1. The idea being you wouldnt need a gui. Just file informations is stored like memories. Problem is wed have to find the correct way to interface without causing a reject around the tissue area. But that already being worked on. The thing is is the implant would actualy have to be dynamic. after implantation it would have to have the cells create connections to understand incoming data. For example if i say the word fast, some of you think plane car rabbit etc. its all diferent yet they mean the same. C ?:err:

[Aphelion]

Originally posted by Scott
... People blink all the time too so that mouse would click all the time.

Not a blink, but a wink. That way you would have a left wink and a right wink. Of course Steve would never approve of a dual wink input device.

[FormatC2]

How about an electronic vagina? That way, Mac users can really show their computers how they feel about them.

Not to leave the ladies out, Apple can build something to _accommodate_ them too.

:lol:

[costique]

Originally posted by reynard
Firewire your brain to a computer--thats beyond ridiculous. You would need a fiber optic connection at least. Well, there is firewire 1600 and 3200 coming out. They may provide enough bandwidth for some people's thoughts.You must be joking. For most people's thought bandwidth a 9600 baud serial cable is overkill.

[Kickaha]

Originally posted by Krassy
whatever - i like the pointing device in minority report...

*AHEM*

See my earlier post. ;)

[cubist]

How about a spaceball which works with all software and doesn't cost a fortune? It gives you movement plus twist, lift and press.

[pooandwee]

They already use a similar mechanism to the one you describe to measure the area value ($) of advertising on a web page. So, in the same way that greater value is place on advertising space on the leading page of a newspaper spread (opening right to left the leading page is the right page), advertisers would be able to place greater value on a said area of a web page.

I don't think it measures pixel for pixel or allows you to click, it just sences the region on the screen that you are looking at. It has a camera that focuses on your pupil and monitors its movement though. I suppose this could be refined to point and click device but it would need to get pretty precise if you wanted to use PhotoShop with it.... Although I can definitely see a practical application for this in the impaired movement community. I saw a boy in a wheelchair with one of those foot and a half log poles with the ball on the end for typing on a keyboard because he was unable to type conventionally. This probably wouldn't have helped him in particular but there must be people this tech could help. Dual arm amputees for instance. I don't know.

Anyway.... I was thinking about this exact same situation about 6 months ago, what would be a killer input device. I was thinking more along the keyboard direction though.

I was looking at my 17" PowerBook and then thinking about Steven Spielberg (or someone who has something to do with video editing etc) using it to edit video. Then I was thinking about these musicians and DJs that I heard about that do all their editing, mixing, recording etc on a Mac. Then I was thinking about the myriad of other applications the Mac has and I realised that the keyboard is a big fat waste of time and space.

What if you could replace the keyboard with a touchscreen. So, you're editing you movie in FCP and the "screenboard" (it would need a better name I think) shows a scrubbing wheel and play/pause/fast forward buttons, sound levels when a track with audio is selected etc etc etc. If you are entering titles it will revert to a traditional qwerty layout on the screen until you are back in editing mode. Then you want to work in SoundTrack so the screenboard changes to an audio mixing desk. If you are a beat mixing DJ, it shows the 2 tracks currently playing each on a turntable that you can scratch etc. If you are using PhotoShop it could have all of your pallettes on it to remove the desktop cluter, or it could be a drawing tablet.

The screen resolution wouldn't need to be that great and not even color. It would need a pretty kick ass touch screen though.... or you could lay it over a similar surface to the mouse on the PowerBook....Just have an elactrostatic field and measure the disturbances.... It would need to support multiple disturbances though in order to make it useful.

So, what do you think??

I don't know if this is economically feasible or even possible.... I just think that it's a not-bad-idea.

[Kickaha]

*ahem*

http://rockfish-cs.cs.unc.edu/pubs/TR04-005.pdf

I've got to whip up a movie to provide a link to...


And yes, some Apple folks have seen it. (But we have the patent filing. ;) )

[Wrong Robot]

Originally posted by Kickaha
*ahem*

http://rockfish-cs.cs.unc.edu/pubs/TR04-005.pdf

I've got to whip up a movie to provide a link to...


And yes, some Apple folks have seen it. (But we have the patent filing. ;) )
http://www.d.umn.edu/~rmyers1/homer_mad.gif
Patent Pending
Patent Pending
Patent Pending

[Kickaha]

Yeah, I know it's lame, but the university requires it. :\

[SonOfSylvanus]

Originally posted by Aphelion
Not a blink, but a wink. That way you would have a left wink and a right wink. Of course Steve would never approve of a dual wink input device.

Heh Heh

[speedfreak]

Chech this out to try the camera interface. http://www.ToySight.com

[Kickaha]

Yup. It's a cute app. :)

Ours runs the whole UI. ;)

[talksense101]

Make the iMac flat plasma display into a touch sensitive device. Add in movement recognition via iSight. Add voice recognition for documents.

It will be great to walk up to your computer and use it like a kiosk and talk to it.

[Kickaha]

Don't need the touch sensitive panel. Just an iSight. (or other FireWire camera).

Can't help you with the voice recog much... other than to suggest a hefty investment in scripting triggered by the current speech recognition.

[ast3r3x]

How about you post a movie and a link to this software so that when I get my iSigh I can try this out!

[Kickaha]

Movie I can do.

A copy of the app... almost certainly not, unfortunately. I don't own it, nor do I have copyright on it. UNC does. We plan on making it publicly available 'at some point'. You'd have to talk to the research lead for that one though - and he's cranky. :)

[concentricity]

Legos.

[Wrong Robot]

Kickaha, you're terrible, I really want this app you're talking about. My iSight is just begging to be used more often.:p

[Vox Barbara]

Originally posted by reynard
Firewire your brain to a computer--thats beyond ridiculous. You would need a fiber optic connection at least. ...

Actually, everything that is contained in a sole brain is trivial anyway, so what?

think different at least, well, at least ...

[OverToasty]

Originally posted by peteman
The mac is 20 years old and we (most of us anyways) are still using QWERTY keyboards and mice, despite their tendancy to cause carpel tunnel syndrome and their inefficiencies. What improvements could be made?


The "Do-What-I-Want-Not-What-I-Tell-You" interface is probably the next big thing ... no matter how dumb-ass your input might be, the systems Baysien analysis of your nonsense usually selects the right option ... you lazy-no-good-RTFM-mofo-you :lol: