Lets brainstorm the ultimate input device...

Posted:
in Future Apple Hardware edited January 2014
The mac is 20 years old and we (most of us anyways) are still using QWERTY keyboards and mice, despite their tendancy to cause carpel tunnel syndrome and their inefficiencies. What improvements could be made?



I'll start with an idea for the pointing device:



If Mac OS X had image recognition software that could run in real time and analyze video of the users eyes taken from an iSight, it might be possible to figure out where they were looking on the screen. If this could be done to near pixel level accuracy, then wallah, throw away your mouse, put both hands on the keyboard and work more efficiently. When you need to click on something, just look at it and blink, or press the foot pedal, or a special "click" key on the keyboard or whatever - but no need to interupt your work by taking your hands off the keyboard.



On a bit of a technical note, I imagine one way to acheive this (on single monitor systems anyways) is to display a colored dot in each corner of the screen and then look at where the reflection of those four dots are in the image of the pupils of the viewer. Find the center of the pupil relative to the dots, compare the coordinates to an interpolated calibration set of measurements and you've found where they are looking. Now accuracy, speed, processor load, and dealing with multiple monitors or multiple viewers could be issues. But wouldn't this beat point and click?
«1

Comments

  • Reply 1 of 35
    kurtkurt Posts: 225member
    How about a Firewire port installed directly in the back of the neck and connected to the brain. That way we wouldn't have to move or even have a monitor. Everything could be fed directly into the brain and our thoughts would control the computer.
  • Reply 2 of 35
    ebbyebby Posts: 3,110member
    Kurt, we think alike. Only mine was the other way around. Instead of using a mouse and keyboard for FPS games, have a direst neural connection. No one could beat me! Muhahaha! There are very simple prototypes out there. Basically they are slow and limited, but I believe the technology will be perfected within 20 years. Then you can sign me up.
  • Reply 3 of 35
    ebbyebby Posts: 3,110member
    messed up again!
  • Reply 4 of 35
    Quote:

    Originally posted by peteman

    Wouldn't this beat point and click?



    No.
  • Reply 5 of 35
    reynardreynard Posts: 160member
    A firewire port on your neck to connect your brain to the computer??? Are you serious?? Firewire your brain to a computer--thats beyond ridiculous. You would need a fiber optic connection at least. Well, there is firewire 1600 and 3200 coming out. They may provide enough bandwidth for some people's thoughts.



    Peteman, in case you were serious about your ultimate "optical mouse", do you have the eye discipline to not look at anything but the pixel of choice? Its like when a buxom woman walks into a room. Dont look at her boobs, dont look at her boob, dont look at her boobs...doh!
  • Reply 6 of 35
    shetlineshetline Posts: 4,695member
    Ultimate input device?



    Two words: anal joystick.
  • Reply 7 of 35
    ast3r3xast3r3x Posts: 5,012member
    Quote:

    Originally posted by shetline

    Ultimate input device?



    Two words: anal joystick.




    I can see it being advertised...clicking also strengthens your PC muscle!



    They are testing devices on monkey that you can control the cursor with your brain, once this is refined and any hazards work out this will be awesome...although I'd guess you need surgery to have it work and that would suck.
  • Reply 8 of 35
    Quote:

    Originally posted by reynard



    Peteman, in case you were serious about your ultimate "optical mouse", do you have the eye discipline to not look at anything but the pixel of choice? Its like when a buxom woman walks into a room. Dont look at her boobs, dont look at her boob, dont look at her boobs...doh!




    Hmm, put it this way: Do you ever click on something that you are not looking at? Sure when you are scrolling or doing repetitive clicks maybe, but for general mousing I bet you'd have a hard time navigating a GUI with a mouse or any other input device while your focusing on the buxom woman who just entered the room.
  • Reply 9 of 35
    scottscott Posts: 7,431member
    Quote:

    Originally posted by peteman

    ...



    If Mac OS X had image recognition software that could run in real time and analyze video of the users eyes taken from an iSight, it might be possible to figure out where they were looking on the screen. If this could be done to near pixel level accuracy, then wallah, throw away your mouse, put both hands on the keyboard and work more efficiently. When you need to click on something, just look at it and blink, or press the foot pedal, or a special "click" key on the keyboard or whatever - but no need to interupt your work by taking your hands off the keyboard.



    ...




    They already have technology to do that. I think it uses a laser. But where you're looking is not where you want you mouse to go. People blink all the time too so that mouse would click all the time.
  • Reply 10 of 35
    ebbyebby Posts: 3,110member
    Quote:

    Originally posted by Scott

    They already have technology to do that. I think it uses a laser. But where you're looking is not where you want you mouse to go. People blink all the time too so that mouse would click all the time.



    Last I heard that technology was in cameras to focus on the subject the photographer is looking at, but I never saw it used with a computer before.
  • Reply 11 of 35
    Anyone ever read the Reality Dysfunction, Neutonium Alchemist, and Naked God books? They have a computer built into their brains! It works with wireless everything. Really cool.



    If not that then maybe like on Andromeda where you can jack your brain into the AI?
  • Reply 12 of 35
    kickahakickaha Posts: 8,760member
    How about a system that watches where you point your finger instead?



    We have just such a system in my research group, implemented initially by yours truly.



    Oh, and it's MacOS X only. The Windows graphics architecture won't let you accomplish the critical part.



    A standard FireWire camera, a simple little app, and a brightly colored thimble is all that's needed, no calibration necessary, anyone can use it.



    I gave a demo at UIST'03 in November in Vancouver, and people couldn't believe how simple it was to use... and that was on my 15" 1.25GHz PowerBook G4.



    On a dual G5, it hardly affects the rest of the system at all.
  • Reply 13 of 35
    kurtkurt Posts: 225member
    Quote:

    Originally posted by reynard

    A firewire port on your neck to connect your brain to the computer??? Are you serious?? Firewire your brain to a computer--thats beyond ridiculous. You would need a fiber optic connection at least. Well, there is firewire 1600 and 3200 coming out. They may provide enough bandwidth for some people's thoughts.







    reynard,

    Don't you understand sarcasm? That's what the meant at the end.
  • Reply 14 of 35
    krassykrassy Posts: 595member
    Quote:

    Originally posted by Kurt

    reynard,

    Don't you understand sarcasm? That's what the meant at the end.




    speech reecognition in a superb form.

    i just want to lean back and have a good time... i could use my hands and eyes for other stuff... but: the keyboard and mouse has to stay for the simple reason that when i'm on the phone and want to make some notes i don't like to make a phone-conference with my computer and customer *lol*



    edit: i thought about this: there has to be an application to automatically make notes from phonecalls - for a start this can write down and save what is said with a speech recognition that can find out which voice is yours and which is the one of the other person.



    also - the applications have to be changed... instead of clicking somewhere the input fields need an identifier which can be said as a placement command. hmmm or we need a combination of a pointing/grabing device and speech recognition to replace the keyboard...



    whatever - i like the pointing device in minority report... we also need bigger screens... do we say: dual-3d-projector or something
  • Reply 15 of 35
    myahmacmyahmac Posts: 222member
    For a brain hookup, you wouldnt have to use such a huge connection!! To connect with the correct input nodes of the brain, it would not require more than USB1.1. The idea being you wouldnt need a gui. Just file informations is stored like memories. Problem is wed have to find the correct way to interface without causing a reject around the tissue area. But that already being worked on. The thing is is the implant would actualy have to be dynamic. after implantation it would have to have the cells create connections to understand incoming data. For example if i say the word fast, some of you think plane car rabbit etc. its all diferent yet they mean the same. C ?
  • Reply 16 of 35
    Quote:

    Originally posted by Scott

    ... People blink all the time too so that mouse would click all the time.



    Not a blink, but a wink. That way you would have a left wink and a right wink. Of course Steve would never approve of a dual wink input device.
  • Reply 17 of 35
    How about an electronic vagina? That way, Mac users can really show their computers how they feel about them.



    Not to leave the ladies out, Apple can build something to _accommodate_ them too.



  • Reply 18 of 35
    costiquecostique Posts: 1,084member
    Quote:

    Originally posted by reynard

    Firewire your brain to a computer--thats beyond ridiculous. You would need a fiber optic connection at least. Well, there is firewire 1600 and 3200 coming out. They may provide enough bandwidth for some people's thoughts.



    You must be joking. For most people's thought bandwidth a 9600 baud serial cable is overkill.
  • Reply 19 of 35
    kickahakickaha Posts: 8,760member
    Quote:

    Originally posted by Krassy

    whatever - i like the pointing device in minority report...



    *AHEM*



    See my earlier post.
  • Reply 20 of 35
    cubistcubist Posts: 954member
    How about a spaceball which works with all software and doesn't cost a fortune? It gives you movement plus twist, lift and press.
Sign In or Register to comment.