Apple's dynamic user interface would adapt to user's proximity, allowing control from close & far

Posted:
in Future Apple Hardware edited March 2014
Apple has shown continued interest in a new input method that would use advanced sensors to detect gestures, movements, location and distance from a user, allowing a person to easily transition from physical input up close to gesture controls from afar.

Patent
Certain user interface elements could enlarge, simplify or disappear as users step further away from their device.


The company's pursuit of a user-sensing, highly interactive computer system was detailed in a patent continuation published by the U.S. Patent and Trademark Office on Thursday. Entitled "Computer User Interface System and Methods," it describes sensors that could measure everything from the presence of a user to their location in the room to any gestures they may perform.

Perhaps most interesting is the filing's mention of a "user proximity context" that would modify a system's user interface based on how close the person is to the device.

The system described in Apple's patent filing would allow users to easily read and control a device from either up close or afar, dynamically adapting to the person's distance."An appearance of information displayed by the computer may be altered or otherwise controlled based on the user proximity context," the filing reads. "For example, a size and/or a content of the information displayed by the computer may be altered or controlled."

With this system, elements on the display, including text size, could be automatically increased or decreased based on how close the user is to the screen. The user interface could also adapt and change based on where a person is in relation to the device.

In another example given by Apple, a device could automatically transfer input controls from a mouse to hand gestures as a person steps away. Yet another situation presented by Apple describes a device increasing its screen brightness based on how far away a user is, allowing them to see the display more easily from a distance.

Apple's concept is in some ways a more advanced and interactive version of the "parallax effect" the company introduced with its iOS 7 mobile operating system upgrade last year. That feature uses the motion sensors in an iPhone or iPad to make background wallpapers move, giving users the illusion that their device is a sort of "window" into a virtual three-dimensional world.



The concepts presented in Apple's filing are not entirely new, though they do suggest that Apple could be interested in offering three-dimensional input and interactivity in future devices. Head tracking, motion gestures and user identification already exist in a number of products on the market, most notably Microsoft's Kinect for its Xbox gaming platform.

The application disclosed this week was filed with the USPTO in August of 2013, and is a continuation of a filing from 2008. It is credited to inventors Aleksandar Pance, David R. Falkenberg, and Jason M. Medeiros.
«134

Comments

  • Reply 1 of 74
    MacProMacPro Posts: 19,718member
    Got to love these filings. R&D for Google and Scamsung for free.
  • Reply 2 of 74
    tylerk36tylerk36 Posts: 1,037member

    2034 Apple announces the new iPhone 18Cr.  Size of a dime and fits in the frontal lobe.  Apple says it won't cook your brain.  You can do all your information via thought and see everything via daytime illusions when your eye lids are closed.

  • Reply 3 of 74
    This makes sense. The "iWatch" is probably a device that will be worn when interacting with the computer using this input method.
  • Reply 4 of 74
    Got to love these filings. R&D for Google and Scamsung for free.

    LoL. That's what I was going to say. Samsung probably has someone monitoring the patent office release to see what they can "borrow" from Apple.
  • Reply 5 of 74
    Quote:

    Originally Posted by bennettvista View Post



    This makes sense. The "iWatch" is probably a device that will be worn when interacting with the computer using this input method.

     

    More likely to be for a TV where the UI will respond to where the viewers are in the room.

  • Reply 6 of 74
    These are all stepping stones to an eventual direct connection between the mind and digital devices.

    Interface progression:

    Input (visual and sound --> non physical communication)
    keyboard --> mouse --> gesture --> vocal (Siri) ---> sub vocal --> direct connection (wired initially, but eventually non-wired)

    Output (visual and sound --> non physical communication)
    monitor ---> increasing monitor display resolution ---> 3D displays "Oculus Rift" (increasing resolution and speed) --> holographic 3D --> direct connection (wired initially, but eventually non-wired)

    Apple needs to get ahead of this in developing their next generation operating systems.
  • Reply 7 of 74
    Quote:

    Originally Posted by tylerk36 View Post

     

    2034 Apple announces the new iPhone 18Cr.  Size of a dime and fits in the frontal lobe.  Apple says it won't cook your brain.  You can do all your information via thought and see everything via daytime illusions when your eye lids are closed.


     

    I know you're joking, but "mind controlled" interfaces are a real thing and getting better and better as time goes. I think we should skip the "gestures in the air" and go straight to this technology.

     

    The Muse headband ( http://www.interaxon.ca/muse/ ) will ship next May for $299, and it's the smallest of its kind. Sure it's still clunky and its applications are not straightforward yet, but these will get smaller and smaller and more software will be written for it (the SDK is open source).

     

  • Reply 8 of 74
    pmzpmz Posts: 3,433member
    Quote:

    Originally Posted by tylerk36 View Post

     

    2034 Apple announces the new iPhone 18Cr.  Size of a dime and fits in the frontal lobe.  Apple says it won't cook your brain.  You can do all your information via thought and see everything via daytime illusions when your eye lids are closed.


    Didn't you get the memo? iPhones are getting bigger not smaller.

     

    I'll see if I can get you a copy of that memo.

  • Reply 9 of 74
    tylerk36tylerk36 Posts: 1,037member

    The year is?  An option of a large screen TV iPhone you carry around in a shopping cart can be yours.  It is covered in your own skin so you can say you have an implant.

  • Reply 10 of 74
    gatorguygatorguy Posts: 24,176member
    Got to love these filings. R&D for Google and Scamsung for free.

    At the pace Google is filing patents it may be some of their competitors and tech neighbors keeping an eye on what they're working on via the patent office. I had no idea until this morning that Google now controls 51,000 patents and patent applications!! To put that in perspective they had only bothered to get 38 patents up thru 2007, depending more on trade secrets than the patent office.
    http://www.technologyreview.com/news/521946/googles-growing-patent-stockpile/
  • Reply 11 of 74
    This is an interesting idea that, yes, will probably get copied by other companies in some shape or form. I remember seeing that Kinect video a few years ago, same basic idea except you had to wear glasses with IR lights on either side for it to track you.
  • Reply 12 of 74
    pmzpmz Posts: 3,433member

    In all of these "gesture" related patents we've seen for so long...I still don't understand the benefit or positive user experience gleaned from waving your hand around in the air. Seems to make about as much sense as a MacBook with a touch screen...holding your arm out gets old in about 20 seconds.

  • Reply 13 of 74
    mhiklmhikl Posts: 471member

    MS tried this, dancing before one's set.

    Apple, plodding Apple, designs behind the scene, waiting panther-like to pounce and amaze the world.

    Talk is cheap.

    Results herald the conjoined twins of time and timing.

  • Reply 14 of 74
    melgrossmelgross Posts: 33,510member
    Got to love these filings. R&D for Google and Scamsung for free.

    Yeah. I'm getting a little tired of Apple as of late. While they seem to do a lot of interesting research, they also seem to rarely do much with it. They appear to be to afraid to actually take a chance. And for those who think that's Cook's problem, it's not. SJ wasn't any better.

    I'd like to see a product (or several) from them that incorporates a number of these inventions, even if they're not entirely ready for prime time. Apple could state that they are "hobbies", or experimental' or whatever. Just DO something! I understand Apple's penchant for only releasing products that they think are perfected (though often we find they are not). But Google has no problem with releasing things that aren't, and doing well with them.

    Apple has lost the belief that they are innovating. Whether true or not, it's one reason the stock is where it is. So it takes them years to come out with a watch. Maybe it will be great, and maybe it will bomb. But meanwhile, give us SOMETHING! There's no reason it took so many years before we got API's for handheld controllers, for example. Why weren't they out in 2008? Think of how things could have been different.

    We're talking about saphirre screens and Liquidmetal on Seeking Alpha. I'm wary of either. While both sound nice, the costs will be significant, and what real benefit will there be? Yes, saphirre is less scratchable. But Gorilla Glass is already pretty hard to scratch. Less breakable? Yes, but better design would go a long way there too. As it is, the 5, 5S and 5C are all much better than the 4 and 4S in that regard. Liquidmetal? What real benefit will that give us, other than bragging rights?

    I'd rather see some of these innovative UI inventions come out, even if Apple makes them optional. And how about finally doing something with Siri? What have they don't with it these past few years? Hardly anything!
  • Reply 15 of 74
    Quote:

    Originally Posted by melgross View Post





    Yeah. I'm getting a little tired of Apple as of late. While they seem to do a lot of interesting research, they also seem to rarely do much with it. They appear to be to afraid to actually take a chance. And for those who think that's Cook's problem, it's not. SJ wasn't any better.



    I'd like to see a product (or several) from them that incorporates a number of these inventions, even if they're not entirely ready for prime time. Apple could state that they are "hobbies", or experimental' or whatever. Just DO something! I understand Apple's penchant for only releasing products that they think are perfected (though often we find they are not). But Google has no problem with releasing things that aren't, and doing well with them.



    Apple has lost the belief that they are innovating. Whether true or not, it's one reason the stock is where it is. So it takes them years to come out with a watch. Maybe it will be great, and maybe it will bomb. But meanwhile, give us SOMETHING! There's no reason it took so many years before we got API's for handheld controllers, for example. Why weren't they out in 2008? Think of how things could have been different.



    We're talking about saphirre screens and Liquidmetal on Seeking Alpha. I'm wary of either. While both sound nice, the costs will be significant, and what real benefit will there be? Yes, saphirre is less scratchable. But Gorilla Glass is already pretty hard to scratch. Less breakable? Yes, but better design would go a long way there too. As it is, the 5, 5S and 5C are all much better than the 4 and 4S in that regard. Liquidmetal? What real benefit will that give us, other than bragging rights?



    I'd rather see some of these innovative UI inventions come out, even if Apple makes them optional. And how about finally doing something with Siri? What have they don't with it these past few years? Hardly anything!



    This post of yours Mel, would be perfect for sites such as CNet or ZDNet, where Apple never did anything of significance. I'm very surprised to see it here.

    Keep in mind that between 2001 (iPod) and 2007 (iPhone) there's a 6 year gap. Ground breaking, trend setting innovation takes time. But it also has to be released at the right time, when the technology supporting it is ripe.  Case in point, tablet computers. Wasn't Microsoft and partners trying to get them in people's hands way before there ever was an iPad?

    Just because Apple won't release a beta product (Google Glass) it doesn't mean that Jony Ive, Craig Federighi, Eddy Cue and company are sitting around downing brewskies and high fiving each other.

  • Reply 16 of 74
    Quote:

    Originally Posted by digitalclips View Post



    Got to love these filings. R&D for Google and Scamsung for free.

     

    While reading this, did anyone else hear "Money For Nothing" (Dire Straits) playing in their head?

  • Reply 17 of 74
    gatorguy wrote: »

    Have you read it? It says that half of these patents were acquired through MM, and the rest was a rushed job:
    Google must have understood how weak its own position was. When Jobs announced the iPhone, Google had earned only 38 patents since its founding. Thirty-eight! To equal Apple’s phone, Google rushed to release Android, relatively old software that Google had acquired. It is open source, and was poorly defended with intellectual property.

    From then on, Google wouldn’t keep product features or new ideas as trade secrets anymore; its researchers, business managers, and advertising experts would work with a rotating cast of outside attorneys to patent whatever ideas they could. Only such a dogged focus can explain the exponential increase in Google patents.

    My glass is always half full, but I fail to see anything positive for Google with this article.
  • Reply 18 of 74
    gatorguygatorguy Posts: 24,176member
    philboogie wrote: »
    Have you read it?

    I always read the articles before I link'em. Google is doing the same thing Apple and IBM does. Flood the USPTO with patent applications, file for everything and anything, try to stake some claim to anything you think might be of value to your future business, understanding that a significant portion of whatever ends up issued will be invalidated if challenged. Quality isn't as important as quantity.
  • Reply 19 of 74
    flaneurflaneur Posts: 4,526member
    melgross wrote: »
    Yeah. I'm getting a little tired of Apple as of late. . . . [plus more petulant foot stamping]

    Your use of Google for comparison gives you away. You have forgotten Apple doesn't do Beta hardware, ever.

    Focus, thousands of noes, product and user-oriented technology, many ways to look at what Apple is doing rather than the impatient and market-focused way you are looking at them.

    I'm surprised at you too.
  • Reply 20 of 74
    Quote:
    Originally Posted by melgross View Post









    I'd rather see some of these innovative UI inventions come out, even if Apple makes them optional. And how about finally doing something with Siri? What have they don't with it these past few years? Hardly anything!

     

     

    Be Zen. Empty your mind. Do not pay attention to noise. Have faith in the prophets (OK, they do not say much, but ...).

     

    (may be what Apple needs are prophets, after all)

Sign In or Register to comment.