Apple looks to curb Web browser UI spoofing using real-time camera images

Posted:
in General Discussion edited January 2014
The U.S. Patent and Trademark Office on Thursday published an Apple patent filing for an anti-spoofing method in which camera output, as well as data from other on-board sensors, are used to change a browser's GUI in real-time.

Spoof
Source: USPTO


As Apple's iOS and Mac computing devices become more popular among mainstream consumers, the company has faced a number of security threats attempting to garnish sensitive user information. A new patent application discovered on Thursday addresses a specific type of Web-based attack called spoofing.

Apple's filing for a "Graphical user interface element incorporating real-time environment data" is an attempt to deal with nefarious code that can "spoof," or mimic, a Web browser's graphical user interface. In such cases, the malicious creator can redirect a user to another spoofed webpage, tricking them into giving up personal data, like usernames, passwords and credit card numbers.

To address the threat, Apple has devised a system which uses a device's various on-board sensors, including cameras, ambient light sensors and microphones, among others, to constantly update a browser's GUI -- sometimes referred to as "chrome" -- with real-time environment data.

According to the document, the incorporation of real-time video into a UI element provides added assurance that an interface is legitimate. In theory, as long as the content being provided to the content renderer does not have access to a device's on-board sensors, the chrome cannot be spoofed.

The application offers the example of a generic browser interface, as seen above, which displays forward and back buttons, a "Go To" bar and search bar. The elements are disposed in the chrome above an active webpage.

In this particular example, a device's front-facing camera is used to generate a real-time video feed. The chrome presents the image in background chrome. Said image can be blended in with the default background color or mirrored to create the illusion that the browser chrome is reflective.

Spoof


In another embodiment, the camera image is not mirrored, creating the illusion of a transparent window.
If in doubt, the user can check, e.g., by moving the camera (which may entail moving the entire device if the camera is built into the device) or moving an object in the field of view of the camera (e.g., waving a hand).
In addition, data from other components like an ambient light sensor or microphone can be used to update the UI in real-time. For example, a chrome's background can be changed dynamically to reflect changes in surrounding light, or its color can be "pulsed" in time with an environment's sounds.

The remainder of the patent filing deals with alternative implementations of real-time environment data, including user-selectable chrome tweaks such as transparency, frame rate and blurring, among other choices. As seen below, a UI can also be presented to help make these changes, with a preview window showing available options in real-time.

Spoof


It is not known whether Apple will implement the tech into future versions of its iOS or OS X machines, though malware is becoming a more legitimate threat to the company's ecosystem. One notorious malware called "Flashback" surfaced in 2011, targeting Macs by posing as an Adobe installer. After tricking users into installing, the nefarious program was built to connect to a remote server, with an end game of sending out payloads containing snooping software and other malicious code.

Still, for the proposed system to work efficiently, Apple must first ensure security in the apps it allows through the review process. In August, a research team from Georgia Tech managed to get a malicious app into the iOS App Store. Named "Jekyll," the program disguised itself as a news delivery program and, once installed, was able to post tweets, send email messages and direct mobile Safari to a malicious website.

Apple's real-time chrome updater security patent was first filed for in 2012 and credits Scott A. Grant as its inventor.

Comments

  • Reply 1 of 17
    irelandireland Posts: 17,798member
    How about Apple first plug the hole that allows websites to add cookies even when cookies is set to 'block always'. An ongoing bug in Safari for OS X.
  • Reply 2 of 17
    "attempting to garnish sensitive user information."

    Garnish them? What with, parsley?
  • Reply 3 of 17
    I'm sure there is more to it than this but what's stopping a malicious site from also legitimately presenting the chrome effect to the user. The problem is just like SSL the user will think if they see their face reflected in the browser skin (or currently the key or blue bar or whatever inconsistent clue browser makers have implemented) that everything is great when in reality all they have is a secure connection to the the crooks. Hardly anyone checks the actual detail anymore and the bigger and more obvious you make it apparent that the site is legit the less likely they are going to see that the certificate is issued to Bank of Amereca and not Bank of America.
  • Reply 4 of 17
    kibitzerkibitzer Posts: 1,114member
    webweasel wrote: »
    "attempting to garnish sensitive user information."

    Garnish them? What with, parsley?

    I saw that, too. Probably meant "garner" - "glean" would be better.
  • Reply 5 of 17

    I thought about this for a bit when I first hear Apple was patenting incorporation of ambient video into the interface -- and I thought it was just for some "super kewl" gee-wiz interface elements.

     

    However this sounds like it is a really great idea for knowing a "trusted system interface element" from something on screen. It's vitally important that no application but the OS be given this ability -- it could "request" the ability for authentication and there would have to be a lot of hand-shaking to PROVE the requester authentic.

     

    I've had a huge problem these days, even downloading an update from a legitimate site, because a "download your file" button from an advertiser -- or someone who has hacked that mechanism, looks exactly like the download your file button from the legitimate website. The problem with PayPal or any website is that everything on the screen is just pixels -- and there is no real way to tell if you have a man-in-the-middle attack if a URL is actually the URL that went to Google.

     

    On my mother's Windows computer, she would type in "google.com" and she would go to a google that was covered by adds and "click me to accelerate your computer" and it was peppered with "trusted" or "verified" authentic looking seals of approval. It was hijacking every URL and downloading files. We did a virus scan, we reset the router, we put in software that said it was good at protecting the machine from trojans. Either they somehow were controlling the DNS records or the hack had put itself in the registry. Destroying every browser and re-installing browsers that she never used before did not resolve the problem. The only thing preventing an advanced user like me from being taken in by this man-in-the-middle attack is the fact I was on a Mac and the "download an exe" link trick does not work.

     

    So I don't think this idea is trivial or "kewl" -- it's absolutely vital and a smart way to let people know interface elements that are authentic. IF it is to remain useful, nothing else on the computer must be allowed to use the camera or mic to execute the "chrome" reflection. Of course, a lot of people have allowed Flash access to just that sort of thing, so Apple is going to have to make that a System Property that cannot be granted without allowing a user allowing it each time.

  • Reply 6 of 17
    Quote:

    Originally Posted by Eric Swinson View Post



    I'm sure there is more to it than this but what's stopping a malicious site from also legitimately presenting the chrome effect to the user. The problem is just like SSL the user will think if they see their face reflected in the browser skin (or currently the key or blue bar or whatever inconsistent clue browser makers have implemented) that everything is great when in reality all they have is a secure connection to the the crooks. Hardly anyone checks the actual detail anymore and the bigger and more obvious you make it apparent that the site is legit the less likely they are going to see that the certificate is issued to Bank of Amereca and not Bank of America.

     

    The Difference is that the Mac OS would not allow another program access to the Video Camera. You could "animate something" on a button, but you can't create a mirror image without the front-facing camera. So Band of America would have to set up authentication procedures with Apple to provide this extra test of legitimacy. The weakness in such a scheme is similar to the 3rd party certificates we use now which our computers must actually TRUST. Apple can improve on this because they can create a unique key between the OS and Apple first, and the other party. Likely they will use a private key and authorize each transaction on a case by case basis.

     

    Hopefully they can avoid the mistakes made by VeriSign and all these broken NSA certificates we've got. Of course, there's no way to be sure the NSA doesn't strong-arm them into being part of the process. But at least we are secure as long as they don't take an interest in stealing from me my money rather than my privacy.

     

    >> The main backlash will be from people who freak out when they see their "body" on the screen. They will assume the video feed is going to Apple, which it isn't. Most people don't know that a lot of PC video cameras can be enabled without the light, and that there is tech to grab images from LCD monitors (not sure if it's being used yet or not).

  • Reply 7 of 17
    flaneurflaneur Posts: 4,526member
    kibitzer wrote: »
    I saw that, too. Probably meant "garner" - "glean" would be better.

    Or maybe "harvest." Reminds me of an unforgettable shaggy dog story about this mohel in a shtetl who hung watches in his window . . .

    Edit: added "unforgettable."
  • Reply 8 of 17
    flaneurflaneur Posts: 4,526member
    Deleted. Huddler needs help forgetting today. Memory is leaking.
  • Reply 9 of 17
    mobiusmobius Posts: 380member
    webweasel wrote: »
    "attempting to garnish sensitive user information."

    Garnish them? What with, parsley?

    I nearly splattered my half-chewed lunch all over my monitor when I read that! lol
  • Reply 10 of 17
    kibitzerkibitzer Posts: 1,114member
    flaneur wrote: »
    Or maybe "harvest." Reminds me of an unforgettable shaggy dog story about this mohel in a shtetl who hung watches in his window . . .

    Edit: added "unforgettable."

    Ah, my friend, it is flattering to be remembered. So vat vould ve hang in the vindow - Forstall's foreskin?
  • Reply 11 of 17
    lilgto64lilgto64 Posts: 1,147member
    How about finding a way to shut down the criminals entirely? But I suppose in a global world that is just too big a challenge.
  • Reply 12 of 17
    solipsismxsolipsismx Posts: 19,566member
    kibitzer wrote: »
    I saw that, too. Probably meant "garner" - "glean" would be better.

    Garnish can also mean to seize, but I wouldn't say it's the most appropriate usage since its meaning in law is quite specific.
  • Reply 13 of 17
    You also garnish wages and trust me if it happens to you then you will realize that garnish has a second meaning: "to take without consent".
  • Reply 14 of 17
    Garnish can also mean "to seize" typically used in the legal sense of "to garnish wages", second definition in OS X's dictionary app. I assume they're using it in that sense. Albeit, it's a bit of a stretched sense.

    UPDATE: Ah boo on the fact that AppleInsider only shows the first X number of comments on the article's main page. :( I had to click through to see "all" comments before I realized I was 2 hours behind the times on pointing out the alternative meaning of "garnish." Thanks anyway.
  • Reply 15 of 17
    Huh?

    There must be a serious lack of information here (details). I don't see how this protects anything.

    Raise your hand if anyone here can't tell the difference between your browser's toolbars and a web site.

    How does this prevent anyone from spoofing Chase's web site, or any other web site? Apple doesn't control the other end.
  • Reply 16 of 17
    great - something else wanting to access my camera. NSA exploitation in 3... 2... 1...
  • Reply 17 of 17
    Quote:

    Originally Posted by mytdave View Post



    Huh?



    There must be a serious lack of information here (details). I don't see how this protects anything.



    Raise your hand if anyone here can't tell the difference between your browser's toolbars and a web site.



    How does this prevent anyone from spoofing Chase's web site, or any other web site? Apple doesn't control the other end.

     

    If we ASSUME Apple is doing this to create a more secure environment -- it's a visual CUE to let you know the "Operating System has a verified tool in front of you."  Right now, everything on the screen is just pixels. If you've ever taken a screen shot of an interface and left it as a desktop without any visible things to click on -- you will know this prank makes people think their computer is stalled.

     

    Anyone who hijacks your browser or puts buttons that look legitimate -- right now there is no way to tell.

     

    So if you can move around in front of some interface element, and it appears like it's reflecting you on the screen -- you know that it's "verified" by Apple. Now it remains to be seen how well they secure that and verify authenticity -- but it's a subtle and brilliant way in my book to distinguish one set of pixels from an official set of pixels.

Sign In or Register to comment.