A unique attack could steal your passwords from Apple Vision Pro

Posted:
in Apple Vision Pro

Apple Vision Pro's eye-tracking technology offers a new way to interact with typing, but hackers are already exploiting it to steal sensitive information. Here's what you need to know to protect your data.

Apple Vision Pro on a white table with a silver rectangular device, blurred colorful background.
Apple Vision Pro



New technologies always come with new vulnerabilities. One such vulnerability, GAZEploit, exposes users to potential privacy breaches on Apple Vision Pro FaceTime calls.

GAZEploit, developed by researchers from the University of Florida, CertiK Skyfall Team, and Texas Tech University, uses eye-tracking data in virtual reality to guess what a user is typing.

When users don a virtual or mixed reality device, like the Apple Vision Pro, they can type by looking at keys on a virtual keyboard. Instead of pressing physical buttons, the device tracks eye movements to determine the selected letters or numbers.

Diagram showing facial biometric extraction, eye aspect ratio, gaze estimation, and keystroke inference. Includes charts for click events and keystroke sessions, concluding with keyboard mapping and key guess inference.
Overview of the attack



The virtual keyboard is where GAZEploit comes in. It analyzes the data from eye movements and guesses what the user is typing.

GAZEploit works by recording the movements of the virtual avatar's eyes of the user. It focuses on the eye aspect ratio (EAR), which measures how wide a person's eyes are open, and eye gaze estimation, which tracks exactly where they're looking on the screen.

By analyzing these factors, hackers can determine when the user is typing and even pinpoint the specific keys they're selecting.

When users type in VR, their eyes move in a particular way and blink less often. GAZEploit detects this and uses a machine learning program called a recurrent neural network (RNN) to analyze these eye patterns.



The researchers trained the RNN with data from 30 different people and got it to accurately identify typing sessions 98% of the time.

Guessing the right keystrokes



Once a typing session is identified, GAZEploit predicts the keystrokes by analyzing rapid eye movements, called saccades, followed by pauses, or fixations, when the eyes settle on a key. The attack matches these eye movements to the layout of a virtual keyboard, figuring out the letters or numbers being typed.

GAZEploit can accurately identify the selected keys by calculating the gaze's stability during fixations. In their tests, the researchers reported 85.9% accuracy in predicting individual keystrokes and nearly perfect 96.8% recall in recognizing typing activity.

Since the attack can be carried out remotely, attackers only need access to video footage of the avatar to analyze eye movements and infer what is being typed.

Remote access means that even in everyday scenarios such as virtual meetings, video calls, or live streaming, personal information like passwords or sensitive messages could be compromised without the user's knowledge.

How to protect yourself from Gazeploit



To protect against potential attacks like GAZEploit, users should take several precautions. First, they should avoid entering sensitive information, such as passwords or personal data, using eye-tracking methods in virtual reality (VR) environments.

Instead, it's safer to use physical keyboards or other secure input methods. Keeping software updated is also crucial, as Apple often releases security patches to fix vulnerabilities.

Finally, adjusting privacy settings on VR/MR devices to limit or disable eye-tracking when not needed can further reduce exposure to risks.



Read on AppleInsider

Comments

  • Reply 1 of 7
    mknelsonmknelson Posts: 1,143member
    The method makes sense, but where is Gazepoit running? Is it on your Vision Pro (and shouldn't privacy settings limit access to the avatar?) or is it running on another device you are communicating with?
    watto_cobra
  • Reply 2 of 7
    gatorguygatorguy Posts: 24,605member
    mknelson said:
    The method makes sense, but where is Gazepoit running? Is it on your Vision Pro (and shouldn't privacy settings limit access to the avatar?) or is it running on another device you are communicating with?
    No, it's a remote monitoring of your virtual Persona. You can watch how it works, already linked in the Apple Insider article

    edited September 12 muthuk_vanalingam
  • Reply 3 of 7
    Apple already fixed this. 1.3 turns off the persona when typing.
    watto_cobra
  • Reply 4 of 7
    gatorguygatorguy Posts: 24,605member
    Apple already fixed this. 1.3 turns off the persona when typing.
    It was never going to be a huge thing to fix anyway. Something as simple as dumping every other frame or something to that effect would eliminate it. 
  • Reply 5 of 7
    Since Apple already manipulates one’s gaze (at least on iOS) so that it appears as though one is looking right at one’s FaceTime caller, why not just fix the pupils (or otherwise naturally animate them) while one has a virtual keyboard active?
    watto_cobra
  • Reply 6 of 7
    I frankly do not understand how this could potentially work on the Apple Vision Pro. The eyes are totally covered by the device. there is no way to watch the users eyes from let’s say a remote camera (the eyes on the front display are not real) . The hacking software would already need to run on the Vision Pro, which would’ve meant that the device is  anyway already severely compromised and most likely passwords and everything else are accessible to the hacker. Maybe in some European Union side loading scenario this is possible if the user unintentionally installed the software from a hacked App Store.
    watto_cobra
  • Reply 7 of 7
    bohler said:
    I frankly do not understand how this could potentially work on the Apple Vision Pro. The eyes are totally covered by the device. there is no way to watch the users eyes from let’s say a remote camera (the eyes on the front display are not real) . The hacking software would already need to run on the Vision Pro, which would’ve meant that the device is  anyway already severely compromised and most likely passwords and everything else are accessible to the hacker. Maybe in some European Union side loading scenario this is possible if the user unintentionally installed the software from a hacked App Store.
    It is fixed. It never ran on the AVP. It worked by watching users of AVP persona which relayed what users were looking at. 
    watto_cobra
Sign In or Register to comment.