Apple TV
Apple was judged to be infringing OpenTV’s video streaming patents in Germany. Pictured: A man plays a video game controlled by the remote that comes with the Apple TV at an Apple Store in Los Angeles, Oct. 30, 2015. Reuters/Jonathan Alcorn

Owners can control the latest Apple TV using a remote, touchpad or even their voice via Siri. But in the future, users may also be able to add eye and hand gestures to that list as well.

A patent titled “Gaze Detection In A 3D Mapping Environment” was granted by the U.S. Patent and Trademark Office, revealing an invention which could enable users to control objects on screen by just gazing at items presented in an on-screen menu. To accomplish this, Apple proposes using the camera sensor and an additional sensor to create a 3D map of the room.

Apple 3D Eye gaze
A future product from Apple could use camera sensors to give users control of their television or computer via eye gestures. USPTO/Apple Inc.

In addition to eye controls, users could also point to objects on screen to control the interface from a distance, according to one version of the invention. The patent could also extend to other devices such as Macbooks, iPads and iMacs.

Another potential application for the technology is its use in virtual and augmented reality. The two technologies are related but use different takes on head-mounted displays. With virtual reality (VR), a head-mounted display is used to fully immerse its user in a 3D environment. Augmented reality (AR) instead overlays computer-generated images on top of real-world objects, similar to the display a fighter pilot sees when in flight.

Apple is said to have hundreds of engineers on a virtual and augmented reality project, according to the Financial Times. But officially it hasn’t said that it’s in the works at all.

Eye Gaze
A diagram illustrating how a user could control an on-screen interface by gazing at the screen. USPTO/Apple Inc.

The patent is one of many acquired by Apple when it purchased Israel-based PrimeSense in 2013, the maker of the first Microsoft Xbox Kinect motion-tracking sensor. As with many of Apple’s patents, it’s unknown if or when it plans to use it in one of its products.

The patent credits former Primesense engineers Eyal bychkov, Oren Brezner, Micha Galor, Ofir Or, Amir Hoffnung, Tamir Berliner and Apple Engineer Jonathan Pokrass.