ITU GazeGroup

Research on eye tracking and gaze interaction

  • Increase font size
  • Default font size
  • Decrease font size


E-mail Print PDF

The GazeGroup is doing research within the following areas:

Development of Gaze Tracking Technology

Gaze tracking is the process of measuring the "Point of Regard" (PoR) or the "Line of Sight" (LoS) of the eye, and tracking it over time. This process can be divided into two subprocesses:

- Eye tracking, i.e. detecting and tracking eye features and movements
- Gaze estimation, i.e. calculating the eye gaze from eye features.

A gaze tracking system, or gaze tracker, is a device that measures eye movements and estimates gaze. Our group focuses on video-based gaze tracking, which consists of one or more cameras that record the eye(s) of the user. Like most commercial systems, we make use of infrared illumination to enhance the image and to estimate gaze. A diagram of a general gaze tracking system is shown in the following figure.


In the initial step, the camera grabs an image, which is transferred to a computer. The gaze tracking software will extract some eye features from the image, such as pupil center or iris center. These eye features are then mapped to eye gaze through a user calibration process. Due to noise, gaze data might be jittery, and therefore a fixation detection step is added to smooth data when a fixation is detected.

Low-cost gaze tracking

Our group focuses on gaze tracking technology using low-cost and off-the-shelf components, such as webcams and videocameras. We avoid hardware modifications, such as building an infrared LED, and hardware calibration, i.e. we work in uncalibrated setups where the location of the different components is unknown. This improves flexibility, but is more challenging to estimate gaze.

We aim to build a gaze tracking system that works indoor and outdoor under heavy illumination disturbance, and that can be used in mobile scenarios. For this, we have developed different algorithms to improve eye tracking under changing illumination conditions.

Hansen, D. W. and Hansen, J. P. 2006. Eye typing with common cameras. In Proceedings of the 2006 Symposium on Eye Tracking Research &Amp; Applications (San Diego, California, March 27 - 29, 2006). ETRA '06. ACM, New York, NY, 55-55. DOI=

Hansen, D. W. and Pece, A. E. 2005. Eye tracking in the wild. Computer Vision and Image Understanding 98, 1 (Apr. 2005), 155-181. DOI=

In April 2009 we launched the ITU Gaze Tracker, an open source gaze tracker developed by our group that seeks to be a starting point both for people who want to try gaze interaction for a low price and who want to develop gaze tracking software and gaze-based applications. You can read more about it here.

Mobile gaze tracking


Application of Eye Tracking for User Experience studies and HCI

By applying eye tracking methodology within the field of user experience and usability research, we can provide objective and quantifiable data, that is not available when using conventional usability methods. This is why the use of eye tracking analysis is increasing within the area of UX and usability research; however, the eye tracking method is still far from being a broadly recognized part of the toolbox for practitioners. Eye tracking is criticized for being costly and tedious, thus not delivering adequate perceived benefit.

In the Gaze Group PhD student Sune Alstrup Johansen is doing research to improve the benefit of eye tracking analysis by validating and defining suitable methods for different kind of stimuli and test situations. E.g. studies of gaming experience needs a different approach than studies of corporate websites.

Another way to make the eye tracking method more relevant to practitioners is to make it cheaper and easier to perform eye tracking analysis. In the group we are doing research into ways of using low cost eye tracking technology for remote usability studies.

 Johansen, S. A. and Hansen, J. P. 2006. Do we need eye trackers to tell where people look?. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 - 27, 2006). CHI '06. ACM, New York, NY, 923-928. 


Gaze based Interaction

Text Input

Gaze typing is text production with the gaze only. In many cases, gaze typing is done by people with severe disabilities (e.g. people who are locked in or amytrophic lateral sclerosis, ALS) as a single modality but in some cases multimodal interaction is used (e.g. EMG, EOG, EEG). These people have a strong need for having an accurate and efficient communication system that convey their gaze into readable text on a computer screen. Most typing interfaces are based on variants of on-screen keyboards and by tracking the user’s point of gaze, the computer decides which letter the user is focusing on. The feedback to the user is typically done with highlights and small animations on the keys (dwell selections) and sometimes by audio feedback.

Text input is a common task in interaction research and has been intensively investigated and alternative text production systems are typically tested by usability studies and by standardized performance measurements and comparisons of results (e.g. word-per-minute, WPM and Error Rates). The metrics are highly sensitive to learning and design features.


A good example of dwell selections and animated boxes is the gaze typing system called GazeTalk. The GazeTalk typing system was designed and developed in collaboration between the Eye Gaze Interaction Group (now at the IT University of Copenhagen and the IT-Lab at the Royal School of Library and Information Science, Copenhagen. The GazeTalk research project aims to develop and research low-coat gaze-tracking based augmentative and alternative communication (AAC) system for people with severe disabilities on a restricted on-screen keyboard.

The flexible design of GazeTalk allows the individual user to have a complete-feature system, but it has also been used in various research experiments. Recent observations of regular users of GazeTalk indicate that they can gaze type with an average of 10 words per minute when the adaptive language model becomes familiar with their individual vocabulary. Some early research on GazeTalk can be found here:

Itoh, K., Aoki, H., and Hansen, J. P. 2006. A comparative usability study of two Japanese gaze typing systems. In Proceedings of the 2006 Symposium on Eye Tracking Research &Amp; Applications (San Diego, California, March 27 - 29, 2006). ETRA '06. ACM, New York, NY, 59-66. DOI=

Hansen, J. P., Tørning, K., Johansen, A. S., Itoh, K., and Aoki, H. 2004. Gaze typing compared with input by head and hand. In Proceedings of the 2004 Symposium on Eye Tracking Research & Applications (San Antonio, Texas, March 22 - 24, 2004). ETRA '04. ACM, New York, NY, 131-138. DOI=

Johansen A. S. and Hansen J. P., Augmentative and Alternative Communication: The Future of Text on the Move, Proceedings of 7th ERCIM Workshop "User Interfaces for all", Paris (Chantilly), France, 23-25 October 2002, pp. 367-386.

A longer manual of GazeTalk is available here


An innovative 3D interface which can be used for gaze typing was conceived as a master thesis project and ended up as research application which addresses noise tolerant navigation in graph-based structures by utilizing pan and zoom.

The project is developed at the IT University of Copenhagen with support by the COGAIN network. This project aims at the investigating new gaze interaction paradigms and the current design of StarGazer is set to text production by gaze.

With the current setup of StarGazer we have reported typing speeds with an average of 4.7 words per minute but in a speed test, we observed up to 8 words per minute without word or character predictions. Later experiments show that a low-cost gaze tracker performs equally with commercial eye trackers (i.e. Tobii and SMI) when using the StarGazer. Some of the research on StarGazer can be found here:

Hansen, D. W., Skovsgaard, H. H., Hansen, J. P., and Møllenbach, E. 2008. Noise tolerant selection by gaze-controlled pan and zoom in 3D. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (Savannah, Georgia, March 26 - 28, 2008). ETRA '08. ACM, New York, NY, 205-212. DOI=

San Agustin, J., Skovsgaard H.H., Hansen J. P. and Hansen D. W. (2009) Low-Cost Gaze Interaction: Ready to Deliver the Promises. CHI '09 Extended Abstracts on Human Factors in Computing Systems. ACM.

Gaze-based selection tool

Performing selections of small targets using the gaze only is hindered by the limited accuracy of gaze tracking. This project explores the potentials for small-target selection using existing activation methods (single- and two-step dwell) and a novel zoom activation method in single- and multiple-target layouts. Some of the research performed on gaze-based selections can be found here:

Henrik H. T. Skovsgaard, John Paulin Hansen and Julio C. Mateo, How to hit tiny buttons with gaze only? In Proceedings of COGAIN 2008


Gaze Driving

Control of wheelchairs and remote vehicles could both benefit from effective hands-free input. Issues with highly variable lighting conditions (e.g. sunlight and neon lights shining directly into the camera pointing upwards a user’s face) have been reported and vibrations from the moving wheelchair also complicated tracking. This suggests that gaze tracking needs to be precise and robust to control a vehicle. There are three different design approaches to controlling a vehicle by gaze: 1) Directly by just looking where you would like to drive. 2) Indirectly by gazing at on-screen buttons that executes commands like “forward”, “stop” etc.; 3) By gazing at an image of the front-view. We have decided to explore the last version since it provides a direct mapping between “target point” and gaze point that is the virtue of the first approach, but with a fast and obvious way of braking – namely to look away from the screen.

Tall, M., Alapetite, A., San Agustin, J., Skovsgaard, H. H., Hansen, J. P., Hansen, D. W., and Møllenbach, E. 2009. Gaze-controlled driving. In Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI EA '09. ACM, New York, NY, 4387-4392.