EYEGAZE ANALYSIS SYSTEM

There are many situations in human factors research when it is desirable to know where a person is looking. The Eyegaze Analysis System is a tool for measuring, recording, playing back, and analyzing what a person is doing with his eyes. It is an instrument for both developing and running custom eyetracking applications. It includes all the basic video equipment, computer hardware, and Eyegaze software and C/C++ development libraries necessary to build and run eyetracking application programs.

Gazepoint tracking measurements are made unobtrusively via a remote video camera mounted below the computer monitor. Nothing is attached to the subject. The Eyegaze System tracks the subject's gazepoint on the screen automatically and in real time, with Gazepoint measurements made at a 60 Hz rate. Gaze direction is determined using the pupil-center-corneal reflection (PCCR) method and the image processing and gazepoint calculations are performed in software on a Windows XP computer. At each camera image sample, the Eyegaze System generates:

• an eye-found flag indicating whether or not the eye is visible to the camera and thus whether or not a valid gazepoint is calculated,
• x-y coordinates of the subject's gazepoint on the computer screen,
• pupil diameter,
• 3-dimensional location of the eyeball center within the camera field-of-view, an indicator of head location and movement, and
• fixation and saccade analysis.

• Head Range Variation
• Pupil Diameter Variation
• Corneal Reflection Straddling Pupil Edge

Camera and Monitor Configuration

Return to Top

Detailed Specifications:

The Eyegaze System uses the Pupil-Center/Corneal-Reflection method to determine the eye's gaze direction. A video camera located below the computer screen remotely and unobtrusively observes the subject's eye. No attachments to the head are required.

A small, low power, infrared light emitting diode (LED) located at the center of the camera lens illuminates the eye. The LED generates the corneal reflection and causes the bright pupil effect, which enhances the camera's image of the pupil.

Camera Image of Eye, Illustrating Bright Image Pupil and Corneal Reflection

Specialized image-processing software in the Eyegaze computer identifies and locates the centers of both the pupil and corneal reflection. Trigonometric calculations project the person's gazepoint based on the positions of the pupil center and the corneal reflection within the video image. The Eyegaze System generates raw gazepoint location data at the camera field rate of 60 Hz.

Fixation Analysis

A function called DetectFixation() analyzes the eye's motions to identify its saccades and fixations. This function takes the raw gazepoint data as uniformly sampled at the camera frame rate, detects the saccadic motions between fixations, and reports the positions and durations of fixations as they occur. The raw gazepoint data may be passed to the DetectFixation function either in real time or in post analysis.

LC Technologies' fixation function source code is available here. This code may be reused as desired as long as proper credit is given.

The Trace program (discussed below) illustrates the use of the DetectFixation function and plots fixation histories on the computer screen.

Development Tool Kit

The Eyegaze Analysis System includes a C/C++ Language software tool kit to support custom eyetracking applications programs in a Windows 2000/XP environment. The tool kit contains a library of user-linkable functions that track the eye, analyze fixations, and make the eyetracking data readily accessible to the application program, either on a real-time or after-the-fact basis. Source code is provided for a variety of functions to analyze, store and display eye-motion data. Complete source code is also provided for several example Eyegaze applications programs to illustrate typical usage of the Eyegaze functions. The Eyegaze software supports applications programs that run either on the Eyegaze System computer itself or on a client computer which treats the Eyegaze computer as a peripheral device (see Single vs Double Computer Configurations below).

Run-Time Operation

The image processing functions in the Eyegaze System are all implemented in software. The eyetracking functions compute the raw gazepoint coordinates each 60^th of a second in synchronization with the field rate of the video camera. At each sampling iteration, the eyetracking software places the measured eyegaze data into data structures where it is readily available to the application program.

Calibration Procedure

Prior to tracking a person's eye, the Eyegaze System must learn several physiological properties of the person's eye in order to be able to project his gazepoint accurately. It must know the radius of curvature of the eye's cornea and the angular offset between the eye's optic and focal axes. The system learns these parameters by performing a calibration procedure where the subject sequentially looks at a series of calibration dots displayed at different locations on the screen. The procedure takes about 15 seconds and need not be repeated for a given person on different work sessions. The calibration results are passed to the eyetracking functions for purposes of future gazepoint projection, and are also placed in disk files for access by the applications programs if desired.

Return to Top

Eyetracking Application Programs

Eyegaze applications programs access the raw eyetracking data generated by the eyetracking functions and perform the display, recording, analysis, and replay functions desired by the developer. The Eyegaze Analysis System includes the source code for several complete applications programs that may be used as is, modified to meet custom needs, or used as references for preparing other Eyegaze applications programs. The following are brief descriptions of the applications programs.

GazeDemonstration Program

The simplest application program is the GazeDemonstration program. It positions the mouse cursor on the desktop display at the person's projected gazepoint, i.e., the cursor simply follows the user's gazepoint.

Trace Program

The Trace program displays a user-prepared image on the computer monitor and passively collects the eyegaze activity as a subject observes the screen. The eyegaze history is stored in a disk file. After the data collection phase, the eyegaze history is played back both as a time history and as a trace superimposed on the original screen image. The trace may be paused, reversed and replayed at different speeds. Different eyegaze variables, such as the pupil diameter or the x and y coordinates of the gazepoint, may be plotted out as a function of time.

The following figures show examples of a graphical gaze trace replay. A link below the figures points to the corresponding gaze history file produced by the trace program.

Raw Gaze Trace (Individual points are 60th of a second samples)

Fixation Gaze Trace (circle sizes indicate fixation duration)
Gazetrack Data File for above gaze trace

Return to Top

Eyetracker Outputs

The Eyegaze System generates the following data from each camera field image:

Eyegaze System Performance Specifications

Accuracy

* Bias errors result from inaccuracies in the measurement of head range, asymmetries of the pupil opening about the eye's optic axis, and astigmatism. They are constant from frame to frame and cannot be reduced by averaging or smoothing.

+ Frame-to-frame variations result from image brightness noise and pixel position quantization in the camera image and may be reduced by averaging or smoothing.

Speed

Angular Gazetrack Range

As the eye's gaze axis rotates away from the camera, the corneal reflection moves away from the center of the cornea. Accurate gaze angle calculation ceases when the corneal reflection "falls off" the edge of the cornea. The eye's gaze axis may range up to 40 degrees away from the camera, depending on the arc of the person's cornea. The lower 15 degrees of the gaze cone, however, is generally clipped due to the upper eyelid blocking the corneal reflection when the eye is looking down below the camera.

Tolerance To Head Motion

The eye must remain within the field of view of the camera. However, if the subject moves away from the camera's field of view, eyetracking will resume once he returns to a position where his eye is again visible to the camera.

Computer Usage

Light Emitting Diode

Operational Requirements

Low Ambient Infrared Light

There must be low levels of ambient infrared light falling on the subject's eye. Stray IR sources obscure the lighting from the Eyegaze System's light emitting diode and degrade the image of the eye. The sun and incandescent lamps contain high levels of infrared light. The environment may be brightly illuminated with lights such as fluorescent or mercury-vapor which do not emit in the infrared region of the spectrum. The Eyegaze System also works well in the dark.

Eye Visibility

The camera must have a clear view of the subject's eye. If either his pupil or the corneal reflection are occluded, there may be insufficient image information to make an accurate gaze measurement. The camera's view of the eye can be obstructed by, for example, 1) an object between the camera and the eye, 2) the person's nose or cheek if his head is rotated too much with respect to the camera, or 3) by excessive squinting. Alternative Eyegaze software is included to accommodate for an obstructed image of the top of the pupil, usually caused by a droopy eyelid or an unusually large pupil. The software returns a "false" condition for the EyeFound flag whenever an adequate image of the eye is not present.

Glasses and Contact Lenses

In most cases, eyetracking works with glasses and contact lenses. The calibration procedure accommodates for the refractive properties of the lenses. When wearing glasses, the glasses may not be tilted significantly downward, or the reflection of the LED off the surface of the glass is reflected back into the camera and obscures the image of the eye. The lens boundary in hard-line bifocal or trifocal glasses often splits the camera's image of the eye, and the discontinuity in the image invalidates the image measurements.

Soft contact lenses that cover all or most of the cornea generally work well with the Eyegaze System. The corneal reflection is obtained from the contact lens surface rather than the cornea itself. Small, hard contacts can cause problems, however, if the lenses move around considerably on the cornea, and the corneal reflection moves across the discontinuity between the contact lens and the cornea.

Return to Top

Single vs Double Computer configurations

With the Eyegaze System, eyetracking application programs may be configured to run directly on the Eyegaze System computer, or to run on another computer with the Eyegaze System acting as a peripheral eyetracking instrument.

Single Computer Configuration

If it is desired to run the eyetracking application program directly on the Eyegaze System computer,

1. the Eyegaze camera is mounted below the Eyegaze computer's monitor,
2. the Eyegaze computer performs the applications functions and drives the application display on its monitor, and
3. the Eyegaze computer also performs the Eyegaze image processing functions required to track the test subject's gazepoint.

The single computer configuration is preferable if it is desired to minimize the amount of equipment needed to implement the project.

Double Computer Configuration

If it is desired to run the eyetracking application program on a separate (client) computer with the Eyegaze System acting as a peripheral device,

1. the client computer performs the application functions and drives the applications display on its monitor,
2. the Eyegaze camera is mounted below the client computer's monitor,
3. the Eyegaze computer performs the Eyegaze image processing functions and transfers the measured gazepoint data to the client computer via an Ethernet or a serial communications link in real time, and
4. the Eyegaze computer's monitor displays the subject's relative gazepoint in real
   time, allowing a test observer to view the gazepoint activity on-line during a test.

The double computer configuration is preferable if the application code consumes a large amount of CPU time (i.e. if there is not enough CPU time to execute both the application and Eyegaze image processing code in real time), if the application program is not compatible with Windows 2000/XP, or if an observer station is desired for viewing gaze activity on line.

PROGRAMMING INTERFACE

The programmer's interface to Eyegaze is accomplished through a calibration program and three basic functions, illustrated in the following pseudo code:

{
 if (new user)
 {
 run Calibrate.exe    //have person perform the calibration procedure
 }
 setup Eyegaze data structures
 EgInit();           // create and start the Eyegaze thread
 for (loop)
 {
 EgGetData();        // wait for Eyegaze to process next camera field
 use Eyegaze data
 }
 EgExit();           // terminate the Eyegaze thread
}

In the single computer configuration, the Eyegaze software operates as a thread within your applications program.

In the double computer configuration, the client computer program uses an almost identical programming approach but image processing runs on the Eyegaze Computer and data is sent over the Ethernet/serial link.

The Eyegaze System includes source code for the EgClientDemo program which runs on the client computer and demonstrates communication with the Eyegaze System. To act as a peripheral device to a client computer, the EgServer program is run on the Eyegaze System computer.

Return to Top