Eyegaze Human-Computer Interface for People with Disabilities

ABSTRACT

The LC Technologies Eyegaze System provides an eye-controlled human-computer interface (HCI), allowing people to interact with computers by pointing with their eyes. A video camera mounted below the computer monitor unobtrusively observes the user's eye and specialized image processing software analyzes the video images of the eye and determines the eye's gazepoint on the monitor screen in real time. Early applications of the Eyegaze System addressed an HCI for people with severe motor disabilities. Simply by looking at control keys displayed on a computer screen a disabled user can type, generate synthesized speech, control lights and appliances, operate a telephone, play games, and run DOS-compatible off-the-shelf software.

Creating an eyegaze HCI that accommodates a variety of physical disabilities presented our engineering team with several technical challenges. The Eyegaze System has to be accurate enough for a user to trigger the 5/8-inch keys of an on-screen computer keyboard. The calibration procedure needs to be simple, and the system needs to maintain calibration when the user leaves the computer and returns. Finally, for people with uncontrolled head motion, it needs to be tolerant to head motion. The accuracy and calibration objectives have been achieved. A solution for accommodating head motion is under development.

INTRODUCTION

Image processing eyetrackers exist in two categories: head-mounted and remote. For disabled people operating computers, it is appropriate to sense the eye unobtrusively, with remotely mounted cameras. The user need not be mechanically "hooked up" to access the system, and has no need for cumbersome equipment on his body.

In 1988, LC Technologies completed development of the first Eyegaze Computer System designed for use by people with severe motor disabilities. Eyegaze is a PC-based system, requiring only the control of one eye. Selections are made by fixing the gaze in control "keys" on the screen. Nothing is attached to the user.

As illustrated in Figure 1, a video camera located below the computer screen continually observes the user's eye, and specialized image-processing software determines the eye orientation and projects the subject's gazepoint on the computer display. With a person sitting between 18 and 24 inches from the computer screen, the system predicts the gazepoint with an average accuracy of better than 1/4 inch. The system also generates information regarding pupil diameter, blinking, and eye fixations, useful for other eyetracking applications.

Figure 1: Eyegaze System Configuration

METHOD

Figure 2: LED-illuminated eye image showing bright pupil. Small spot of light directly below the pupil is the corneal reflection.

Prior to operating the eyetracking applications, the Eyegaze System must learn several physiological properties of a person's eye in order to be able to project his gazepoint accurately. It must know the radius of curvature of the eye's cornea and the angular offset between the eye's optic and focal axes. The system learns these parameters by performing a calibration procedure. To calibrate, the user fixes his gaze on a sequence of small circles that the computer displays at specific locations on the screen. The calibration procedure usually takes about 15 seconds and can be performed independently.

The Eyegaze System can save calibration results for future use, and it will retain current calibration data even if the user moves away from the system. When he returns to his position in front of the camera, Eyegaze will resume its gazepoint determination, enabling the user to continue to operate the system.

Eyegaze Selection Method:

Figure 3a: Main Menu screen with program options and selection keys.

Figure 3b: Appliance control screen with an exit key to return to the Main Menu

Eyegaze Applications Programs:

The Run Second PC program provides the user with access to off-the-shelf keyboard-activated software, such as word processors and spread sheets. The user runs programs by visually operating an on-screen computer keyboard. A Read Text program enables the user to access any text that is in a computer-readable format. The disabled user turns the pages by looking at "up" and "down" keys. Eye-operated games include Paddle, Klondike Solitaire and a slot machine.

Human-Computer Interaction Issues:

When the user fixes his gaze in a key long enough to activate it, the key either flashes a color or, in the case of optional 3-D keys, appears to have been depressed. Further feedback is provided on all the typewriter and computer keyboard screens by a "click" sound, similar to the sound made by a manual keyboard as its keys are depressed.

A variety of modifications to the Eyegaze software can be made to accommodate limitations of eye movement and control that may exist in some disabled users. The gaze duration for key activation can be increased or decreased as needed. In general, a gaze duration of between 1/4 and 2/3 second is most comfortable. A key press delay can be added to give users with slowed eye movements additional time to move their gaze off of a key without repeatedly activating it.

Operational Constraints:

Eye Visibility: The camera must have a clear view of the subject's eye. If either the pupil[2] or the corneal reflection are occluded, there is insufficient image information to make an accurate gaze measurement. The user must also be able to maintain his head in a position that keeps his eye visible to the Eyegaze camera, which is a problem for people with uncontrolled spastic head movement, common with some types of cerebral palsy.

Glasses and Contact Lenses: In most cases, eyetracking works with glasses and contact lenses. If bifocal glasses have a distinct lens boundary that splits the camera's image of the eye, the discontinuity in the image invalidates the image measurements. Graded bifocals, however, typically do not interfere with eyetracking. Soft contact lenses that cover all or most of the cornea generally work well with the Eyegaze System. The corneal reflection is obtained from the contact lens surface rather than the cornea itself. Small, hard contacts can cause problems, however, if the lenses move around considerably on the cornea.

RESULTS

Disabled users range in age from 7 years to around 70 years. Their disabilities include: cerebral palsy, spinal cord injuries, traumatic brain injuries, strokes, amyotrophic lateral sclerosis (Lou Gehrig's disease), and multiple sclerosis (Cleveland & Doyle, 1992).

Several Eyegaze Systems are in schools, used by one or more disabled students. Some systems are in private homes, offering a level of independence and quality of life to people otherwise unable to interact with the outside world. Eyegaze Systems in the workplace are enabling a physicist, a newspaper editor, and a store owner to continue to work productively. Other Eyegaze Systems are in rehabilitation facilities.

FUTURE DIRECTIONS

To help incorporate eyegaze into advanced human-computer interfaces, LC Technologies has created an eyegaze tool kit that allows developers to integrate gazepoint tracking equipment and software into computer applications. The Eyegaze Development System includes the source code for several complete applications programs that may be used as is, modified to meet custom needs, or used as references for preparing other Eyegaze applications programs.

As an example of an eyegaze monitoring application, the Trace program (Figure 4) displays a user-prepared image on the computer monitor and collects the eyegaze activity as a subject observes the screen. The eyegaze history is stored in a disk file. After the data collection phase, the eyegaze history is played back both as a time history and as a trace superimposed on the original screen image. The trace may be paused, reversed and replayed at different speeds. Different eyegaze variables, such as the pupil diameter or the x and y coordinates of the gazepoint, may be plotted out as a function of time.

Figure 4: Eyegaze history recorded by Trace program

For general usage in HCI, eyetrackers must be more tolerant to head motion than they are today. In 1993, LC Technologies completed Phase I of an SBIR Medical Rehabilitation Research grant sponsored by the National Institutes of Health. The research demonstrated the feasibility of a headtracker device which keeps the camera pointed at and clearly focused on a person's eye as he moves his head around a one cubic foot region with speeds of up to eight inches per second (Cleveland, 1990, 1992, 1993a, 1993b, Cleveland & Cleveland 1992). In keeping with the philosophy of not burdening the disabled user with intrusive devices, nothing is attached to the user.

REFERENCES

• Cleveland, D. (1990). Focus Control System. U.S. Patent #4,974,010.
• Cleveland, D. (1992). Method and Apparatus for Mirror Control. U.S. Patent #5,090,797.
• Cleveland, D. (1993a). A headtracking device for the Eyegaze eyetracking system. Proceedings of the 5th International Conference on Human-Computer Interaction. August 11-13, 1993. Orlando, Florida.
• Cleveland, D. (1993b). Method and Apparatus for Locating Image Features. U.S. Patent #5,231,674.
• Cleveland, D. & Cleveland, N. (1992). Eyegaze eyetracking system. Proceedings of 11th Imagina International Forum on New Images. January 29-31, 1992. Monte Carlo, Monaco.
• Cleveland, N. & Doyle, M. (1992). Eyegaze Communication System-How does it work? Who can use it?". Proceedings of the 10th Annual Conference on Microcomputer Technology in Special Education & Rehabilitation. October 22-24, 1992. Minneapolis, Minnesota.
• Cornsweet, T.N. (1973). Eye Tracker. U.S. Patent #3,712,716.
• Hutchinson, T.E. (1989). Eye Movement Detector. U.S. Patent #4,836,670.
• Mason, K.A. (1969). Control apparatus sensitive to eye movement. U.S. Patent #3,462,604.
• Merchant, J. & Morrisette, R. (1973). A remote oculometer permitting head movement (Report No. AMRL-TR-73-69). Wright-Patterson Air Force Base, Ohio: Aerospace Medical Research Laboratory.
• Young, L. & Sheena, D. (1975). Survey of eye movement recording methods. Behavior Research Methods and Instrumentation, 7, 397-429.

Footnotes:

1. The intensity of infrared light emitted by the Eyegaze System's LED and reflected off the retina is one-fifth of the recommended safe level of intensity established by NIOSH.
2. The upper portion of the pupil does not have to be visible for accurate eyetracking. Special software can be called up to accommodate those users with a ptosis of the eyelid blocking the top of the pupil.

This address is http://www.eyegaze.com/doc/cathuniv.htm

Back to Top