William D. Walker - 1
Mark E. Novak
Henry R. Tumblin
Gregg C. Vanderheiden
While electronic office equipment has offered increased productivity and independence to many individuals, it has also presented barriers to some users, namely those with disabilities. In 1986, the federal government enacted Section 508, which is an amendment to the Rehabilitation Act of 1973. This amendment requires federal agencies to include accessibility provisions or requirements in the RFP process for electronic office equipment. In addition, the American Disabilities Act of 1990 requires all employers to provide an accessible workplace for their employees.
In this paper, we describe issues regarding accessibility for people with physical and sensory disabilities. We also present plans that address the accessibility issues of X by providing multiple methods for information input and output. Some of the plans discuss "direct access" features that can be built directly into the base X product, and other plans discuss hooks that can be added to the base products. These hooks allow developers to extend the capabilities of X and could be considered features which facilitate "alternative access" solutions by third parties.
It is important to understand there are many different types and severities of impairments that can lead to disabilities. An abbreviated list includes movement, visual, hearing, and cognitive/language impairments.
People with a movement impairment typically have difficulty using a computer's input devices. Examples of people with movement impairments include people with advanced arthritis, carpel tunnel syndrome, spinal cord injury, head injury, multiple sclerosis, muscular dystrophy, and cerebral palsy. Over thirty million people have one or more of these conditions.
Visual impairments cause people difficulty reading and seeing information displayed on the computer screen. Although many people have visual impairments from birth, people often acquire visual impairments as a result of the natural aging process. The National Society for the Prevention of Blindness estimates that eleven million people in the United States have visual impairments.
People with a hearing loss find it difficult to distinguish important audible information the computer might be creating from the normal environment background noise. Like visual impairments, hearing loss is part of the natural aging process and more than fifteen million Americans have some form of hearing loss. In addition, almost two million Americans are deaf.
Cognitive and language impairments include a wide range of information processing skills such as problems solving, memorization, language skills, learning and perception, and deficits in orientation, attention, or judgment.
In this paper, we address issues regarding graphical user interfaces (GUI's) and physical and sensory disabilities.1 Although they are extremely important and still need considerable attention, the issues regarding cognitive and language impairments are complex and merit a separate discussion.
Many reasons exist for making the X Window System accessible. Some of the most significant reasons include the following: the number of people with disabilities is growing, making computers more accessible usually helps all users, and federal legislation has been enacted to help ensure computers and operating systems are accessible.
Between thirty and forty million people in the U.S. have disabilities which affect their ability to use computers. Since people often acquire disabilities with age, the number of individuals with a disability or functional limitation will grow increasingly larger as the "baby boomers" age. Based on the US Congress Office of Technology Assessment OTA-BA-264, in fifty years it is estimated that one third of the population will be over fifty years of age and one sixth will be over seventy. In addition, by age fifty-five, twenty-five percent of us will experience functional limitations. This number increases to seventy-five percent for those who are over seventy.
It has been estimated that seven to nine out of every ten major corporations employ individuals with disabilities. Because the X Window System is very popular among corporations, universities, research facilities, and the federal government, there is a need to address the accessibility issue of X.
Many times, making computers and their operating systems more accessible can actually make them easier to use by everyone. For example, the accessibility feature commonly referred to as MouseKeys has helped users other than those with disabilities. MouseKeys allows individuals to do all the mouse functions from the keyboard, including precise pixel-by- pixel movement. While this feature has been of great use to people who cannot use the mouse, many computer users without mobility impairments have commented favorably on the use of MouseKeys, especially when doing graphics layout or using CAD software.
Another powerful argument for making computers and their operating systems more accessible has to do with legislation enacted by the U.S. Federal Government. Specific guidelines are outlined in this legislation, and each federal gency must comply with accessibility requirements when purchasing electronic office equipment. The guidelines and requirements are itemized in the U.S. Federal Government's General Services Administration's (GSA) handbook, "Managing Information Resources for Accessibility," and are referred to throughout this document.
In order to identify some of the issues regarding disabilities,use the basic input/output model of a graphical user interface is pictured in Figure 1. In this model, the keyboard and the mouse are the primary tools for data input, and the display is the primary tool for data output. In addition, the keyboard speaker as well as high fidelity audio are tools for data output.
The keyboard and mouse provide a way for navigating through data in addition to providing a method for data input. For example, when the mouse is used for drawing pictures, it is being used primarily for data input, whereas when it is used to select windows on the screen, it is being used primarily for navigation.
This section and the next describe issues regarding graphical user interfaces and physical and sensory disabilities. Provided in each section are examples of direct access and alternative access solutions developed for the Apple Macintosh and the IBM PC.
With respect to physical disabilities, the major problems with the keyboard/mouse input model are as follows:
Many of the problems listed above can be addressed through direct access solutions, or solutions built into the base product. An example of this approach is the EasyAccess utility included with every Apple Macintosh computer sold. Another example of the direct access approach is AccessDOS, which is available free-of-charge from IBM for all users who request it. With the exception of a keyguard, the access features provided by AccessDOS meet all of the GSA recommendations for input for people with a physical disability, and are as follows:
StickyKeys. StickyKeys solves the problem of simultaneous key presses by letting the user first type one key, then the other. For example, in order to enter a Ctrl-Z, the user can press the Ctrl key, release it, and then press the Z key.
RepeatKeys. Some users have trouble releasing a key once they have pressed it. RepeatKeys allows the user to control the time before a key repeats, and the rate at which it repeats.
SlowKeys. Some users with physical impairments accidentally generate unwanted keystrokes while moving their finger or mouth stick to the desired key. SlowKeys allows the user to tell the computer to accept a key as pressed only after it has been held down for a given period of time.
BounceKeys. Some users with physical impairments press a key once, then accidentally press it again as soon as they release it. BounceKeys allows the user to tell the computer to ignore the second press of a key if it happens within a given period of time.
MouseKeys. Some users cannot use the mouse, but can use the keyboard. MouseKeys allows people the use the numeric keypad to simulate all of the mouse functions.
Because they approach the response issues of the keyboard, the RepeatKeys, SlowKeys, and BounceKeys solutions are often grouped together as the keyboard response group.
In addition to providing recommendations for direct physical access solutions, the GSA handbook recommends the capability to augment or replace the standard input devices with commercially available augmentative and alternative communication (AAC) devices. In these cases, some vendors offer special hardware that acts as a mouse or keyboard, and plugs directly into the mouse or keyboard ports. A trackball, for example, can be viewed as an alternative access solution to the mouse.
Other AAC devices provide a standard serial interface, and can be used on any computer system with a standard serial port. For example, the SerialKeys option of AccessDOS provides a "hook" that facilitates the connection of alternative access devices and allows individuals to simulate keyboard and mouse functions by manipulating alternative access devices plugged into the IBM PC's serial port. Another example of an alternative access solution is the DragonDictate /Handcraft /XTrap voice input solution for X described by Keith Packard in his paper, "Using XTrap to Help People with Manual Disabilities," from the 6th Annual X Technical Conference.
The major problems with the screen and audio output models are as follows:
Direct access solutions for people who have trouble hearing include the ability to raise or lower the keyboard volume. In some instances, the keyboard volume may not be loud enough, or the environment may not allow users to have the volume as loud as they need. In these cases, users requiring additional amplification can use special devices which meet the GSA handbook's recommendation of amplification up to thirty decibels.
To accommodate people who cannot hear sounds made by the system, the GSA handbook recommends both visual and auditory notification of significant system activity. As a solution for supporting the variety of audio information available, Vanderheiden suggests the ShowSounds and SoundSentry concepts ("Full Visual Annotation of Auditorially Presented Information for Users Who are Deaf: ShowSounds." 15th Annual RESNA Conference Proceedings, 1992).
The ShowSounds concept consists of a system-wide flag users can set if they would like visual cues for important audio information. Applications can be written to support the ShowSounds flag, and would provide visual cues for auditory information if the ShowSounds flag is turned on. For example, a multimedia presentation with voice annotation could supply an optional dialog box that displays the textual version of the words being spoken if the ShowSounds flag were turned on.
For applications that are not ShowSounds compliant, a special sub-function of the ShowSounds approach, called SoundSentry, can be used. The SoundSentry function essentially monitors the software/hardware that drives the speaker, and provides a visual indication whenever it detects activity. The SoundSentry's utility is limited in systems that have multiple methods for generating sounds (where the more cooperative ShowSounds approach would work best), but is useful with older applications that generate only simple sounds. An example of the SoundSentry sub-function can be found in the ShowSounds implementation in AccessDOS: ShowSounds provides a visual cue when the IBM PC's speaker is activated. Another example is the visual bell in xterm that flashes the xterm window instead of ringing the bell.
People with low vision, or people who can see the screen but need magnification in order to make sense of the data, can use a technique known as screen magnification in order to examine the information on the screen. While the GSA handbook only recommends screen magnifiers to magnify text and/or graphics from two to eight times its normal size, a screen magnifier should also provide the following:
In order to allow blind people to determine the state of latching keys such as the Caps Lock key, audio cues can be provided that sound when a key such as the Caps Lock key has been latched or unlatched. An example implementation of this functionality is the ToggleKeys functionality of AccessDOS that produces an "up-siren" when a key is latched, and a "down-siren" when the key is unlatched.
Solutions that provide alternative access to information on the display can be classified as screen access systems. A screen access system allows a user to navigate through the information on the display and have it represented on a device other than the screen. In this section we describe issues necessary for providing complete access to information on the screen: a two-class visual information model, output devices, and information navigation.
Vanderheiden and associates suggest a two-class information model in order to separate the nature of visually presented information in a graphical user interface (" A Two Class Information Model for Access to Computers and Information Systems by People who Are Blind." 15th Annual RESNA Conference Proceedings, 1992). In this model, the first classification of information presented visually is known as Class 1 information, or information that could have been presented in words (e.g. spoken or in Braille). For example, all textual information can be classified as Class 1 information, as can simple graphics such as icons. Class 2 information is inherently graphic and cannot be described easily and completely with words. For example, pictures and animated images can be classified as Class 2 information.
To provide full access to the information on the screen, a screen access system should output both Class 1 and Class 2 information in a form the user can understand. Class 1 information is commonly represented through speech synthesizers as well as refreshable Braille displays. Class 2 information is much more difficult to represent, and current solutions often present the information in a tactile/auditory manner. An example of a tactile device is the Optacon, which has a small array of vibrating pins that can be controlled by the serial port on a computer. As the user moves the pointer around the screen, the screen access system will map the pixels around the pointer to the vibrating pins, thus allowing the user to feel the graphics on the screen.
In addition to providing access to both Class 1 and Class2 information, a screen access program should also provide multiple methods for information output. For example, some users, especially those who are deaf and blind prefer to use a Braille display over a speech synthesizer. Other users, including those who cannot read Braille, may prefer a speech synthesizer. Another example is the need for a device other than the Optacon. Although the Optacon provides a solution for some people, it is unacceptable to others for different reasons: it does not provide access to grey-scale images or complex graphics, some people do not have sensation in their fingertips, many people do not like the way the Optacon feels, and others complain it makes their fingers sore after extended use.
In order for a screen access system to allow users to be productive, it must provide a way to navigate easily through information. This not only means allowing the user to move from one window to another, but it also means providing context sensitive information such as whether the user is over a Motif push-button or an Athena Widget Set simple menu. The GSA handbook also recommends the following functionality of a screen access system:
In this section, we describe several approaches taken in attempting to provide solutions for X with respect to the problems regarding accessibility. In addition to the underlying requirement of addressing each of the previous issues regarding accessibility, the following guidelines were considered:
Essentially, the overall goal is to modify the X Window System as little as possible, yet provide the basic features necessary to enable a large range of disabilities to be addressed. If this goal is accomplished, it is more likely the necessary modifications will become a default part of the X Window System, and people with disabilities will be able to use a consistent interface from one workstation vendor to another. It is also much more likely users coming upon an X Window System workstation in a public or shared environment will have the access features they require to use the system.
We investigated several solutions for adding direct physical access to the X Window System, and they include the following: an application that sits between the server and the clients, an XTrap system that filters and generates events on the client side, a server extension that filters and generates events on the server side, and modifications to the server itself.
In each of these solutions, the main goal is to provide a mechanism that sits between the input devices and the clients. This mechanism filters the keyboard events by capturing some events and passing on other events. In addition, the mechanism also generates synthetic events when required. For example, when MouseKeys is active, all keyboard events will be captured by the mechanism. Events not generated by the numeric keypad will be passed on to the clients. Events generated by the numeric keypad will be discarded and synthetic mouse events will be generated instead.
The xscope client, which is provided in the contrib sources for the X Window System, allows applications to examine all server traffic. It works by sitting between the clients and the server: all events from the server pass through xscope and are forwarded to the clients, and all requests from the clients pass through xscope and are forwarded to the server.
By making many modifications to the xscope client, an AccessFilter client capable of filtering and generating synthetic events can be created. This method seems extremely trivial and requires no modifications to any parts of the X Window System. Further investigation, however, shows many limitations that include timing and resource problems.
One of the timing problems of the AccessFilter client is associated with the keyboard response group (RepeatKeys, SlowKeys, and BounceKeys). Each of the solutions in the keyboard response group requires timers to determine how long a key has been held down. By placing the timers on the client side rather than the server side, any delays in the network can result in unpredictable keyboard behavior. For example, SlowKeys requires the user to hold a key down for a specific period of time. If this time is inconsistent due to network delays, the user will be presented with an interface that is confusing and erratic.
The resource problems with this solution have to do with both memory and processor usage. The fully-implemented AccessFilter client will actually be a pseudo-server that sits on the client side. In order to get the direct physical access functionality, all clients must connect to the AccessFilter client instead of the X server. Not only will this create a client that consumes a lot of valuable system memory, but it will also spend a lot of its time processing server data. In addition, due to its complexity and need to precisely emulate server information, maintaining the client will be difficult between releases of the X Window System.
In addition to having technical problems, this solution also requires a system administrators to set up the system to start the AccessFilter client before it is used. This problem is often overlooked and has two discouraging results:
In addition to the ability to generate synthetic events, the XTrap server extension allows a special XTrap client to be written that filters all input events before they are passed on to "normal" clients. These particular capabilities of XTrap lend themselves well to creating an AccessFilter application forproviding direct physical access.
Because the XTrap extension is really not intended for the purposes described here, however, we encountered several stumbling blocks while prototyping this system. The major stumbling block was XTrap's event gate that clients can open and close by sending special XTrap requests. If the gate is closed, all events are trapped by XTrap. If the gate is open, all events pass through. Because the AccessFilter client filters events, it must keep the gate closed the majority of the time. In the case where the AccessFilter client wishes to pass an event to the clients, it must send a request to the XTrap extension to open the gate, pass the event, and then send a request to close the gate. Because most of the keystrokes a user generates will be events to be passed on to the clients, the gate mechanism results in a lot of unnecessary server activity.
Despite the stumbling blocks, this solution works fairly well. A well-written AccessFilter could provide customization mechanisms that allow users to set up an environment that works well for them. However, this solution has the limitations of the previous example: it uses timers on the client side and the solution needs to be set up before it can be used. In addition, XTrap is not available on every server, thus the AccessFilter will not be available to all X Window System users.
In an attempt to decrease the amount of server traffic necessary to filter and generate synthetic events, we developed a special-purpose AccessX server extension. The methods used in the AccessX server extension are similar to those of XTrap, except all events are filtered and generated on the server side rather than being passed to a client.
Because a supported way to implement timers on the server side does not exist, the major stumbling block we encountered while implementing this solution was related to the need for timers to handle the keyboard response group. As a workaround, we developed AccessX requests and events that pass information to and from a special client whose sole purpose is to act as a timer for the server extension.
While this solution contains most of the problems associated with the previous solutions, the major problem with this solution is that it requires a special-purpose server extension. This violates the rules of the guidelines mentioned previously, and will most likely result in vendors not including it in their server implementation.
The majority of the problems listed in the previous examples could be eliminated if the server itself was modified to handle direct physical access. Unlike the previous examples, this implementation is relatively straight forward: the server's input event routines need to be modified to handle the event filtering, and timers need to be added to the server's main loop to handle the keyboard response group.
Since this solution requires special modifications to the server, it violates some of the guidelines previously listed. However, because the direct physical access features are fundamental features that enable a majority of users to use the system, this is one case where bending the guidelines is necessary. In addition, because this solution is directly in the server, it is more likely to become part of the base X Window System and will not require extra setup on the part of a system administrator. The resulting solution will be one where direct physical access is available on every system running X, and the independence of end users is increased.
As stated previously, direct access solutions tend to supply necessary basic functionality that meets the majority of people's needs. These features, however, do not fully address the needs of some users, especially those with more severe disabilities. Because of this, it is necessary to support third-party alternative solutions that provide different ways to access X.
One example of an alternative physical access solution is covered in Packard's paper, "Using XTrap to Help People with Manual Disabilities," from the 6th Annual X Technical Conference. In this paper, Packard presents a highly customizable client that uses the XTrap server extension and a new language, handcraft, to generate synthetic events. With this implementation, alternative access devices such as the DragonDictate voice recognition system can connect to an XTrap client that generates synthetic keyboard/pointer events.
In order to support a wide variety of third party alternative access hardware, however, some type of standard format and connection point should be provided within the X server. An alternative physical access solution we investigated is a server input extension that communicates with assistive and augmentative communication (AAC) devices connected to the serial port.
The Trace Center has written a standard communication interface protocol for connecting AAC devices to computers and information systems. This interface, named the Generic Input Device Emulating Interface (GIDEI), has been accepted by several AAC device vendors, and it appears to be an emerging standard. Because of this, continued investigation into the input extension will most likely result in a GIDEI-compliant input extension.
In order to implement the ShowSounds concept, we propose a new environment variable, called "SHOW_SOUNDS." Applications that are ShowSounds compliant would check for the existence of the "SHOW_SOUNDS" environment variable, and provide visual cues for auditory information if this environment variable is defined.
For applications that are not ShowSounds compliant,we expanded upon the SoundSentry concept to make a SoundSentry client that provides visual cues for sounds created by the system. The SoundSentry client uses XTrap to listen for bell requests, and in the case of an audio toolkit, connects with an audio server to listen for audio requests. As bell requests are made, the SoundSentry provides a visual cue such as flashing the window of the client that requested the bell. When the audio server generates sounds, the SoundSentry displays windows that give visual cues as to what sounds are being made.
In our investigations of direct access solutions for people with low vision, we investigated several solutions that magnify the information on the screen: using the enlargement capabilities of the applications themselves, using magnification utilities such as xmag, and using a window manager that manages a virtual screen.
Many applications have built-in magnification features. For example, most CAD applications provide methods for users to zoom-in on portions of a drawing. In many cases, these features work great for providing access to users with low vision. The biggest drawbacks to this solution are that not every application provides the ability to magnify its information and each application provides a different method for magnifying its information. In addition, applications rarely magnify their cursors. Thus, the user might be able to see the information in the application, but will not be able to see the pointer and/or the input cursor. The end result is that users with low vision will not have the ability to use a multitude of applications, and there will be a myriad of ways to magnify information in those applications that do support magnification.
The xmag client, which is available as part of the X11 Window System, allows users to selectively magnify portions of the screen. There are several problems with xmag, however, that might make it unsuitable for some users with low vision:
Xmag is not dynamic, however, and requires users to click on the information they wish to have magnified.
The information that is magnified is merely an enlarged area of a portion of the screen. Because of this, magnified characters tend to be blocky and difficult to read.
Even if xmag were modified to update its display as the user moved the pointer, users still would not be able to see what they are typing. For example, if a user were typing in a terminal window, the magnified portion of the screen would show the area around the pointer instead of the area around the input cursor.
Another technique people with low vision use to make text readable is increasing the font size. In addition to enlarging the text so it is easy to see, this technique results in text that is easier to read because the characters are smooth instead of being a magnified bitmap. In many cases, however, the fonts need to be so large that the user continually needs to move the windows around to see information not on the screen. This can be extremely cumbersome and results in a decrease in productivity.
As a solution to the limitations in the size of the physical display, some window managers provide the concept of a virtual screen that is larger than the physical display. With the virtual screen, the user can position windows as though they are on a very large desktop, and the window manager provides the users with methods for panning around the desktop. An example of such a window manager is tvtwm, which is available from the X Window System contrib sources. With tvtwm, the user can pan around the desktop in several ways: one feature automatically pans the display as the user moves the pointer near the edge of the screen, and another feature is a small window which serves as a map to the desktop.
As with the xmag solution, the major problem with this solution, however, is that the virtual screen will not follow input focus. For example, if a user has a terminal half on the display and half off the display, the window manager will not pan if the user types a line so long it goes off the display. To solve this problem, we are investigating a window manager property that clients can set to specify where the current input focus is in a window. The window manager could watch this property and automatically pan the display if the input focus moves off the screen. For example, each time the cursor position changes, a terminal application could set a window manager property specifying the rectangular region covered by the input cursor. This functionality could also be added to toolkits so applications would automatically get the functionality. For example, if a toggle button were to get input focus in a dialog box, the toggle button widget could set the window manager property to the bounding box of the toggle button.
We have investigated several screen access systems for the XWindow System, and some of them are similar to the Mercator Environment work being done by the Multimedia Computing Group at the Georgia Institute of Technology. Because the Multimedia Computing Group is presenting a paper on the Mercator Environment at this conference, we will not discuss our screen access system investigations in this paper.
In this paper we have presented minor modifications to the base X Window System that make it possible to add direct access features to the base product as well as provide hooks for complex alternative access features. In addition, each of the accessibility features can co-exist in the same environment, thus providing access to people with multiple disabilities. As a result, the X Window System can be extended into a flexible and customizable system that provides people with disabilities the access needed to be productive and independent individuals.
The plans presented in this document are currently under development by the multi-company organization, DACX, which is the Disability Action Committee for X. All implementations will be submitted back to the X Consortium. For more information on DACX, please contact the Trace Research Center at the University of Wisconsin at Madison.
Drake, K. X11 Test Extension , Version 2.1. MIT X Consortium Draft, 1992.
General Services Administration, Clearing House on Computer Accommodation. Managing Information Resources for Accessibility.
Mynatt, E., W.K. Edwards. The Mercator Environment: A Nonvisual Interface to X Windows and Unix Workstations Multimedia Computing Group, Georgia Institute of Technology.
Novak, M.E., J.M. Schauer, J.D. Hinkens, G.C. Vanderheiden. "Providing Computer Access Features Under DOS: AccessDOS." 14th Annual RESNA Conference Proceedings, 1991.
Packard, K. "Using XTrap to Help People with Manual Disabilities." The X Resource: A Practical Journal of the X Window System. Issue 1, Winter 1992.
Peterson, C.D., "editres-A Graphical Resource Editor for Users and Programmers." The X Resource: A Practical Journal of the X Window System. Issue 0, Winter 1991.
Public Law 101-336, American Disabilities Act of 1990.
Trace Research Center. General Input Device Emulating Interface (GIDEI) Proposal, DRAFT Version 1, Revision 3. Trace Research Center, September 1990.
Vanderheiden, G.C. Making Software More Accessible for People with Disabilities. Release 1.2. Trace Research Center White Paper, June 1992.
Vanderheiden, G.C. Accessible Design of Consumer Products . Preliminary/Working Draft 1.4. Trace Research Center. University of Wisconsin-Madison, November 1991. Vanderheiden, G.C. "Full Visual Annotation of Auditorially Presented Information for Users Who are Deaf: ShowSounds." 15th Annual RESNA Conference Proceedings, 1992.
Vanderheiden, G.C., T. Andersen, J. Mendenhall, K. Ford. "A Two Class Information Model for Access to Computers and Information Systems by People who Are Blind." 15th Annual RESNA Conference Proceedings, 1992.
1. Will Walker is a Senior Software Engineer at Digital Equipment Corporation. Mark Novak is an Engineer at the Trace Research & Development Center at the University of Wisconsin at Madison. Henry Tumblin is a Principal Software Engineer at Digital Equipment Corporation. Gregg C. Vanderheiden is the Director of the Trace Research & Development Center at the University of Wisconsin at Madison.