Preliminary Draft -- Check Back Soon
The Talking Fingertip is a package of access techniques which allows individuals with a variety of disabilities or limitations to access and use touchscreen-based kiosks. The hardware requirements for the Talking Fingertip techniques are:
Almost all touchscreens available provide continuous feedback of finger positioning and can therefore be used. Software voice synthesizers are available which will work on any standard multimedia PC that is Soundblaster compatible (as well as for any standard Macintosh). Any momentary switch can be used for the Talking Fingertip, although the description of its use here specifies that the button should be a green diamond-shaped button oriented sideways. The button is typically wired to function as the right mouse button, but can be connected to the computer in any fashion.
The Talking Fingertip techniques provide voice and other auditory feedback to the user regarding the content of the screen. It has several modes of operation which different individuals with different types of disabilities can use to access the system:
The Touch and Hear mode is most useful to individuals with low vision or with mild to moderate reading problems. The Touch and Hear mode allows users to operate a touchscreen-based kiosk in the standard fashion. However, whenever they encounter screens they are unfamiliar with, words they cannot read, or text with which they are unfamiliar and which is too small to read, they simply press the green diamon button and hold it down while touching the screen. As long as the green diamond button is held down, touching the screen will cause no action on the part of the kiosk other than the reading whatever text is on the screen at the point they touched. Using the Touch and Hear technique, an individual can therefore easily have any portion of the screen read aloud to them. A description of the screen can also be obtained by touching the far upper left-hand corner of the screen. Touching the upper left-hand corner repeatedly will elicit additional in-depth help on the layout of the screen, the purpose of the screen, how to use the screen, etc. This information can be used to complement any on-screen help already provided by the kiosk.
Releasing the green diamond button causes the kiosk to return
to its standard (silent) touchscreen mode of operation.
In the Touch and Confirm Mode, the Talking Fingertip feature is locked on. Whenever the individual touches the screen, all that happens is that the contents of the screen where it is being touched are read aloud. Special auditory cues are provided to help the individual tell whether they are on a button, over a field, or over blank space on the screen. Text is read aloud as it is encountered. Fields are announced; hot lists are enunciated as the person runs their finger up and down the list; and button status is announced as buttons are encountered. When the individuals finds a button on the screen that they would like to have activated, they press the green diamond button below the screen to activate it. They may take their finger off the screen and then press the button, or they may press the button with one hand while exploring the screen with the other hand.
Individuals with low vision who can see the general layout of the screen but cannot read any of the text, as well as individuals who are unable to read the text for any other reason, can use this technique to easily explore and use screens, whether the screen is familiar or unfamiliar. If the screen is familiar, the individual may be able to go directly to the button or field they are interested in. Touching it will give them a verbal confirmation that they are correctly positioned. They can then push the green diamond button to active the on-screen button or have the entire field read to them.
If the screen is unfamiliar, they can use the verbal description of the screen to get an idea of the layout. This is particularly useful if some of the buttons are close to or the same color as the background, so that they may not be distinguishable by someone with low vision. The individual can also touch each object on the screen and have it announced to them as they explore and use the screen.
This technique is also usable by individuals with physical disabilities.
Even individuals with very severe movement impairments which would
make it difficult to reliably hit the correct key or button can
use the touchscreen when this feature is active. They simply try
to press the desired button. If they hit it, it will be announced
aloud, and they can then hit the green diamond switch below the
screen. If they miss and hit an adjacent button, nothing happens
except that they would hear the button announced. Realizing they
had missed their target, they could simply try again. On well-designed
screens, even people with very severe movement impairments should
be able to hit most button on the first or second try, and the
remaining buttons after a few tries.
With the Speed List technique, a list of all of the buttons, text fields, and other objects on the screen is provided down the left edge of the screen. By running a finger up and down the left edge of the screen, an individual can access and have read aloud any of the items on the screen. As the user runs their finger up and down the list, the name of each object, its status, and any other necessary information about each object is read to the individual. The speech will track the individual's movement up and down the list, interrupting itself if necessary, in order to keep up with the change of position. As a result, it is possible for a user to very quickly scan down a list for an object they know to be there. They can move their finger along listening for just the first couple syllables of each item to find the item they are interested in. If the item being read does not match the beginning of the desired target, they can simply move on to the next item. The individual can also use their knowledge of the approximate location within the list to rapidly jump to the approximate position in the list. Individuals unfamiliar with the screen can move their fingers up and down the list more slowly. They can also make use of the first item on the list, which is always a description of the screen and its function, etc.
As with the Touch and Confirm mode, the individual would press the green diamond switch to confirm any choices once they had been selected from the Speed List. The individual can remove their finger from the Speed List and press the green diamond switch, or use two hands, one on the Speed List and the other on the green diamond switch to confirm selections. Even if the individuals is using two hands, however, it is recommended that they lift their finger from the Speed List momentarily before pressing the green diamond switch, so that they will hear the confirmation announcement of their selection. (If they keep their finger on the screen, they may interrupt the confirmation announcement if they move their finger, since the system will automatically try to keep up with any finger movements since they may be trying to make their next selection.)
The Speed List mode provides quick and easy access to the kiosk
for individuals who cannot see the screen or the screen layout
enough to use it. It provides quick and easy feedback for individuals
who are blind by allowing them to access the entire contents of
the kiosk simply by running their finger up and down the left
edge of the screen, along the edge of the kiosk screen's frame.
People with physical difficulties have also found the Speed List
to be a convenient way to "trap" their finger in the
groove formed by the left edge of the screen and frame, and to
then just slide their finger up and down the list to select items.
Implementing all of these techniques is fairly straightforward.
All that is needed is a list of the objects on each screen, a
verbal name for each object, the boundaries for each object, and
its visibility status. Much or all of this information is generally
available within the modern programming environment, and/or can
be easily generated for each screen. Multi-purpose screens can
be described in generic form, since the content would provide
specific context information. General purpose Talking Fingertip
routines can then be used to present this information to the user
and implement the overall technique.
The technique requires a built-in voice synthesizer. Very soon, these will be standard parts of all operating systems, and there will be essentially no cost beyond the purchase of a multimedia computer and operating system. Today, software voice synthesizers can be licensed for as little as $15.00.