(This is a working paper of the Trace R&D Center. This paper supercedes the preliminary work presented at the 1996 RESNA Conference as a poster session. We will be continuing to update this document as work in this area evolves.)

Table of Contents:

1. Problem Statement
2. Need for a Standard Protocol
3. Goal and Objectives of the UAI Protocol Effort
4. Status of the Effort
5. Requirements Identified
6. Overview of the Protocol
7. How You Can Participate
8. Other Information
9. Draft Protocol
   
   1. PART I: Protocol for communication from the "auxiliary input" to the "information device"
   2. PART II: Protocol for communication from the "information device" to the "auxiliary interface"

Public information systems and appliances are increasingly being designed to include features which allow them to be used directly by people with disabilities. However, we do not know at present how to design such systems so that they can be accessed by everyone, especially those with severe or multiple disabilities where use of a personal assistive device might be required.

One mechanism for allowing these individuals to access public information appliances would be to include a low-cost infrared link as a part of the devices' design. Users could then interface with the device using their own interface, or on their personal assistive technology. For example:

• Individuals who are blind or deaf-blind could have the information displayed on the ATM screen sent out over the infrared link to a personal display system (braille, voice, etc.).
  
• Individuals with severe physical disabilities could use the infrared link to send selection commands to an electronic library terminal device from their augmentative communication AUXILIARY INTERFACE or portable interface system.

The Universal Auxiliary Interface (UAI) Protocol is intended to be used with a number of electronic appliances that have already begun to appear in the market place. In most cases, the protocol would be used over an infrared link such as the IrDA. Examples of electronic products that the UAI Protocol might be used with include:

• ATMs
• Information Kiosks
• Building Directories
• TV Set Top Boxes
• Bus Stops (Electronic Interactive)
• Fare Machines (ticket machines, etc.)
• Home Appliances (especially touchscreens)
• Informational Telephones, Screen Based Telephones
• POS, point-of-sale
• Home Environmental Control and Security
• Whiteboard, classroom and interactive systems
• Games and Entertainment

The protocol, used in conjunction with a low-cost infrared link, will provide a simple, rugged, low-cost and flexible mechanism for supporting a wide range of auxiliary input interfaces with these products.

The goal is to develop an auxiliary interface protocol "standard" suitable for use over infrared and other communication links which would allow auxiliary input and display interfaces to be used in lieu of the built-in interfaces on a wide variety of mainstream electronic products.

A major purpose of this protocol would be to allow interfaces which are quite different from the interface used on the product to be used to access and use the product. For example, a product which uses a touchpad and LCD display could be controlled by an auxiliary interface device which had only a speaker for output and a microphone and voice recognition for input. It could also be controlled by a device which had only a dynamic braille display for output and an alphanumeric keyboard for input. In order to achieve these goals, a protocol is needed which will allow an auxiliary interface device to easily and efficiently secure, display, and control information from a DEVICE in a text format that can be easily displayed in any desired modality (auditorially, visually, tactually, etc.). The system should allow the auxiliary input device to secure and display any important information needed for operation of the device, and should provide a means of operating all controls needed for operation of the device.

Objectives

The overall objective of this project is to develop an industry standard for allowing assistive technologies or other auxiliary interfaces to connect via an infrared link with standard devices. Some specific objectives include:

• To develop a bi-directional link suitable for use with infrared technologies whereby individuals with different disabilities equipped with specialized access devices could both locate and interact with various electronic information systems.
  
• To develop guidelines or a "standard protocol" for an infrared linkwhich builds upon accepted industry standards, such as the Infrared Data Association (IrDA).
  
• To advocate and work closely with product developers to get the ideas and "standard protocol" and infrared link incorporated into as wide a variety of main stream electronic devices as possible.

The Trace Center, as part of the Electronic Curbcuts Program, in cooperation with CSLI, Stanford, and others internationally, have initiated a project to investigate the use of infrared technology as a means for individuals with disabilities to access electronic information systems.

A series of meetings which coincided with major disability related conferences have been held to gather input, ideas, and feedback from researchers, developers, consumers, etc., regarding the implementation and usage of infrared technology to access electronic information systems. The first such meeting was held in conjunction with the California State University Northridge (CSUN) Conference in March, 1995. Similar meetings have been held at RESNA '95, Closing the Gap '95, CSUN '96, and RESNA '96. As an outcome of these meetings, and communication via the listserve and email, a UAI Protocol with the following objectives and requirements was defined (see below). From these, requirements and a draft protocol were proposed. Subsequently, a prototype system was created; the protocol has been refined based on experience in implementing this prototype.

In addition:

• The Trace Center joined the Infrared Data Association (IrDA) to both listen and contribute to ideas for the application of existing infrared technology in disability related areas.
• A project team and funding have been secured at the Trace Center to keep development of the UAIP moving forward.
• A listserve (IRLINK-L) has been established at the Trace Center to further discussion and to allow for input from individuals unable to attend the meetings.
• The UAIP working group will continue to have meetings to gather input from all interested parties. All information gathered at these meetings will be added to that already available in print or electronic format via the Trace Center Web Site (http://trace.wisc.edu/).

• There should be some type of security provided.
  
• The link should not interfere with other IR uses.
  
• The link should not be blocked by other IR uses.
  
• The link must be bi-directional.
  
• The link must always be looking for a connection (so that the person with a disability can approach a system and have the system recognize them without requiring the person to activate the link from the system).
  
• The link should support talking sign technologies (either directly or by allowing the talking sign technologies to exist side-by-side on the same device so it can be located by a user who is blind).
  
• The user should always be considered in control of the link. (The user makes a request via the link, and the system then responds via the link.)

At the present time, the overall desired connection is seen as consisting of three basic components.

1. a hardware/software protocol for establishing a physical infrared connection between the auxiliary interface and the standard device.
2. a protocol for establishing secure communications over this physical link (when security is required).
3. the establishment of a communication protocol for securing information from and transmitting commands to the standard device.
   

(Item #3 would then define the information which must be supplied by the standard device and would also define the type and format of the information which would be received by the auxiliary interface.

1. Hardware/Software Link

Any connection mechanism can be used for this stage. An RS232 serial connection, for example, would meet the requirements, as would a parallel connection.

In order to provide a low-cost, robust, vandal-resistant and flexible interface, however, an infrared or other wireless link is proposed. Further, the use of the standard infrared link being promulgated by the IrDA (Infrared Data Association) is recommended. Various different but compatible infrared links are being proposed by IrDA to meet the different requirements of consumer and high-speed data equipment. Since the data demands for this application are not high, the recommended standard would be the one with the best range and angle of view to reduce the amount of positioning necessary on the part of the auxiliary input in order to work successfully.

2. Security

In applications such as control of home or office appliances, the security component may not be necessary, and may be omitted. However, in other interactions and transactions, the user may want to maintain secure communications between the auxiliary interface and the standard device. Examples include ATMs or other activities including the transfer of money, or personal and private information.

At this time, we have not established any standards in this area. Perhaps a standard will evolve out of the other IrDA work. We have discussed strategies involving the exchange of public keys in a two-key encryption scheme, with the auxiliary interface first providing a key and then the standard device providing a decryption key in encrypted form.

3. Communication Protocol

The third component would be the communication protocol: what the auxiliary interface and the standard device would use to communicate with each other. The proposed protocol is termed the "Universal Auxiliary Input Protocol" or the "UAI Protocol."

Our objective in creating the protocol was to create something which was as simple as possible and which would place minimum demands on the standard devices. This was done to enhance the adoption and implementation of the protocol by industry.

We also recognize that the nature of interfaces may change over time. As a result, the protocol allows for extensions at a later date as needed in order to support very different interface technologies which may emerge.

The current protocol consists of just four commands which can be sent from an auxiliary interface to a standard device, and two spontaneous messages which are sent from the standard device back to the auxiliary interface.

The four commands are:

1. Reset
2. List
3. Activate
4. Settings
   

The spontaneous messages (e.g., messages not sent in response to a request from the auxiliary interface) are:

1. "One moment"
2. "List changed"
   

Each of these is explained briefly below. A technical description for the commands as well as their data formats are presented at the end of this paper, in the section "Technical Format for the Transmission Protocol."

Reset

This command simply causes the standard device to go back into a state which would be equivalent to it just having been turned on. The purpose of this command is to allow someone who comes up to a kiosk, ATM, or even an appliance where the device has been left in an unusual state by the previous user to reset the device into a known start-up condition.

List

This command causes the standard device to send to the auxiliary interface a listing of all of the pieces of information or commands (buttons, etc.) which are available to the user on the standard device. This listing includes all of the fields, the labels, the text, and other information displayed on the screen, as well as any buttons, controls, or commands which can be issued by a user. Because all of the information presented visually or auditorially by the system would be available in this list, the user would not have to be able to see or hear the standard device in order to have access to all of the information being presented by the device. Because all of the commands and controls would also be available in this list, the individual would not have to be able to touch or reach the standard device in order to completely operate it.

Activate

The "activate" command is used by the auxiliary interface to activate any of the commands in the list, or to request the content of any of the information items in the list.
For example, the "activate" command could be used to push any button on a touchscreen, or to activate any control. It can also be used to ask for the contents of any block of text or other information item on the screen.

Whenever an "activate" command is issued, the standard device echoes an aknowledgement of the command activated (or sends an error message if a nonexistent command is attempted.)

If the command causes the device to be busy for a period of time, the device would issue a "One moment" message. When the device finished being busy, it would respond with a comment indicating its current status.

Any time that the listing of current commands and information changed (whether because of a command issued by the user or for any other reason), a "List changed" message would be sent to the auxiliary interface. The usual response of the auxiliary interface to this message is to immediately request a new list, as discussed under "List" above.

Settings

Depending upon the device, there may be some special settings which pertain to the use of the auxiliary interface link. If this is so, sending the settings command would result in a list of the setting options.

For example, it might be possible to turn off the standard user interface when the auxiliary interface was being used, for either security or privacy reasons. The user may want to turn off the standard controls but keep the display a tive, or turn off the display, or both.

Another option might include a change in the behavior of the standard user interface to enhance its use in conjunction with the auxiliary interface. For example, items might be highlighted on screen as they were activated.

You are welcome to join the infrared list serve discussion by sending email to:

• LISTPROC@TRACE.WISC.EDU

do not put a heading or signature in your email

in the body of your message type:

• SUBSCRIBE IRLINK-L Robinson Crusoe

(if your name is Robinson Crusoe; if not, substitute your own name)

This document and other information on this topic is available at Designing an Accessible World in "Infrared and Other Interface Standards" at http://trace.wisc.edu/world/irstds.html.

THERE ARE FOUR COMMANDS THAT CAN BE SENT BY AN AUXILIARY INPUT:

Two basic commands:

1. LIST
   Function: Send a list of information and action items
   Form: #L<ret>
   

2. ACTIVATE
   Function: Activate given item
   Form: #item reference number<ret>
   or
   "verbal name of item"<ret>
   Examples:
   #12<ret>
   or
   main menu<ret>
   

Two set-up commands:

1. RESET
   Function: Reset system to start-up
   Form: #R<ret>
   

2. SETTINGS
   Function: Send a list of the setting options
   Form: #S<ret>

NOTE: When the user sends the "list" command (#L), the "information device" (e.g., kiosk) may (at its own option) display the names or reference numbers next to the items on the screen. This would be done to facilitate access by users who can see the screen but who must use an alternate device connected via the IR link to activate the buttons on the screen (e.g., someone using a sip-and-puff keyboard to activate a kiosk).

All information sent by the "information device" is in response to commands from the user ("auxiliary interface") or to notify the user of a change in context.

1. In response to LIST, the DEVICE sends a tab delimited table listing all items (information items, action items or commands) in the current context (that is, items which are currently available for reading or action). (See below for details on format for this tab-delimited table.)
   

2. In response to ACTIVATE, the DEVICE would echo the verbal name of the item which was activated.
   
   • If the device is busy for a period of time, it would send a "ONE MOMENT" message.
   • If other events occur as a result of the "ACTIVATE" command which the user should be aware of, messages should be sent to the AUXILIARY INTERFACE so that the user who cannot see the screen is aware that these events or transitions have occurred.
   • If a "ONE MOMENT" message has been issued, then some message needs to be given when the DEVICE is finished. If the DEVICE changes contexts (e.g., displays a new screen), the name of the new screen could be sent as a signal that the DEVICE was finished. If no other appropriate message is sent, the DEVICE should simply send "READY."
   • Whenever the context changes (e.g., the tab-delimited table which would be sent in response to a LIST command has changed), the information device should send the message "LIST CHANGED." (Most assistive technology devices, especially those being used by individuals who are blind, would automatically re-issue a LIST command to fetch the new list whenever they received a LIST CHANGED message.)

3. In response to RESET, the DEVICE sends "RESETTING NOW" when the reset command is received. When it is done resetting, it sends a status message indicating the context (for example, "Idle Screen" or "Main Menu").
   

4. In response to the SETTINGS command, the DEVICE goes to an auxiliary interface settings screen (context), and sends the "List changed" message. The user can then issue a "LIST" command or type in a string to directly activate a setting. Typical settings available through this mechanism would be "blank screen," and "mute speaker." (When a user approaches the kiosk, they might just send a string of commands which would reset the kiosk and turn off the screen and sound and then return to the main menu.)
   

NOTE 1: When replying to an AUXILIARY INTERFACE, a carriage return is not transmitted by the INFORMATION DEVICE, so that the response could appear on the same line of the user's display (although the device could also choose to display each message on a separate line).

NOTE 2: All messages sent by the INFORMATION DEVICE should be terminated with an END OF TRANSMISSION (EOT) (control-d) (hex 004) character.

FORMAT OF TAB-DELIMITED "LISTING" TABLE

• The transmission of the table information should begin with the heading. The heading has three parts:
  1. Table format (only format L1 is defined at this point)
  2. The name of the current context described by the table
  3. The suggested point the user should be placed in the table (usually 1).

• The form of the heading is: <SOH>L1<tab>Name-Of-Context<tab>HiLite<ret>
  where <SOH> is control-A or hex 001.

• The second line in the transmission would be the first line in the table.
  

• The first line in the table (second line of the transmission) would always represent a "virtual" item on the screen (or context) and usually called "Description of Screen and Help." It provides information about the current screen (or context). Activating this first line would cause information about the current screen (context) to be sent to the user. Activating the first line successively would cause more information to be given as follows (the numbers below indicate the information that would be sent in response to that activation -- e.g., the third time the item was activated, the choices available at that point would be described):
  
  1. where the user is (e.g., screen or context name)
  2. the function(s) of this screen (context)
  3. the choices and information available here
  4. layout description of the items on screen (or context)
  5. graphic description of the screen (or context)
     

• <Return>s may or may not be accompanied by <linefeed>.
  

• Each information or action item in the current context (e.g., "on the screen") has one line in the table describing it, and would consist of the following fields of information each separated by tabs (fields 3 to 5 may be left blank). Field 4 should be filled in if the object has a value, such as a checkbox. Trailing tabs are optional. <tab>, <ret>, and <lf> are not allowed in table data fields, since they are used as delimiters.
  
  1. reference number for the item (e.g,. #2)
  2. verbal name for the item (e.g., "Main Menu")
  3. type or description (e.g., button)
  4. value (e.g., whether a checkbox was set or not set)
  5. position (e.g., bottom right of screen, etc.)
     

• The table transmission would be terminated with an END OF TRANSMISSION character (EOT) (control-d) (hex 004).

• Occasionally it may be useful to have the INFORMATION DEVICE specify where the user's highlight should rest when they are given a new list. In some cases, this feature may be used to reposition the highlight in a location similar to where they were in the previous list (of only a single or a few items change in the list, for example). On the other hand, it might be done to reposition the individual to the next most logical or useful place in the list (rather than at the top of the list). To indicate that the highlight should be placed in some position other than the top position when a new list is sent to the AUXILIARY INTERFACE, the INFORMATION DEVICE would provide a "Suggested Start Line" on Line 2 of the transmission as noted above.
  

EXAMPLE OF THE TAB-DELIMITED TABLE SENT IN RESPONSE TO "LIST"

<SOH>L1<tab>Name Of Context<tab>1<ret>
#1<tab>Description and Help<tab>(blank)<tab>(blank)<tab>"invisible located at 0,50,0,30 of 0,0,480,640<ret>
#2<tab>(name of button)<tab>button<tab>(blank)<ret>
#3<tab>(another button)<tab>button<tab><tab><ret>
#4<tab>(name of text)<tab>text block<tab><tab><ret>
#5<tab>(name of field)<tab>scrolling field<tab><tab><ret>
#6<tab>(name of button)<tab>check box<tab>true<tab><ret>
<EOT>