AudioMORPH
Project Team
Project Director: Melody Moore
Co-Investigator: Dan Ratanasit
Summary / Outcome Goals
Many computer applications, particularly proprietary corporate systems, are difficult for people with visual disabilities to operate without customized modifications to existing screen readers. These customizations require programming expertise and can take several months to complete. The AudioMORPH project is developing a system that automates the process of creating and customizing auditory interfaces for business applications in order to make them accessible to people with visual impairments. The specific goals of this project are to develop software that will:
- Provide auditory access to proprietary and commercial financial and database applications software for users who are blind or have a severe visual impairment; and
- Significantly reduce the amount of time required to create an auditory interface for a graphical application, and eliminate the need for outside professional customization.
Background
Graphical user interfaces have long presented a challenge for users who are blind or who cannot easily see a standard computer screen. Typically, some form of screen reading program is used to scan screens and present information in an auditory form via a voice synthesizer. However, simply reading the entire contents of a screen can be inefficient and tedious, particularly for data-intensive applications such as financial and accounting packages.
The graphical presentation of application software varies widely depending on its purpose and the programming style of its author. Because of the wide variation between programs, adapting a graphical interface for a vision-impaired user can be difficult. A customized configuration must be created so that the auditory interface will behave appropriately for a specific screen layout. This may include enabling features to recognize menus, dialog boxes or other screen objects. It also may include deactivating automatic reading of screen areas that are updated frequently with unneeded information such as a clock display or cursor position display.
The vendors of commercial screen reading programs often provide configuration files for commonly used software such as word processors or spreadsheets. However, any proprietary applications in the workplace must be custom-adapted for an auditory interface. The person performing the customization needs to have a programming background and be completely familiar with the screen reading software’s features and strategies. Most users of screen reading software rarely attain the degree of proficiency required to do the customization; instead, a consultant or employee of the screen-reading vendor must be contracted to do the work. Developing custom software for a small number of users is expensive, and as a result, employees with visual disabilities are at a distinct disadvantage in the general workplace. Experience shows that if custom screen reader scripts are created for proprietary systems, blind employees can perform data entry and data retrieval tasks as efficiently as other employees. Therefore technology that reduces the time, effort, and expertise required to produce customized auditory interfaces for proprietary business systems could open many employment opportunities for people with visual disabilities.
Progress to Date
The initial prototype of the AudioMORPH system is complete. It was designed and implemented for the industry-leading JAWS screen reader (Freedom Scientific). The AudioMORPH toolset includes:
- A user interface that allows a domain expert (sighted co-worker) to automatically define paths through a proprietary business application and map them to the keystrokes of choice
- A data model describing the user interface paths defined by the domain expert
- A code generator that produces JAWS scripts from the data model
- A mapping facility that automatically produces a JAWS keymap from the user interface data model
- Handling capabilities for JAWS special keys
- Demonstration programs showing the utility of the AudioMORPH system and the JAWS scripts that are automatically generated
- User testing is now being conducted
Key Findings
- Usage paths through a graphical interface when being used by a domain expert can be captured and represented to guide the generation of an auditory interface.
- Customized screen reader scripts can be generated from a user interface model in order to produce an efficient and effective auditory interface.
Conclusion
- The AudioMORPH project, completed in 2007, is a toolset that automatically creates an auditory user interface from a graphical user interface for people with visual disabilities. Rather than sonifying every aspect of the graphical interface, which creates a confusing and inefficient auditory interface, AudioMORPH captures paths through an application, "learning" as a user operates the system. The result is an auditory interface that represents the main activities of the system without extraneous information.
Selected Publications / Presentations
- Moore, M., & Ratanasit, D. Representing Graphical Interfaces with Sound: A Survey of Approaches. Journal of Visual Impairment & Blindness. Journal of Visual Impairment & Blindness; Feb 2005, Vol. 99 Issue 2, p69-84.
- Moore, M. and Ratanasit, D. (2005, March 19). AudioMORPH: A Tool To Support The Conversion Of Graphical User Interfaces Into Auditory Interfaces. Paper presented at the Technology and Persons with Disabilities 2005 Conference, Los Angeles, CA.
