IE9’s Assistive Technology Interface Enhancements

February 15th, 2011

IE9 introduces improvements to the interfaces used by assistive technologies (ATs) to enhance the accessibility of Internet Explorer. These changes relate to IE9’s switch to the hardware-accelerated rendering of HTML content. Previously, some accessibility tools depended on IE’s GDI-based rendering to scrape information from Web sites. Since IE9 has no GDI-based rendering, these features do not work with IE9. Anticipating this problem, Microsoft has worked closely with assistive technology vendors (ATVs) throughout the IE9 development cycle to devise a solution. This blog post describes the changes made in IE9 to enable assistive technologies—particularly screen readers—to work well with its hardware-accelerated architecture.

How Screen Readers Work

Screen readers read web pages as if they were straightforward text documents: users can use simple keyboard shortcuts to read the characters, words, lines, and paragraphs of a Web page. Users access other parts of a web page such as hyperlinks and images in a similar fashion. Additional shortcuts allow the user to interact efficiently with the input controls of a form and other elements of rich Web pages. In general, screen readers enable the user to move seamlessly through a web page, just as they would move through a text document with a cursor.

However, moving through a Web page becomes more difficult when the page employs modern Web techniques such as dynamic content, complex layout, and AJAX technologies. For this reason, screen readers often build an off-screen model of the Web page. The purpose of the off-screen model (OSM) is to keep an updated version of the Web page and the spatial relationships of its text, images, edit boxes, and form controls. The OSM essentially maintains a snapshot of the page, making it easier for the user to navigate around and hear the content on the page.

Efficiently building and maintaining an off-screen model is a tricky task employing the use of many technologies including: the HTML DOM, W3C Accessible Rich Internet Applications (WAI-ARIA) markup, the Microsoft Active Accessibility (MSAA) interfaces, the Microsoft UI Automation (UIA) APIs, and GDI-based mirror drivers.

It is this last technology, GDI-based mirror drivers, which does not work with IE9’s DirectX-based hardware-accelerated rendering. In its place, IE9 expands its UIA interfaces to provide the information ATs previously gathered with a mirror driver.

IE9’s Expanded UI Automation Support

In order to provide the information previously inferred from mirror driver output and to provide a robust, forward-looking solution for building off-screen models, IE9 adds support for the UIA Text Pattern (ITextProvider interface).

Using the methods on this interface and the objects it returns, the assistive technology can gather the semantic unit (character, word, line, paragraph, etc.), attributes (size, style, color, etc.), and layout (bounding rectangles) of all the text on a page or just the text in a specific range (visible, selected, etc.). The methods return text ranges in screen-render-order, allowing the AT to keep track of the z-order of ranges that may overlap.

Using UIA’s rich interfaces and caching abilities, coupled with the other technologies listed previously, we believe it is possible for assistive technology vendors to achieve performance equal to the old mirror driver approach.

Working with ATVs

Microsoft is working closely with assistive technology vendors as they update their products for IE9’s release candidate and general release. Hardware acceleration provides significant performance advantages for all users and with support for UIA Text Pattern, IE9 continues to be accessible to our customers who rely on screen readers.

—Sharon Newman, Program Manager