It can be a challenging enough to develop programs (and apps) that provide a first class user experience for users of Assistive Technology (AT) such as screen readers. There are plenty of examples of programs ‘out there’ with varying degrees of broken accessibility when used with AT. Certainly enough to demonstrate that good accessibility is not a given and requires effort. So if you want to design specifically for users of AT you can expect a good challenge indeed.
If you add the requirement to support a range of platforms then the challenge gets even more interesting . There appears to be little information ‘out there’ to guide developers in the best approaches and tools available for AT friendly cross platform solutions. In this post I explore the underlying issues. In a later post I’ll present some of the available solutions. While I concentrate on the technical aspects, I do hope it is obvious that good UX design for AT users is of paramount importance.
In general there are 2 key aspects of accessibility and programs designed specifically for AT users; the accessibility of the format of the primary information or documents and the accessibility of the User Interface (UI). Developers want to use a toolkit that enables them to ensure great interaction with both by AT users but are often not sure how to approach it.
Thoughts about developing such AT targeted programs led Prateek Tandon of ProjectPossibility to ask us this question:
is there a particularly good language / toolset / framework to build out-of-box working applications that will work on the major desktop platforms with popular screen readers such as NVDA, JAWS, and VoiceOver?
The answer to that is “it’s complicated”, especially when I generalise it to “will work with AT” Each platform, whether desktop or mobile has development tools which will mostly create accessible application UIs, if you pay attention to detail and perform proper testing. In addition, there are several tools available for creating cross platform native solutions,. Many of these are mature opensource projects, including those for C++, Java and Python. In addition HTML apps and widgets are fast becoming a viable alternative, though not without some complications as we’ll explore below. The requirement for 1st class AT integration adds another dimension of complexity. The ATs on each platform are different (with the notable exception of OS X and iOS, and the Win8 versions). Plus the critical Accessibility APIs that provide the glue between the apps and AT are also different. These accessibility APIs enable AT users to explore and manipulate the application UI and be made aware of status changes or other events.
Considering screen readers, for example: on Windows there is NVDA and JAWs using MSAA, UIA and IAccessible2 APIs, on Linux we have Orca using AT/SPI, on OS X and iOS there is VoiceOver and on Android it’s EyesFree. This complexity makes it much harder to create apps that work as desired on all platforms, especially if, as a developer, you make life hard for yourself by creating your own custom controls. It’s so much better to use standard native controls provided by the OS (buttons, lists etc) as they have accessibility baked in. The next preferable choice is to use toolkits with custom controls that have well developed and tested accessibility and AT support. Developing custom controls from scratch requires you to implement all the accessibility yourself, not an easy task to get right given the variations and edge cases to be considered.
As an example of this complexity Java Swing is popular for UI development of Java programmes. However, it has a least one short-coming for AT users, namely the need to perform a fiddly manual install of an extra package for AT access on Windows (Java Access Bridge). Of course an excellent solution to this specific problem would be for the open a11y community to improve the Java Access Bridge so the problem goes away. This would also be of benefit to users of the Windows Screen reader NVDA who want to access OpenOffice. For OpenOffice users, however, another solution is looking feasible as the community are looking at porting IBM’s work on supporting the IAccessibile2 open standard from Symphony to Apache OpenOffice).
HTML applications are rapidly becoming popular with developers, and look set to become the preferred technology for creating cost effective, portable and maintainable programs or apps. A key advantage is that the browser becomes a virtual platform that is (at least in theory) the same on any platform using any browser. This requires the standards to be followed carefully by the browser developers. The W3C standards are becoming richer and active community input is helping with speed and requirements. Browsers are now multilaterally implementing the standards more accurately and Javascript performance is no longer an issue. Critically, Javascript APIs providing access to platform capabilities are being specified and implemented. Thus full featured HTML applications to match native version are now being created.
However when we consider HTML app accessibility and integration with AT there is a more complex picture. This is due to the deeper stack of software parts compared with native apps. All the parts have to work together to ensure AT can fully access and present the HTML app’s UI. The web browser itself must implement the platform Accessibility API and cleanly map HTML accessibility through it to the AT. It does this for both it’s own chrome and the HTML content being handled. The HTML apps must be carefully developed to use accessibility best practices, especially those that effect AT. In particular good markup must be used and WAI ARIA adds important extra detail necessary for accessible applications, especially for events and updates.
Again you can make life easier for yourself by sticking to the HTML defined controls or using HTML frameworks with built-in accessibility. Open frameworks such as jQuery and Dojo have had contributions from TPG, Mozilla and others who care about accessibility. Other frameworks others like fluid have had accessibility baked in. In addition, when addressing general platform portability issues, tools like Apache Cordova/PhoneGap and jQueryMobile provide mature open source solutions with reasonable accessibility.
The accessibility of touch screen devices is adding more challenges, and also the potential for innovation in UX for AT users. This is being spearheaded by work on mobile devices (though touch is also now becoming a built in option for laptop and desktop devices as the boundaries blur eg keyboard with a tablet or Windows 8). For example the ways blind users can access touch screens is improving through the integrated provision of solutions like VoiceOver, talkBack, kickBack and others. The solutions for others such as those with physical or cognitive advisabilities are still a way behind. This is fertile ground for research and new solutions.
We have outlined the issues involved when creating applications designed to be used via AT and that run on several platforms. In a future post we’ll explore the options available in more detail, but I need your input. If you have any experience of developing programs with AT users in mind pop your thoughts and experience in the comments and I’ll compile them for the next post.