Apple again raise the accessibility bar with OS X switch control

One of the least promoted features in the Mavericks release of OS X, but in my view one of the most important, is the built-in switch access. Why? Well, one of the least supported group of people are those who rely on simple switch devices operated with limited gestures in order to interact with technology. Connecting switches is now less of a problem than it was with USB now a standard. Previously, hardware mods were required thus invalidating warranty, if at all possible. Also newer switch connectivity solutions for mobile devices like Tecla have arrived. Still, lack of built-in software control this has meant a limited choice of software that directly supports switches (eg a few games) or specialist third party switch access software.

A Mac laptop with a bigh yellow switch connectedAs with VoiceOver, the built-in assistive technology that enables blind people to access OS X and iOS devices, switch access in these OSs means consumer kit is now more easily available to those who really need it. Plus, of course there is the usual so called ‘curb-cut’ benefit to us all when we may find it useful in unexpected circumstances (eg triggering behaviour with a wired remote control when other means are not possible).

Who uses switches and how?

People with severely restricted mobility often have limited ability to make fine and/or course gestures and so are unable to operate a mouse, keyboard or touch device with any degree of accuracy. This obviously excludes interaction with most tech these days. For these people, a range of mechanical switch devices and associated assistive technology software provide alternative access through very basic gestures. Such gestures include press/release a big button, suck or blow on a straw and in the case of Stephen Hawkins, twitch a cheek muscle to operate a sensitve switch. Another related technology is eye gaze which tracks a user’s eye movements in order to control a pointer or selection on a device

How does switch access work?

As with screen readers for the blind, software is needed to convert between expected user interface interactions and what the operator wants to use. Generally, switch access focusses on input control, though as users may also need visual support speech feedback is often also provided.

Close up of OSK showing row group selection

As input gestures are greatly limited the interaction ‘events; are greatly simplified, often to  ‘next item’ and ‘select item’. This leads to so called ‘scanning’ when the input focus is moved from one item to the next until one is eventually selected by the user. As you can imagine this makes operation of an user interface or entering text extremely tedious! A number of scanning ‘modes’ aim to alleviate this to some degree and also match user capabilities. Such modes include ‘auto scanning’ where the cursor moves continuously and ‘group scanning’ where groups of items are selected rather than each item in turn. However, this is an area that cries out for research and innovation!

Traditionally, the user experience has been with a simple grid of items that can be selected. Movement from one item to the next in a grid provides simple linear access suitable for limited gestures Compare this to the more random access a pointer or keyboard. Grids are either a complete replacement user interface, as in the excellent Grid 2, or an overlay that lies on top of main UI, in the same manner as On Screen Keyboards (OSKs) now familiar from touch devices.

Close up of Mac screen showing in application scanning selectionAnother approach is direct in-application access, something I experimented with a few years back with Mozilla Firefox in a project called Jambu.  A further, example can be found in Special Access to Windows, In this mode a selection marker (blue rectangle in the photo) is drawn directly around items in the interface, such as buttons or menu items. This indicates they have the input focus and can be selected. The movement is then between items, both across groups and within groups, providing a more immediate, less indirect user experience when compared to a grid.

You can lean more about switch access in ‘Switch access to technology’, a free pdf by David Colven and Simon Judge, available from the Ace Centre.

What have Apple added to OS X?

After a quick play with the new switch control options (available in system preferences -> accessibility) it’s clear that Apple have done an excellent job and cover most requirements. The support allows full control of the UI and text entry, plus switch access to log-in is possible. By the way, switch access landed in iOS 7 a while back and while I haven’t tried it it appears similar from the blogs I’ve looked at.

As you can see in the first photograph, I plugged in a single button to a USB port using a  Joy Cable converter. Out-of-the-box, the default settings worked well providing usable auto scan once I had identified my switch  (by pressing it) and assigning it an action. The space bar is treated as a switch and give an action of ‘select item’ by default. Some normal access was required to set up the options but once configure operation was perfectly possible with just a switch. Though, admittedly, I performed limited testing.

A wide range of options are available and cover differing numbers of switches and scan modes. A home panel grid overlay provides user access to a range of interaction modes. Overlays are provided for pointer control and an On Screen Keyboard. Further custom overlay designs can be built with an editor. Further more, In-application scanning is provided and works well, though I had some issues getting to it from the main panel with my single switch. Other options such as speaking items with the TTS voice enhance the experience.

So all-in-all this is another fantastic addition to accessibility from Apple. Once again, Apple have led the way on this. Windows has had basic switch support via games controllers for years, but no built in accessibility support, other than the OSK. Linux (GNOME) has had GOK and Caribu for a while but no in-application access. Let’s hope other desktop, and more importantly, portable, OSs follow Apple’s lead here.

A note for for developers

As with VoiceOver, the switch access used Apple’s Accessibility APIs to provide access and applications must be written to correctly support this and so be accessible. If you are a developer you should use best practices and either stick to built in controls, or be very clear on how to make custom controls fully accessible. The benefit is more users for you programs an apps. AT developers will find that unfortunately the Accessibility APIs are not very open so creating 3rd party AT support is as easy as it should be.

 

 

This entry was posted in a11y, OS X. Bookmark the permalink.
Skip to top

Comments

14 Responses to Apple again raise the accessibility bar with OS X switch control

  1. Denise Baldwin says:

    My son needs as being paralyzed basically except for eye movement desperately needs to use a Mac to run his pro tools with eye gaze systems. All I can find is for PCs. Any ideas? He was a brilliant musician, producer and thou he can’t move his mind and eyes are perfect and he wants to continue his creativity. Wha

    Whyonly PCs, we need MACs to accept third party. Please get me up to speed on how I can make this happen.

    Thanks for any help you may offer.

  2. Myles Dear says:

    I may have an option for you! I am the Dad of a special needs 8 year old boy, and a software designer by trade. In June/2014 I completed a joint venture with Apperian and Tobii to deliver an in-app eye control prototype system for Apple iDevices. Please contact me for more details.

  3. Myles Dear says:

    Here’s another option for you : buy yourself a decent Windows 8 server with a processor (such as i7) that supports Vt-x virtualization acceleration and USB-3. Then buy a Tobii REX eye tracker (or it’s cheaper sibling the EyeX, which has one less camera). On your server install VMWare Workstation (free trial available, but it does cost a few hundred dollars), install Mac OS (for example: Mavericks) and run your Mac as a virtual machine.
    Control the Mac with Tobii Windows Eye Control, which provides basic touch and drag motions. The Mac will run quickly in this environment.

    • Steve says:

      Thanks for the info. Now Tobii have unbundled their eye tracking hardware we have quite a few options.There are other makes as well.

      A couple of comments:
      * You can run virtual machines on a decent PC, not necessarily a server class machine
      * VMWare Player is free and you can create VMs as well as play them.
      * VirtualBox is an alternative to VMWare that is largely free and open source
      * You can also run OS X directly on non-Mac hardware – see Hackintosh

  4. Amanda Hanna says:

    Hi Denise. I am a mother of a child with CP who uses the Tobii for communication on PC and I am also a musician who has had extensive studio exxperience including running a home studio. I wonder if your son would use Cubase on PC with Tobii for the time being until better eyegaze access to Macs is developed? It’s not as ‘industry standard’ as Protools but is a great program and will do basically the same thing as Protools. Just a suggestion. Hope he gets it all up and running. I hope my son can access music through his eye gaze in the future, too. He is only 11 but is very musical and has perfect pitch and a great ear. All the best.

    • Steve says:

      Thanks Amanda, Denise. It’s create to know Cubase is still going and is accessible enough with Tobii and perhaps similar systems. For Windows, there is also the OpenSource (free) OptiKey which might be useful, or perhaps the developer would be willing to make changes.

Leave a Reply

Your email address will not be published. Required fields are marked *