Apple accessibility preview: More for Speech, CarPlay, and Vision Pro
This year’s accessibility feature preview from Apple, timed to get a one-day jump on the annual celebration of Global Accessibility Awareness Day, continues last year’s expansion beyond the traditional sight, sound, and physical/motor categories. And without saying so explicitly, Apple seems to have teased the kinds of advances that AI and machine learning could bring to the company’s overall OS offerings.
Tell your phone what to do
Last year, Apple added speech as a major accessibility category, allowing someone to let their device speak on their behalf with Personal Voice and Live Speech. This year, new tools emphasize controlling the device itself with speech, even if speech is limited or difficult to understand.
Vocal Shortcuts will let you turn an utterance – it could be a word or phrase or even a saved vocalization – into a command that an iOS device will understand. Switch Control already recognizes utterances, like a “P” sound, an “F”, a “CH” or other letter or combination that speakers without fuller speech capability can make. With Switch Control, the sound is turned into a tap, a swipe, or other interface action. Vocal Shortcuts is intended to expand that metaphor to more specific tasks, including running a shortcut, opening a specific app, launching Control Center, and more.
Listen for Atypical Speech captures language that is recognizable as speech but might not be easy to understand. For years, Siri hasn’t done a great job at recognizing accented speech, and it’s worth speculating about how this focus on atypical speech could be applied to the wider voice recognition landscape in all of Apple’s OSes.
Personal Voice, an existing feature that people at risk of losing the ability to speak can use to create a synthetic version of their own voice, will expand to more languages, including Mandarin Chinese.
More access behind the wheel
For the first time I can remember, Apple is bringing accessibility-specific features to CarPlay and naming them. The speech focus is apparent here, too, as CarPlay gets Voice Control – a step beyond using Siri to command CarPlay apps; Voice Control lets you speak interface actions, like “swipe” or “tap” while using CarPlay.
Sound Recognition for iOS flashes an alert when your phone hears a chosen noise, like a smoke alarm, a baby crying, or water running. That’s helpful if you can’t hear these sounds in your environment. The new CarPlay version of Sound Recognition will focus on traffic sounds, like emergency vehicles or vehicle horns, and get the driver’s attention via the car’s display. Then, there are visual enhancements, like color filters, to make the CarPlay interface easier to customize for your visual needs.
A new feature for passengers who are subject to motion sickness when using their phone in the car will place a set of dots onscreen. They form the equivalent of a horizon, so you can keep using the phone without feeling sick.
The eyes have it
Eye tracking is a marquee Vision Pro feature, and it’s now more fully integrated into iPadOS. Developers can currently use included eye-tracking APIs to make hardware accessories compatible with the iPad. But those special-purpose devices – often head-mounted pointers or switches – are expensive. This year, support is coming at the system level, meaning someone who can’t touch the screen can use eye tracking to control the iPad directly.
Within the Assistive Touch interface, there will also be a virtual trackpad feature for iPad users who don’t have a physical one but want to use Assistive Touch gestures. Just like support for physical mice and trackpads, could support for a virtual trackpad find its way to mainstream parts of iPadOS – this year or in the future?
More for Magnifier to do
When Apple converted Magnifier from an iOS feature into a full-fledged app, it began adding seemingly disconnected accessibility enhancements to it, like People Detection and Door Detection. They’re both great features, but Magnifier isn’t necessarily the most intuitive place for them.
More logical is this year’s addition of a reader mode for blind or visually impaired users. Capture a document (like a restaurant menu or a piece of mail) with Magnifier and invoke the reader mode to have the text formatted in an easy-to-read layout that uses your preferred font and/or color. It feels a lot like Reader mode in Safari, and that’s a good thing.
More Vision Pro accessibility
The Vision Pro itself will gain new accessibility features, including systemwide live captions and movable captions that are specifically applied to Apple’s immersive video content. Apple also promises new Vision Pro support for cochlear implants and other hearing devices. For low-vision users, visionOS fills in a few display options that were missing from version 1.0, including Smart Invert Colors.
Something for everyone?
Most of what’s been previewed for accessibility this year is quite modern – even the tools that build on past features are based on relatively new categories, like speech. Enhancements to CarPlay and Vision Pro prove that the company continues to bring accessibility along when it innovates for the user base as a whole. Will we see enhancements to accessibility standbys like VoiceOver, especially on the Mac, where not much has changed in recent years? Perhaps, but probably not until beta time this summer.
[Shelly Brisbin is a radio producer and author of the book iOS Access for All. She's the host of Lions, Towers & Shields, a podcast about classic movies, on The Incomparable network.]
If you appreciate articles like this one, support us by becoming a Six Colors subscriber. Subscribers get access to an exclusive podcast, members-only stories, and a special community.