New Apple accessibility updates focus on Apple Intelligence

In a series of announcements that just might signal a wider focus on AI at the upcoming WWDC, on Tuesday Apple previewed upcoming accessibility features in the run-up to this week’s Global Accessibility Awareness Day.

VoiceOver and Magnifier will gain AI-powered features that can provide enhanced image description, using the device camera. VoiceOver’s Image Explorer will use Apple Intelligence to give more detailed descriptions of what’s in photographs, scanned documents and labels, for example.

Magnifier users will be able to scan a document, then ask questions about its contents, with answers provided by Apple Intelligence. The feature will also be available to VoiceOver users.

With updates to live recognition, VoiceOver users can press the iPhone action button to quickly ask a question about what’s in the camera viewfinder and get a detailed response. Users can also ask follow-up questions in their own words to get more visual information. These question features resemble what’s available to users of the Be My Eyes app’s Be My AI feature, but it’s unclear whether Apple’s offerings will go further.

Magnifier for iOS and Mac will include the same Apple Intelligence-powered options, which can be used with speech or high-contrast onscreen text. Magnifier users will also be able to speak to the app, to get more specific information about their surroundings, or to ask follow-up questions.

Voice Control is set to get an Apple Intelligence boost, giving users the ability to describe an element onscreen they want to act on, instead of using a numbered grid, or remembering an item’s label. The natural language support should also allow Voice Control users to navigate apps or elements that aren’t labeled for the feature.

Accessibility Reader, which renders onscreen text in ways that are visually more accessible, including larger fonts, high-contrast backgrounds, and clutter-free layouts, will provide AI-generated summaries on demand, and can translate text into the user’s chosen language.

AI-generated captions will be available alongside standard SDH and closed captions, and also in places where no captions are provided otherwise. They’ll be available on macOS, iOS, Apple TV and Vision Pro, and they can be styled to meet the viewer’s taste or needs.

Power wheelchair users looking for a reason to try Vision Pro might find one in this year’s accessibility announcements, especially if they use an alternative drive controls to steer the chair. Those who can’t use a standard joystick to navigate often employ sip-n-puff switches, head arrays or other devices. With this year’s updates, Vision Pro users will be able to use eye-tracking to control compatible alternative drive systems. At launch, Vision Pro will be compatible with TOLT and LUCI systems.

Other updates coming this year include motion cues for VisionOS, improved Apple device handoff for Made for iPhone hearing aids, larger text support in the tvOS interface and more.

Today’s preview marked the fifth straight year Apple has used GAAD week to preview accessibility features coming to its platforms in the fall. GAAD celebrates its fifteenth year.