According to Microsoft, more than half of all photos on the web lack alt text. Image labels (alternative text) are essential for many people who are blind or have impaired vision because they give text for screen readers to read aloud. Microsoft is now aiming to fill the void by providing auto-generated alt text for web photos.
Existing screen readers currently read out “unlabeled graphic” if an image has no description, which is inconvenient. Microsoft Edge has been updated to improve the screen reader experience for all photos on the internet that do not have alt text.
“When a screen reader encounters an image that lacks a label, that image can be automatically processed by machine learning (ML) algorithms to describe the image in words and capture any text it includes,” explains Travis Leithead, a program manager on Microsoft’s Edge platform team. “The algorithms aren’t flawless, and the quality of the descriptions will vary, but for screen reader users, having some context for an image is generally preferable to none at all.”
Microsoft Edge may now send unlabeled photos to Microsoft’s Azure computer vision API for processing, in accordance with Microsoft’s privacy guarantees. The vision API generates alt text in English, Spanish, Japanese, Portuguese, or Chinese Simplified, which screen readers can comprehend. Microsoft Edge will not attempt to apply automatic labels to photos smaller than 50 × 50 pixels, very big image files, images categorized as ornamental, or images classified as pornographic, gruesome, or sexually suggestive by the Vision API.
This new functionality will be accessible immediately in Microsoft Edge for Windows, Mac, and Linux, but it will not be available in Edge for Android or iOS. You may test the new feature by enabling “Get image descriptions from Microsoft for screen readers” in edge:/settings/accessibility and browsing the web with Narrator or another screen reader.
“This functionality is still new, and we know we’re not done,” Leithead says. “We’ve already discovered some ways to improve this functionality, such as when photographs do have a name, but that label isn’t very helpful.” Continuous picture recognition and algorithm enhancements will also increase service quality.”