Article
0 comment

AI-driven Accessibility helper for images in SharePoint Online

Recently a new feature has been added to the authoring process in SharePoint online. This new feature helps you to define appropriate alternative text descriptions for your images on pages. In general to specify alternative text for images is possible the first time in SharePoint history. Previously only the title of an image could be defined.

When a new image gets inserted on a page, the image gets analysed by a background service, which assumes the content of the image and returns an appropriate description. Through the help of pattern recognition and artificial intelligence, the returned values are pretty accurate, at least with the pictures I tested this feature so far.

Take for example this picture.

Lit sign in night saying "do something great"

Picture by Clark Tibb from Unsplash

I insert it on a page, and then after a couple of seconds, a result of the analysis was returned. The finish of this process gets indicated by an exclamation mark button.

Screenshot of showing the alternative text suggestion in addition the property pane for longer alternate text informations

Highlights the missing alternative text suggestion after analyses

The suggestion for an appropriate alternative text returned was “sign lit up at night”. If you are okay with this description, you can either accept or define your own. In the web part properties, this value is stored and can be changed in future if needed.

Why is it important?

When someone visually impaired reads the text on the page and recognise the image the title assigned to the picture should always return something related to the content. Not the content of the image. The alternate text should describe what gets shown in the image. A visually impaired person reading the article get to know what other people see there.

Limitations

There are some limitations attached to this overall great new features. The alternative text specified on the image gets written to the web part configuration only. The same picture, out of an image pool used a second time, forces again analyses and enter the appropriate information.

The image in the header is just a background image via CSS; a so-called “decorative image” for that kind of pictures no alternative text is applicable. Visually impaired people won’t even recognise that header image. The solution is to add the same photo to the content again.

In the modern experience, the header image is shown in the rollups of pages, for example, the news section on the landing page. For those images, no alternative text gets rendered, cause this behaviour is because of the missing property directly on the image. People who rely on the alt-tag through assistive technologies will recognise a picture without knowing the content of the image. At least the title, which is a property of the image, gets displayed there.

Privacy concern

By the time I wrote this blog post, I could figure out where the image gets analysed by browsing the network trace. It will be sent to a background service in the Office 365 space, hosted by Microsoft.

This image recognition is a great feature. As far as my test with goes, it was pretty good in recognising what the image shows.

The only concern I have is if the picture shows some confidential data. In this case, you might tend to disable the feature for specific areas. I hope there will be some trust documentation on what happens with the image. For example, no OCR gets applied to the image and so on. This documentation is currently missing, or at least I’m not aware of it.

Conclusion

This new feature is a great addition to the overall functionalities of the “modern experience” stack. Several things are not yet that optimal, but I’m sure we will see this issues fixed.

The yellow exclamation mark display on the image raise a lot of awareness for accessibility. Not every author of an article can be an accessibility expert, but with the small explanation provided on the web part properties, people should get a sense for that.

Finally, let me give you an example that the visual appearance matter for everyone. It must have been already 15 years ago. I worked for the government in upper Austria at this time. We built the first website at this time and accessibility was one of the fundamental requirements for all users. So we consulted an accessibility expert from the University in Linz.

When he looked at the prototype of the web site he recognised that the blue and the yellow we use doesn’t match perfectly together. This expert never saw colours his whole life. The fact he realised that the colours don’t match was all based on the calculation of both colour codes about each other.

Right now this feature is only available on first release tenants.