đŸ€– Day 16: Evaluate adopting AI for accessibility testing and share your findings

Welcome to day 16! Today we turn our focus towards the potential of AI to enhance accessibility testing.

Accessibility testing helps ensure that applications are usable and inclusive for individuals with diverse abilities, such as visual, auditory, cognitive, or motor impairments. Ensuring applications are accessible to a wide range of users is, in many cases, a legal requirement but also a moral imperative. As we have seen throughout this challenge, AI can offer advantages in various areas of testing, and its potential to enhance accessibility testing is no different. Let’s dive in and discover how AI can be leveraged to improve accessibility testing!

Task Steps

  • Research AI and Accessibility Testing: Research how AI is currently being used to support accessibility testing. Explore resources like articles, whitepapers, or case studies that discuss:

    • Tools that leverage AI to automate accessibility checks, analyse user interfaces for compliance with accessibility standards, or even generate alternative text descriptions for images.
    • How AI can be used to address accessibility challenges beyond basic UI checks, such as testing for screen reader compatibility or detecting colour contrast issues.
  • Evaluate Applicability in Your Context: Consider your context and reflect on:

    • Accessibility testing needs: Identify areas where accessibility testing is crucial for your projects
    • Potential benefits of AI: Evaluate how AI-powered accessibility testing tools or techniques could improve your current testing process.
    • Potential limitations: Be mindful of the potential limitations of AI for accessibility testing.
  • Share Your Findings: Respond to this post with your insights and learnings. Consider sharing:

    • Links to the resources you found helpful
    • How AI could benefit accessibility testing in your context
    • Potential limitations you identified and how they could be mitigated
    • Your overall perspective on the role of AI in accessibility testing

Why Take Part

  • Improve Accessibility Testing: Gain valuable knowledge about how AI can contribute to creating inclusive applications.
  • Identify Opportunities: Discover potential ways to enhance accessibility testing in your projects.
  • Sharing Your Thoughts: By sharing, you contribute to the collective understanding of AI’s role in accessibility testing.

:mortar_board: Support your learning and the community. Go Pro!

8 Likes

:mag: Research Findings:

Tools Utilizing AI for Accessibility Testing:

In my exploration, I uncovered several tools leveraging AI to enhance accessibility testing in iOS apps:

  1. VoiceOver and TalkBack: These screen reader functionalities utilize AI algorithms to interpret and relay interface elements to users with visual impairments. According to Apple, VoiceOver supports over 35 languages and is used by millions worldwide. [1]

  2. Accessibility Inspector: Developed by Apple, this tool assists developers in identifying and resolving accessibility issues during app development. It provides insights into accessibility properties and allows for manual inspection of UI elements. [2]

Automated Testing Frameworks:

To automate accessibility testing in iOS apps, developers can employ frameworks such as Appium or XCTest combined with accessibility-focused libraries:

  1. Appium: An open-source test automation framework, Appium supports testing of native, hybrid, and mobile web applications. It provides cross-platform support, enabling accessibility testing across various iOS devices. [3]

  2. AXElements: This library facilitates programmatic interaction with macOS accessibility APIs, enabling developers to automate accessibility testing on macOS and iOS platforms. It allows for the creation of custom accessibility tests tailored to specific app functionalities. [4]

:mag: Evaluation of Applicability:

Accessibility Testing Needs:

Identifying areas where accessibility testing is crucial for iOS projects is essential. Statistics indicate that approximately 15% of the global population lives with some form of disability, highlighting the significance of inclusive app design. [5]

Potential Benefits of AI:

AI-powered accessibility testing tools offer several benefits, including:

  • Improved Efficiency: Automating accessibility checks reduces manual effort and accelerates the testing process.
  • Enhanced Accuracy: AI algorithms can detect accessibility issues more comprehensively and consistently than manual testing.
  • Increased Inclusivity: By ensuring apps are accessible to users with diverse abilities, developers foster inclusivity and expand their user base.

Potential Limitations:

While AI enhances accessibility testing, it’s crucial to acknowledge potential limitations:

  • Complex Interfaces: AI algorithms may struggle to interpret intricate app interfaces accurately, requiring human intervention for nuanced assessments.
  • False Positives/Negatives: Automated tests may generate false positives or negatives, necessitating manual verification to ensure accuracy.
  • Evolving Standards: Accessibility standards and guidelines evolve over time, requiring continuous updates to AI algorithms to remain effective.

:books: References:

[1] Apple Inc. (n.d.). VoiceOver. Retrieved from Accessibility - Vision - Apple
[2] Apple Inc. (n.d.). Accessibility Inspector. Retrieved from https://developer.apple.com/documentation/xcode/accessibility-inspector
[3] Appium. (n.d.). Appium. Retrieved from http://appium.io/
[4] AXElements. (n.d.). AXElements. Retrieved from GitHub - AXElements/AXElements: UI Automation for OS X
[5] World Health Organization. (2021). World Report on Disability. Retrieved from World Report on Disability

:rocket: Closing Statement:

In conclusion, leveraging AI for accessibility testing in iOS apps offers tremendous potential to create inclusive and user-friendly experiences. By embracing AI-powered tools and frameworks, developers can streamline testing processes, enhance app accessibility, and contribute to a more inclusive digital ecosystem. Let’s continue to harness the power of AI to build a future where technology is accessible to all. #MinistryOfTesting #Day16Challenge :sparkles:

6 Likes

Hi my fellow testers, for today’s challenge I chose this article on Medium Accessibility testing UI with AI. In recent years, there has been a
 | by Andrew Antoshkin | Bootcamp and from there explored the website of one of the tools the article recommended which was an open source Chrome extension https://chrome.google.com/webstore/detail/axe-devtools-web-accessib/lhdoppojpmngadmnindnejefpokejbdd?utm_source=deque.com&utm_medium=referral&utm_campaign=axe_chrome_logo.

Identify areas where accessibility testing is crucial for your project

As this is a browser extension this would be of most use on our company website & the web based data store that we create & develop for. It contains a lot of combo boxes that I do wonder if they have the required accessible names and if all the buttons do as well.

The tool chosen claims that it can automatically surface the majority of common accessibility issues—almost triple the industry average and that it will give no false positives in it’s scanning. It also claims to tell you exactly what the source of the problem is and gives you guidance for how to fix it. This all sounds great if true but I do wonder how much of those benefits are actually gated behind their paid version.

I’ll definitely be trying this out next week and will report my findings.

7 Likes

Hello @sarahk and fellow colleagues,

Today’s challenge was nice as it opened me to some exciting possibilities for incorporating AI for Accessibility Testing.

I read multiple references and tried out two tools, TestCraft and Axe.

I have video blogged the demos of TestCraft & Axe on my YouTube Channel. Here is the link:

You can find the full summary here:

Do share your feedback and thoughts.

Thanks,
Rahul

6 Likes

Hi, everyone,

for today challenge I look to information about AI and Accessibility Testing, what kind of tools can by used for testing and what is advantages of AI involving in accessibility Testing.

AI tools for accessibility testing:

Digital.ai

Digital.ai Continuous Testing tool provides comprehensive coverage for functional, performance, and accessibility use cases.

aXe by Deque

Axe DevTools appears as a panel in the development tools and is accessible as a browser or Mozilla extension. Before asking the tester questions to produce a result, intelligent guided tests identify the areas to test and perform as much of the hard work as possible.

AccessiBe

AccessiBe offers cutting-edge AI-powered solutions, such as accessWidget and accessFlow, to speed up and streamline creating websites compatible with legal requirements and accessible to persons with disabilities.

The screen reader and keyboard navigation optimization are handled by the AI-powered process, while the accessibility interface is in charge of UI and design-related changes.

Testsigma

A cloud-based automation testing solution, allows users to test online and mobile applications. It captures and logs issues using images and videos transmitted over Slack, email, Teams, and other messaging apps.

Potential benefits of AI:

:heavy_check_mark: AI can analyze website and identify a potential accessibility issues, provide detailed reports.
:heavy_check_mark: AI can automate many of the tasks involved in accessibility testing.
:heavy_check_mark: AI can help to ensure that websites and applications are accessible to people with different types of disabilities.
:heavy_check_mark: AI tools are able to test across multiple platforms and devices.
:heavy_check_mark: AI can analyze large amount of data, identify all potential issues, reduce human error.
:heavy_check_mark: AI algorithms can learn from individual user interactions, for example, an AI-powered accessibility tool could adjust text size, font styles, or navigation options based on an individual’s specific needs.
:heavy_check_mark: AI tools are trained to understand accessibility guidelines, international standards and the specific needs of different user groups.

Recourses:

How Generative AI is Helping to Enhance Accessibility Testing - Accessible Minds (accessiblemindstech.com)

Accessibility testing UI with AI. In recent years, there has been a
 | by Andrew Antoshkin | Bootcamp (uxdesign.cc)

10 Best Web Accessibility Testing Tools (testsigma.com)

4 Likes

I don’t work much on UIs as I believe that time is better spent elsewhere and the naked eye is sufficient for 20+ year old windows forms that rarely change.

So I asked Bing :
as an automation engineer can you give me an overview of accessibility testing

  1. What Is Accessibility Testing?
  1. Key Components of Accessibility Testing:
  • Validation of HTML:
    • Start by validating your HTML code. Correct HTML is essential for accessibility.
    • Use tools like HTML Validator or the W3C Markup Validation Service to ensure your HTML is error-free.
  • Automated Tools:
    • Run automated accessibility tools within your browser.
    • These tools catch obvious accessibility barriers but won’t find everything.
    • Recommended browser plugins include Arc Toolkit, Axe DevTools, and Wave.
    • These tools combined can identify up to 50% of common accessibility issues.
  • Manual Testing:
    • Manual testing involves human evaluation.
    • Testers interact with the application, focusing on aspects like keyboard navigation, screen readers, color contrast, and form elements.
  • Assistive Technology Testing:
    • Use assistive technologies (such as screen readers, magnifiers, or voice recognition software) to verify accessibility.
    • Ensure that your application works seamlessly with these tools.
  • Integration with Acceptance Tests:
    • Incorporate automated accessibility testing into your acceptance tests.
    • Run accessibility checks during code commits and release tagging using tools like axe-core or PA11Y.
  1. Best Practices:
  • Follow the correct order: Validate HTML, run automated tools, perform manual testing, and test with assistive technologies.
  • Address accessibility issues promptly to avoid breaking changes.
  • Collaborate with design systems (e.g., GOV.UK Design System) to resolve any issues related to components or patterns.

Remember, accessibility testing isn’t just about compliance; it’s about creating an inclusive digital environment for all users!

Based on this I investigated the recommended tools Arc Toolkit , Axe DevTools , and Wave

Arc Toolkit ARC Toolkit Overview (youtube.com)

I followed this video and found the tool relatively easy to use, considering
I have at best highly limited experience of html and css.
Then again, the tool was so easy to use I didn’t need much more than that.

Axe DevTools

This tool went further, so perfect for me being a cynic of UI Automated testing. I mean, even a Developer couldn’t go wrong with this, or could they :smiley:

fully interactive and not a piece of css or html in site.
But of course, you get what you paid for. Not sure on costs but I would spend a lot more time investigating this, if we moved to modern day web applications.

Wave

Another tool for the cynic in me on UI Automated testing. Very simple and straightforward to use, and no ccs and html in sight :smiley:
The tool will display any issues and warnings, and guide you to fix these.

All of these tools are useful to meet the minimum requirements, but ethically we should use them to go beyond legal requirements

For a small to medium sized company which attempted the AX route, to adopt accessibility testing for any web applications, then these Tools make a lot of sense over training in Accessbility, HTML and CSS.
As someone who works mostly with data in the backend using SQL and APIs, the Tools would be more attractive than going off to learn more than my basic knowledge of HTML and CSS.
So for me, these Tools would rate very high on the need scale.

The third tool looked the easiest to use and the demonstrator spoke openly about using this as a base tool and advised on some of the limitations. This to me is what we want, honestly and transparency.

With all things, the ethics are a limitation, but this is down to the individual and/or company. These tools will be designed to meet legal requirements as Errors, but we should also always take care of the Warnings and make our sites truly Accessible.

2 Likes

1. About Research AI and Accessibility Testing

Regarding the practice of using AI tools for accessibility testing, I consulted ChatGPT4, and in its reply, it recommended some AI tools that can be used for accessibility testing. The response from ChatGPT4 is as follows:

The application of AI technology in the field of accessibility testing is increasing, aiming to help developers and testers more efficiently discover and solve accessibility issues. Here are some AI tools that can be used for accessibility testing:

1. **Axe Core**: Although Axe Core itself is an automated accessibility testing library, its enterprise edition offers some AI-driven features for identifying and solving accessibility issues.

2. **AccessiBe**: AccessiBe uses AI to address accessibility issues on websites. It can automatically identify and fix a variety of accessibility problems, such as navigation, keyboard functionality, and screen reader optimization.

3. **UserWay**: UserWay provides an AI-driven website accessibility plugin that can automatically resolve a variety of accessibility issues, including color contrast, font size, navigation, and other accessibility-related problems.

4. **Evinced**: Evinced is a development tool that uses AI to help developers identify accessibility issues during the coding process. It can be integrated into the development workflow to provide real-time feedback and suggestions for fixes.

5. **Fable**: Fable uses AI technology to provide accessibility testing that simulates real users, especially for those with visual and motor impairments. It helps developers understand how these users interact with their products.

These tools can help organizations discover and solve accessibility issues earlier in the development and testing process, ensuring their products and services are accessible to everyone. When using these tools, it's important to combine them with manual testing and user feedback, as AI and automated tools may not catch all types of accessibility issues.

But based on previous AI testing challenges, I learned about an AI accessibility testing tool, Applitools, that has been deployed. My task today will focus on exploring the use of the Applitools tool for accessibility testing.

Applitools tool’s official page on accessibility testing: https://applitools.com/platform/validate/accessibility/

Introduction to Applitools’ AI Accessibility Testing Assistant, Applitools Contrast Advisor:

  • Applitools enables teams to run automated accessibility tests to verify WCAG compliance, helping ensure regulatory compliance.
  • Run accessibility tests on every release for maximum coverage.
  • Applitools seamlessly integrates into your existing test automation workflow. It applies visual AI to analyze whether web and mobile applications have potential contrast accessibility violations.
  • Contrast Advisor helps you focus on problem areas, ensuring you don’t waste time and cycles on areas that don’t need attention. Additionally, we’ve integrated Contrast Advisor directly into Eyes, so there’s no need for time-consuming setup steps or workflow changes. Once enabled, you don’t even need to rerun existing tests - you can jump straight to your existing dashboard to start reviewing results.
  • Contrast Advisor is not limited by webpage structure scanning, so it can provide contrast suggestions for websites, PDFs, UX design models, and applications designed for Web, mobile Web, native mobile, desktop, etc.
  • As W3C points out, mobile devices are more likely to be used under conditions such as strong sunlight, which increases the need for strict contrast compliance. Compared to traditional detection methods, Contrast Advisor uses visual AI, enabling it to identify violations in native mobile applications and mobile web.
  • WCAG specifies minimum contrast for text as well as graphics and user interface components, but traditional tools cannot detect such violations. Contrast Advisor uses visual AI to detect contrast in images, graphics, icons, UI components, and plain text.
  • Contrast Advisor can run with Applitools Ultrafast Grid to render and detect subtle contrast differences and potential violations on Chrome, Firefox, Safari, Edge, and IE. Contrast Advisor complies with WCAG 2.0 and the updated 2.1 standards. This includes “AA – Minimum Contrast” and the stricter “AAA – Enhanced Contrast” options.

Official demo introduction of Applitools Contrast Advisor provided by Applitools https://www.youtube.com/watch?v=sGXjPJiQwdk

2. About Evaluate Applicability in Your Context

Accessibility Testing Needs for My Current Project

Unfortunately, the delivery cycle for my current project is quite tight, and the importance of accessibility testing requirements is not too high. However, I will still use Google’s Lighthouse tool to conduct accessibility tests and scoring for each core page of the product, ensuring that each core page has a high accessibility test score. For pages with low accessibility test scores, I will schedule defect cards for repair.

Below is the accessibility test score for a core page of the current project product:

Accessibility Test Score

Potential Benefits of AI

By using the Lighthouse tool for accessibility testing evaluation, it can somewhat meet

the project’s accessibility testing needs. However, after reviewing the accessibility testing scoring rules of Lighthouse, I found that Lighthouse can identify very limited accessibility testing issues. A high accessibility test score does not necessarily mean that the page is truly accessible to everyone.

If there is a professional accessibility testing AI tool that complies with various accessibility regulations, it would definitely improve the efficiency of the project’s accessibility testing and ensure that the project’s accessibility testing results comply with regulations.

Potential Limitations

Because it involves using AI tools, and the current project is an unreleased product, there are certain risks associated with general data privacy security and result bias uncertainty.

3. About Sharing Your Findings

I applied for a trial of Applitools’ Accessibility Testing Assistant, Applitools Contrast Advisor. Below is the trial report:

Trial application link: https://applitools.com/platform/validate/accessibility/

Trying out Applitools first requires registering an account, which must be a company email address and requires providing company information.

After registering an account and verifying the email, there are some tool surveys:

Tool Surveys

Afterward, I chose Playwright and followed the official introduction documents for initialization, https://applitools.com/tutorials/quickstart/web/playwright/typescript/quickstart

  • Obtain APPLITOOLS_API_KEY
  • Set up APPLITOOLS local environment
  • Install Applitools
  • Run tests

The official demo test code is as follows:

import { test } from '@playwright/test';
import { BatchInfo, Configuration, EyesRunner, VisualGridRunner, BrowserType, DeviceName, ScreenOrientation, Eyes, Target } from '@applitools/eyes-playwright';

export let Batch: BatchInfo;
export let Config: Configuration;
export let Runner: EyesRunner;

test.beforeAll(async() => {

    // Configure Applitools SDK to run on the Ultrafast Grid
    Runner = new VisualGridRunner({ testConcurrency: 5 });
    Batch = new BatchInfo({name: `Playwright Typescript Quickstart`});

    Config = new Configuration();
    Config.setBatch(Batch);
    Config.addBrowsers(
        { name: BrowserType.CHROME, width: 800, height: 600 },
        { name: BrowserType.FIREFOX, width: 1600, height: 1200 },
        { name: BrowserType.SAFARI, width: 1024, height: 768 },
        { chromeEmulationInfo: { deviceName: DeviceName.iPhone_11, screenOrientation: ScreenOrientation.PORTRAIT} },
        { chromeEmulationInfo: { deviceName: DeviceName.Nexus_10, screenOrientation: ScreenOrientation.LANDSCAPE} }
    )
});

test.describe('ACME Bank', () => {
    let eyes: Eyes;
    test.beforeEach(async ({ page }) => {
        eyes = new Eyes(Runner, Config);

        // Start Applitools Visual AI Test
        // Args: Playwright Page, App Name, Test Name, Viewport Size for local driver
        await eyes.open(page, 'ACME Bank', `Playwright Typescript: Quickstart`, { width: 1200, height: 600 })
    });
    
    test('log into a bank account', async ({ page }) => {
        await page.goto('https://sandbox.applitools.com/bank?layoutAlgo=true');

        // Full Page - Visual AI Assertion
        await eyes.check('Login page', Target.window().fully());

        await page.locator('id=username').fill('user');
        await page.locator('id=password').fill('password');
        await page.locator('id=log-in').click();
        await page.locator('css=.dashboardNav_navContainer__kA4wD').waitFor({state: 'attached'});

        // Full Page - Visual AI Assertion
        await eyes.check('Main page', Target.window().fully()
            .layoutRegions(
                '.dashboardOverview_accountBalances__3TUPB',
                '.dashboardTable_dbTable___R5Du'
            )
        );
    });

    test.afterEach(async () => {
        // End Applitools Visual AI Test
        await eyes.close();
    });
});

test.afterAll(async() => {
    // Wait for Ultrast Grid Renders to finish and gather results
    const results = await Runner.getAllTestResults();
    console.log('Visual test results', results);
});

However, running the test failed.

Test Failure Screenshot

After investigation, I found that I made a mistake in the first step of setting environment variables. After resetting the environment variables, the demo test could run and pass normally.

Successful Test Run Screenshot

Then, I logged into Applitools Eyes to view the test results.

My Demo Screenshot

When viewing the accessibility test results and launching Applitools Contrast Advisor on the results page, I found that the test results did not display the Applitools Contrast Advisor indicator, showing a difference from the official introduction video.

Official Promo Demo Screenshot

Today’s trial did not go smoothly, but I will continue to try and update more usage results later.

My blog post link:30 Days of AI in Testing Challenge: Day 16: Evaluate adopting AI for accessibility testing and share your findings | Nao's Blog

4 Likes

Day 16

Research AI and Accessibility Testing

After search for articles, this one gathered my notice:

The hypothesis is that AI can simulate the use of screen readers and other assistive technologies, while providing consistency and comprehensive coverage. This seems a big claim to me, but I guess the world of AI is full of them.

It mentions three tools, and three is always a nice number for research, rather than just a couple:

  1. Axe by Deque Systems - this appears to be a scanner of some kind, although the AI part is hard to decipher - Axe AI | Deque Systems. I think the AI tailors to the type of site, if you are image heavy then the tests will focus there.
  2. Tenon.io - Tenon doesn’t exist, but Level Access does. I could see no mention of AI.
  3. AccessiBe - This is a littl more open with its application of AI. AI-Powered Accessibility - accessiBe. They have a model trained on ‘past encounters’ with what works and what doesn’t. Also OCR and Image Recognition technologies for images and automatically adding alt text for example.

Evaluate Applicability in Your Context

I currently work on a mobile app where the extent of accessibility thinking is on text displays sizes. Often the app has not been able to cope with larger sizes. It is important for us as many users are older with poor eyesight and are often using the app on building sites for example.

Of the three above, Axe and AccessiBe would be to investigate further. To be honest, a basic scanner would be a good start for us, never mind AI powered. That isn’t to say a basic scanner is enough, but one must start somewhere.

The tone of AI using your site like someone who is visually impaired (for example) leaves a poor taste. Automation, even enhanced by AI, cannot give you that experience.

3 Likes

Hey All :wave:

For this task, I searched for AI + Accessibility testing and chose a random tool to check. The tool is accessibe.com

I ran an audit on a website that I’ve tested before that is ADA-compliant.

To be honest, I didn’t expect it to be that good, it provided a detailed report with scores for each aspect in terms of screen readability, keyboard navigation, and I, design and readability - it also provided elements that failed any of the guidelines.

However, some of the failed elements in the report were working correctly when I manually inspected it


Failed element in the report:

but it’s there on site

Conclusion: I might use one of those tools to speed the process up if there are some time constraints but will not rely 100% on it. Either way, AI is not human accessibility is adjusted to humans and its testing should not be devoid of human interaction.

2 Likes

Hey there :vulcan_salute:

I searched some tools, and I tried to test Applitools, and for some reason I cannot enter with my account anymore :frowning:.

So I read this article, Why AI-based Accessibility testing falls short and the alternatives | by Experience Dynamics | Medium, and I found some interesting appointments.

It basically explains that the AI tools are great for the basics, but we cannot rely only on tools, we still need to validate the accessibility with the users as it is made to facilitate their lives, so we cannot depend on only the tools, even that it is improving with the AI.

2 Likes

So much has been written here (some of it clearly by AI tools) yet there’s so little insight and so much incorrect information. Most of the tools mentioned don’t use AI at all. Those that do, use it minimally and to little effect.

Right now, even a moderately competent accessibility tester has nothing to gain by trying to use some AI-based tool. They don’t exist yet and probably won’t for some years.

Of course there are charlatans that claim to be using AI, some of whom are mentioned here. I won’t name them because several of them have taken legal action (and sometimes won) against accessibility consultants who quite correctly criticised their products and claims.

The state of AI in general (and accessibility testing specifically) is that it can sometimes enable unskilled people to do mediocre work. If that’s your goal, then go for it. But for anyone who knows what they’re doing, AI isn’t helpful and won’t be for some time.

3 Likes

Web accessibility is important because the internet has to be accessible to everyone for individuals with impairments to have equal opportunities and access online the growing number of areas of our lives, including education, work, govt, business, medical services, entertainment, and more, rely on the Web and the Internet as a whole. (1)

A.I. can help with accessibility testing. (2)

  • A.I. can automate many of the tasks involved in accessibility testing. For example, A.I. can analyze websites and applications to identify potential accessibility issues, such as missing alternative text for images or improperly labeled form fields.
  • A.I. can help to ensure that websites and applications are accessible to people with different types of disabilities. For example, A.I. can simulate the experience of using a website or application with a screen reader or other assistive technology. This can help to identify any barriers that might prevent people with disabilities from using the software.
  • A.I. can help to ensure that accessibility testing is comprehensive and consistent. A.I. can analyze large amounts of data quickly and accurately, ensuring that all potential accessibility issues are identified and addressed. This can help to save time and reduce the risk of human error.

A.I. will improve the web accessibility in a serious way. Involvement of the disabilities community is key. For optimal results, A.I. tools need to be designed specifically for accessibility — and every accessibility fix should be manually tested to ensure that it actually improves on-page experiences for users. (3)

(1) : 10 Best Web Accessibility Testing Tools; 10 Best Web Accessibility Testing Tools

(2) : Accessibility testing UI with A.I.; Accessibility testing UI with AI. In recent years, there has been a
 | by Andrew Antoshkin | Bootcamp

(3) : Will Generative AI Improve Digital Accessibility?; https://www.boia.org/blog/will-generative-ai-improve-digital-accessibility

Other source information

Using Artificial Intelligence for User Interface Accessibility Testing; Using Artificial Intelligence for User Interface Accessibility Testing - SDET Tech

How could AI transform digital accessibility?; Digital automated accessibility testing with AI | test website accessibility

Integrating Accessibility Testing in Software Test Management; Integrating Accessibility Testing | PractiTest

1 Like

I’m falling behind! AI / LLMs for accessibility testing particularly interests me so I spent a bit of time skimming the posts here and searching around. I agree with the folks who point out that vendors are quick to market their products as using AI and ML, when they probably are mostly using good old coding heuristics.

I liked this post from Craig Abbott: Craig Abbott on LinkedIn: A lot of people are suggesting AI and LLMs (Large Language Models) will
 | 12 comments He has this thought-provoking question at the end: Most humans don’t understand the needs of people with disabilities in 2023, so how can we possibly teach a machine to?

I also asked my go-to person for all things accessibility, @adystokes , about this. One thing he replied that resonates with me is, we can use an AI / LLM tool to make sure a page has alt text on all the images. But, can the tool tell us if the alt text is good
 maybe that is coming.

3 Likes

I was thinking about AI glasses that explain things to blind people - and they exist! I googled it and came up with this:

Envision Glasses: Home Edition - AI-powered smartglasses – Envision Store.

It would be interesting to develop a test plan for glasses such as these! They seem a little pricy ($2500), and dangerous if they didn’t work properly. You could supplement them with GPS and other utilities to make them even more useful. I would give them to a blind friend (Andrew) and see what he thought of them! You could also do reverse glasses for deaf people - instead of producing sound for what things look like, it could produce generate subtitles for what things sound like. These would need to be tested as well.

Potential use cases:

  1. Test to see if a blind person could drive around the block.
  2. Test to see if a blind person could go to the store and buy something and come home successfully.
  3. Try blindfolding yourself and attempt tasks such as washing dishes or going to the bathroom.
  4. Try unlocking a combination lock
  5. Try drawing a picture and see if the glasses can tell you what you drew
  6. Have the glasses explain to you what is in the refrigerator
  7. In a blindfold test, see if the glasses can help you not step on a sleeping cat.
  8. If you can afford 2 pairs of glasses, see if 2 people can do tasks together with the glasses like put together furniture.
  9. Successfully get out of an escape room blindfolded with these glasses.
  10. Play frisbee golf with the glasses.

If they passed all ten tests then they would be worth $2500!

-Mark

1 Like

I read : Accessibility testing UI with AI. In recent years, there has been a
 | by Andrew Antoshkin | Bootcamp

Tried: 1. Axe by Deque Systems – Axe is an open-source accessibility testing tool that uses AI to analyze websites and applications for accessibility issues. Its a google extension.

Sample response:


Considering accessibility is important but it can be quite time-consuming. Luckily, with the use of AI accessibility testing tasks can be automated, making the entire process smoother. Nowadays, there are various AI-powered accessibility testing tools available, ensuring that websites and apps are user-friendly for everyone, regardless of disabilities and promoting inclusivity.

2 Likes

Hey Lisa, thanks for sharing such a brilliant article. As a newbie to accessibility testing, every sentence in the post has made me dig deep into this kind of testing.

I resonate with Craig’s statement that, AI can definitely help with some of the automated aspects, but I imagine human analysis, judgement and expertise will still be necessary to make things truly accessible for real people.

1 Like

Greetings Testers!
Resource Link: Accessibility testing UI with AI. In recent years, there has been a
 | by Andrew Antoshkin | Bootcamp
It was a simple yet clearly written blog stating how AI can help in accessibility testing in no time since AI can analyze large amounts of data quickly and accurately, also thereby ensuring all the requirement checks for the system to be used by anyone and everyone.
The blog also mentioned a few of the AI tools used in this section of the testing.

Where the first blog talks about only the good AI can bring in testing the second blog shows both the prons and cons of it.
Second resource link: Will Artificial Intelligence Help Accessibility?
The problem with using AI is clearly mentioned in this one and it also opens up my mind to think in this way as well.

1 Like

Article Link : Accessibility testing UI with AI. In recent years, there has been a
 | by Andrew Antoshkin | Bootcamp

Automated Accessibility Testing Tools:

  • axe Accessibility: This is an open-source library and browser extension that checks for common accessibility issues. It has a feature called “axe Pro” that uses machine learning algorithms to find more complex issues.
  • Tenon.io: It’s an automated accessibility testing tool that uses machine learning algorithms to identify potential accessibility issues in web pages.
  • Deque Systems: They offer several tools such as Axe for Web and Axe for Android & iOS, which use AI and machine learning to provide comprehensive accessibility testing.
  • Lighthouse: This is an open-source, automated tool for improving the quality of web pages. While it’s not purely AI-based, it does employ some machine learning techniques to provide insights into accessibility issues.
1 Like

Hi

Accessibility is one of the important factor while developing an application but most of the time the accessibility basic guidelines are missed during development. Thanks for the topic

I’ve selected the ARC Toolkit and it is a chrome extension. After installation, This tool will be available on Developer tools

Says that it uses the AI algorithm to identify the issues. I’m impressed with its useability and results.

1 Like