I think the approach would be fundamentally the same, although the tooling may vary. WebAIM have some information on accessibility testing around javascript that is worth a look WebAIM: Accessible JavaScript - Overview of Accessible JavaScript. Having never worked specifically on a product using a site exclusively generated by javascript I canāt say I have experience to call upon but the fundamentals of the site structure and identifying accessibility issues should be the same
1 Like
Things like this really grind my gears! Not least is the fact that āsenior developersā who donāt care shouldnāt be senior. Breathe Ady, back to the question.
You raise a really good point Olli and welcome to the Club. The attitude that accessibility is a ānice to haveā or that āweāll think about that laterā has been very prevalent. Hopefully thatās changing as people realise that morally, itās the right thing to do. I have a talk called Accessibility, Assumptions and Arguments on this very subject to try to spread awareness and give people some tools to have these discussions. Iāll try to summarise the key points below and if you want to see the talks itās on YouTube and the link is here, https://www.youtube.com/watch?v=3fYSL77f7T0&list=PLBQEnHHRWGFpDJoG4u42AfX2K-L4Na_So&t=2s&index=2
Legally: In most places in the world there are equality and disability acts covering access to services. Thereās a growing trend of companies being sued over digital accessibility issues with one of the most recent being Dominos Pizza fined for their app not suitable for visually impaired users. In that ruling the judge said it is reasonable for the app to meet Web Content Accessibility Guidelines (Web Content Accessibility Guidelines (WCAG) 2.1). Now that a judge has ruled that it can be used as precedent in other cases.
Reputational: With social media amplifying complaints if a user has a bad experience word can get round quickly doing reputational damage. With enough traction these can get picked up by the media.
Financially: Everyone who cannot use your site is a lost customer leading to lost revenue. If they say something along the lines of, āitās only a small number of usersā hereās a couple of key facts. Around 80 million people in the EU are affected by a disability to some degree. There are over 30 million blind and partially sighted people in the EU. 37% of the EU population aged 15 and over reported physical or sensory limitations. If you then add temporary or situational incapacities (https://www.microsoft.com/design/inclusive/) thatās a lot of lost revenue!
2 Likes
Hi Olli and welcome to testing and Ministry of Testing,
As always with testing, context is important and the answer may be slightly different depending on the purpose, usage and domain of your product. James Sheasby Thomas touched on many of these in his Testbash Manchester talk in 2017 and heās a great guy with lots of knowledge so I make no apology for referencing him ā a blog post on the topic can be found here - Accessibility testing crash course - James Sheasby Thomas (@RightSaidJames).
If we are talking about a product/site in the public domain for example the list is endless. The main factor in my honest opinion is that itās the right thing to do and drives inclusion, in much the same way as itās the right thing to make things like public buildings, transport and employment opportunities accessible. Unfortunately the morale argument is all too often over looked.
So how about the legal aspect?
Level AA of the Web Content Accessibility Guidelines is a minimum legal requirement for any ISSP (Information Society Service Provider ā anyone providing a service through a website). Basically website owners neglecting to provide a service to a disabled person (due to inaccessibility) that is available to other people is deemed unlawful discrimination. Not something heavily enforced but a complaint by a user could see a lawsuit and has seen lawsuits in the US.
If you prefer positive persuasion then an accessible, usable site will ultimately open the product up a much larger market share. More than 12.9 million people in the UK officially have a disability and that doesnāt take into account the work going into improving the diagnosis of challenges like autism. Include the fact that there is an ageing population with a massively increased computer literacy potentially developing restrictions associated with ageing (eyesight, hearing, motor control). Thatās quite a large potential market share competitors with a more accessible product may be appealing to. Even if your product is a service specifically for private companies, employers are increasingly driving to make companies and employment opportunities more inclusive and accessible. If software or online services they use limit this are they likely to start looking elsewhere?
Finally driving usability and accessibility will on the whole create a better user experience of the product and a better user experience has to provide competitive advantage right?
2 Likes
Hello Julian, in terms of getting buy in please see the response to Olli. In terms of what you can do, advocating for accessible software is a great start but not always easy. If you are beginning development then making it accessible is basically a design choice so the cheapest way. There are many places to go for accessible code such as https://inclusive-components.design/ where Heydon Pickering has made many elements freely available. Thereās also no reason not to test for accessibility and raise any issues found.
2 Likes
Hi Julianne,
So this is one of the biggest challenges Iāve seen around Accessibility. Iād be lying if I said Iāve always been clear on the value of accessibility myself. Itās interesting how life guideās us and my engagement in/increased awareness of the importance of accessibility is in a big way down to my son being Autistic. Seeing how efficient he is with technology to then come across challenges because the app/site/product heās using isnāt as accessible as it could be. As I mentioned in the answer to Oli, the three factors (and I picked these up from James Sheasby Thomas) I tend to focus on when trying to convince people of the importance of accessibility testing are ā Legal Requirement (Equality Act 2010), Morality ā itās the right thing to do, market share ā greater potential customer base, competitive advantage ā a better user experience therefore increased user satisfaction.
The important thing here is that you need to know your users to be able to grasp how important Accessibility Testing is to them, and if itās important to the users then that makes the sell to higher ups a little easier.
Again without knowing how you are working currently, Iāve always been a big believer in testing being present at every stage of the development process, in which scenario, if you can identify areas of concern around accessibility based on previous releases you can raise those concerns early and the overall cost of accessibility is reduced. Compare that to the cost of a lawsuit or finding fundamental accessibility issues in a product just before release and the cost saving is quite big.
Interestingly the government website has a plethora of information on both the importance of Accessibility Testing and involvement of the team in accessibility.
2 Likes
Hello Ben. When you say resources do you mean places to practice? If yes then literally anywhere on web, app or mobile you can put those skills to use. I contact site owners if I find accessibility issues, some are more responsive than others! You can also google ābad websitesā and thereās all sorts of terrible ones highlighted to play on, errrr I mean professionally practice your craft, ahemā¦
I suppose one of the most famous bad websites in https://www.lingscars.com/ as an example of what not to do!
2 Likes
Iāve seen teams take a copy of the products they develop and request changes to actually build accessibility issues into the product in order to validate that accessibility tests and tooling is performing as expected. The bonus value from this is that it built awareness with the development team around accessibility issues so they could be understood and considered early on.
Otherwise I know there are sites built to have accessibility issues to allow people to practice accessibility testing against. A couple of examples are:
https://www.washington.edu/accesscomputing/AU/before.html
https://alphagov.github.io/accessibility-tool-audit/test-cases.html
2 Likes
Iām sure there are a few here too but my WiFi has crashed
2 Likes
Thank you for your questions Pekka. There are some things you can add to a CI pipeline, for example we have checks on commit related to a skip to all content being present, the ability to tab to all elements and missing alt-text. From what Iāve read automation can only cover up to 40% of the guidelines and not always completely so there will be things you can do. Iād be very interested to hear how you get on.
From my experience and reading, the most common pitfalls of failing accessibility are often the simplest to fix. Missing or badly described alternative text for images. Contradicting labels which confuse screen reader users. Tabbing and keyboard traps. Poor contrast and small text size. Overly complicated language use. āClick hereā links rather than something clear and descriptive.
3 Likes
Hi,
Accessibility is no different than any other testing with regard to the fact that the quicker the feedback loop the better. So as long as you donāt treat these checks as the full solution to accessibility challenges and understand what value they can give you before you implement then they are definitely worth it. Again the gov website has an interesting Accessibility tool audit which I found interesting:
https://alphagov.github.io/accessibility-tool-audit/index.html
With regard to pitfalls I donāt profess to have done accessibility testing against enough products to really identify pervasiveness with any confidence however based on two things, accessibility testing Iāve been involved in and seeing my son (who has Autism) use software, I would say the following:
- Clarity of controls and links etc ā often a lot of effort goes into creating a design that looks slick and in my experience this can often result in clear buttons, links, input boxes etc being sacrificed for a ācool designā
- Lack of keyboard accessibility ā Having areas of the software including buttons or fields that just canāt be accessed through the keyboard.
- Lack of or poorly named alt attributes ā obviously causes issues for screen readers
- Lack of consistency of design and behaviour ā My son canāt read but he often figures his way through software and websites using some degree of his own intuition/trial and error. Where he tries to use things in a similar way in various points in a product/site or across sites and the behaviour is different or the design in inconsistent, it can result in violation of expectation and in a worst case scenario a melt down.
2 Likes
Hey great question. Thatās an interesting challenge you have their and Iām not going to profess to have any experience with them however I am fascinated as online ads can be an accessibility nightmare for my son in their own right.
With regards to question 1 - How do you typically test them, do you already have or is there an option for some kind of test harness (non frame host) to be developed so you could test the ad in isolation of any page it may appear in? Is testing the iframe itself for accessibility relevant in your context (do readers pick them up as ads for example) or is it just the content that is within your context? Are they cross domain iframes? Youāve probably already looked into them but could Bookmarklets provide any value?
http://pauljadam.com/bookmarklets/index.html
For question 2 I can definitely give an opinion. So whether itās advertising or otherwise the clarity of information being provided is important in accessibility but there isnāt a clear right and wrong as, for example, autism is a spectrum and therefore the challenges can be diverse. If I take my son as an example, typically he has a short attention span so if itās not clear or doesnāt grab his attention he will move on however if you get his attention he may become quite engaged. He canāt read so if the message is clear through some kind of visual representation then he is more likely to pick up on it.
That said itās a fine line because he is very focused and builds expectations of what he is expecting to see/do on a given site. Heās more likely to leave a subtle add on a page however an add that pops up, has sound or in any other way gets in the way of what heās focused on, causes a violation of his expectations and worst case scenario we end up with a meltdown, best case he puts his ipad down and walks away from it. As I mentioned in my answer to Gem, I find understanding your emotional response to something important in considering accessibility ā does it shock you, annoy you, bore you, frustrate you etc. Then consider how you can compare that emotional response to the emotional response of others, ideally a diverse group including disabilities and neurodiverse people.
2 Likes
Hi Bruce. One of the main reasons for agreeing to do this was to push myself and hopefully learn too.
So this is a brilliant question. Butā¦ The honest answer to 1 is Iāve no idea! This isnāt something Iāve any experience of or with but Twitter and The Club might be good places to reach out. The only thing Iāve come across on this is discussion on the frames having titles. This article on WebAIM gives some practical advice, WebAIM: Creating Accessible Frames and Iframes. Iād also experiment with keyboard only use, a screen reader and if applicable, accessible mode on mobiles.
For 2 Iāll refer to an earlier answer and my accessibility quadrants. Readability is important and one of the simplest ways of checking is to copy text into a Word document and spell check with āShow readability statisticsā enabled in options. (File ā Options ā Proofing). When spell check is done it pops up counts, averages and readability details. The three sections are % of passive sentences, Flesch Reading Ease and Fesch-Kincaid Grade level. There are other online tools of various quality available too.
Iād definitely be interested in reading how you get on, obviously in your own inimitable style! 
2 Likes
Hi Heather and thanks for having us at the club.
Iāve never been involved in designing personaās specifically for accessibility testing but Iāve worked with a UX expert in the past to build personaās to better define the user journey and this overlapped into accessibility. Obviously this will very much depend on your product and the potential target audience however Iād be tempted to look online for some examples first before reinventing any wheels, for example the government website has two fairly cool pages, one with interviews with people who are affected by poor accessibility (Accessibility and Me - Accessibility in government) and also some ready made personaās/profiles tied into accessibility (Understanding disabilities and impairments: user profiles - GOV.UK)
Itās difficult to really recreate the true extent of someoneās accessibility issues because of the diverse nature of disabilities however in a team I worked in previously, we actually blind folded one of the testers and setup a screen reader against the site to see if he could navigate the site whilst blindfolded. Unfortunately, itās harder to really recreate the challenges that an autistic person for example might have. Iāve mentioned in a couple of earlier questions building emotional maps of the software/site ideally from as diverse a group as possible.
To this end I would always encourage companies to look at the diversity of their teams, if your teams arenāt inclusive then you will struggle to build an inclusive product. Peoples biases are affected by their challenges and their input into the product will include their own biases. You could look at outsourcing that accessibility testing but ultimately the responsibility for the accessibility of the site sits with the development team (product, dev and test).
2 Likes
Hello Heather, thanks for your questions. Iām going to try to answer them together if thatās ok? Thereās a couple of ways in which Iāve included accessible traits into personas and my testing in general. They come at this from a couple of different directions. We have existing personas that relate to different mortgage āstatesā such as in arrears, has an arrangement to pay back arrears etc. I test as those but add an additional trait such as, keyboard user, screen reader user or some other trait. One of the tools I use is called Funkify a disability simulator available as a Chrome extension. Details can be found at https://www.funkify.org/. So Iāll enable a trait and test in that mode.
Another way is to start with an accessible persona and test as them. There is a great alphabet of accessibility issues that can also be used as personas on The Pastry Box Project at https://the-pastry-box-project.net/anne-gibson/2014-july-31. Thereās also a gov.uk set that can be found at Understanding disabilities and impairments: user profiles - GOV.UK. Let us know how you get on and good luck.
2 Likes
Wow Paul, I think potentially there might be enough there for the answers to fill a book, or possibly two! Thereās certainly not enough time in an hour to really get into all these. Hopefully I can say some things that may help.
I will say that finding industry standard tools may be problematic and Iād suggest thinking about what you want the tools to do first, then find the ones that best match. Several tools have been brought up in the answers so far that are popular and very useful.
I have no experience or knowledge on an SDLC (Software Development Life Cycle) specifically for a team performing accessibility testing, or testing of any kind. My experience is with an SDLC for development that mentions something like, ātesting techniques will be employed through the development processes. It could also reference approach or strategy documents, which might suit your purposes more closely.
It may well also be worth thinking about whatās most important first and making some decisions. Those may well inform the questions that follow nullifying any quick decisions now.
2 Likes
Wow lots to get through here Paul so bear with me, Iāll try my best.
As far as your question with regards to industry standard tooling, Iāve never really bought into the idea of industry standard tooling. There are different tools that do different things well and which tools are best for you will depend on how you work, your tech stack, what you are trying to achieve etc. What you need to look for and ideally trial is which the tooling supports and adds value to your approach/strategy.
Iāve used manual tools to support my accessibility testing, things like colour contrast checker (https://contrast-ratio.com/as recommended by James Sheasby Thomas) and ChromeVox screen reader. Iāve never used it and Iām not sure how much value you would get from it unless you are working in a domain requiring a lot of compliance analysis and auditing, but there is an accessibility test management type tool - Worldspace suite from Deque systems.
With regards to Accessibility testing automation, the tools Iāve used most are WAVE and ASLint, of which Iād probably say I prefer WAVE but that may just be because I used it for longer and āgot used to itā. Iāve not used aXe but Iāve heard positive feedback on it as a tool, mainly from developers who have used it. If you havenāt already seen it the gov.uk team did some pretty comprehensive analysis of accessibility testing tools at:
https://alphagov.github.io/accessibility-tool-audit/
As far as the SDLC is concerned, again it depends on the team, the current approach, tools/skills and knowledge. As with testing in general in my opinion the ideal strategy is for testing to be included from analysis/design right through to release and then continued analysis against live. Iād want to see Accessibility considered in story analysis and in defining the technical solution. Where accessibility is called out in story analysis prior to development it can be treated as any other required behaviour identified in the story and tested accordingly including the development team building it into any integration tests run on build. Including accessibility in any continuous integration/continuous delivery pipeline helps to keep focus on accessibility and identify issues arising against existing product.
Iāve worked in teams where tests were built by developers in aXe for accessibility testing and then testers picked up āmanualā accessibility testing at a story and a release/regression level and teams where testers picked up all of the accessibility testing retrospectively before release as part of regression testing and the gaps identified are those Iāve already mentioned. Build an accessible product rather than checking if itās accessible after building and you will reduce the pain and cost of accessibility.
Hopefully Iāve touched on most if not all of your points. Happy to discuss further if there is anything not clear or not answered.
2 Likes
Hi Dan.
Iāll start by pointing you at the gov accessibility project and the published results. Whilst it might not be 100% up to date, the link below shows a breakdown of accessibility issues intentionally included in a test site and which tools caught which issues.
https://alphagov.github.io/accessibility-tool-audit/results.html
Itās interesting looking through the issues that no tools picked up. There is an interesting array of issues many of which are visual and I think fairly likely to be picked up through completing manual accessibility checks.
The ones that I think are important (and that may be based to some degree by my own biases) because they are potentially too difficult to identify/understand are around Neurodiversity. Things like consistency of behaviour throughout the site, clarity and organisation of content, avoidance of excessive stimulation like visual noise. Making sure non-neurotypical people donāt feel uncomfortable or unable to use the site because there are too many violations of expectations.
2 Likes
Hello Dan, firstly I really hope āif accessibility testing gets doneā isnāt even coming into the equation!
I covered this (hopefully) in my answer to Pekka regarding the most common pitfalls of failing accessibility. Some of those can be automated to some extent but things like consistency, tabbing orders etc. need a human assessment. In terms of priority Iād say the top two are keyboard only use and supporting assistive technologies (screen readers etc.). Ease of use, language and contrast following those. From the updated Web Content Accessibility Guidelines 2.1 it covers third party tools so it is really not getting in the way of or restricting them.
2 Likes
Great question and screen name Professor! Particularly over the last few years, accessibility developer tools have improved in browsers as has the ability to provide accessible mobile devices. Progressive web applications have meant, in some ways, the best of both worlds. HTML5 is a step towards accessibility as standard in the way we build online services.
We are making progress; we just need to keep pushing the will for everyone to consider accessibility. It is something we should be doing and legally it is moving towards having to do this or be punished.
Thereās a lot of discussion about the future of testing being synonymous with Artificial Intelligence and Machine Learning (AI and ML) and they could definitely play a big part. But I believe the future of testing is definitely tied to accessibility testing not only as a fundamental requirement but as a basic human right.
As an example of the way accessible software can improve peoples lives take a look (and everyone else reading can too 
) at this buzzfeed.com piece and see how technology is helping make peopleās lives better, This Is How Blind and Visually Impaired Individuals Use Technology (With Audio Descriptions) Iād highly recommend a look.
2 Likes
Hi,
This is a great question if you donāt mind me saying.
As with much of this context is important. Touch centricity introduces an abundance of accessibility issues across the accessibility spectrum especially smaller touch devices. That said however I think this in turn has pushed providers of touch devices and apps to come up with solutions to those accessibility issues.
You only have to look through the Accessibility options on the iPhone/iPad for example to see the emphasis accessibility has in modern touch devices.
Interestingly I think arguably the proof is in the pudding. The uptake of touch devices is a reflection of how they generally are opening up the market. Would they be doing this without some reasonable level of accessibility. Certainly for neurodiverse accessibility I can tell you my autistic son finds touch devices easier to use than pretty much anything else.
2 Likes