How to write a good Accessibility Conformance Report?

🙋 Questions
, , , ,
1

This is kind of a broad question (intentionally) but there is also a specific example in there to get us going.

I am increasingly required to fill in VPATs to report on our products’ accessibility. We use WCAG 2.1 and 2.2. I have a good testing procedure. I find issues. And I am confident in writing reports for our devs to fix issues.

However, an ACR is eventually going live on a website where it is meant to inform users. And I am not sure what is the best language for this purpose.

Here is an example:
Using automated tools, i have identified an issue with 1.3.1. Info and relationships: The pagination list on our search results page has the aria role=“presentation”. But lists should only directly contain li, script or template elements. I suggested removing the role as it did not appear to make any difference to tabbing order or otherwise. But the developer explained it had a purpose. No need to go into detail, the point is that the issue remains but it is academic as far as i can tell because tabbing and screen reader appear to not have an issue - pagination works fine. Of course, other AT might not, I don’t test for every possible tech. I need to report it. But it is so technical! How is this going to make sense to any non-techy user? In my opinion, the ACR is there to give an average user an idea and overview of accessibility issues they may encounter using the site. And with this one, I just don’t know how to get it across.

This is meant to kick of a more general discussion about how to write a good ACR or fill in a VPAT. How technical should it be? How detailed? Should it point out (potentially minor) issues that I have found even though they don’t match any of the WCAG criteria? Or maybe after much deliberating, I decided that an issue is not actually a rule break but it is still an issue in my opinion - do I mention it or not? Do you write down any explanations for those criteria that pass? Or that don’t apply? Or do you just leave them empty?
I know some of this is up to me as the accessibility expert writing the report, but I wonder if others / the community has come up with best practices for this.

3 Likes
2

Much depends on what you intend the ACR to achieve. If you fill-in the Remarks and Explanations column of the VPAT, you can provide a lot of technical information in a way that’s very readable because it is split-up by success criterion. It also allows you to describe how you meet the success criteria as well as how you fail and what you intend to do about it. How much detail you include is a matter of company policy - it’s not mandated anywhere.

Remember that the VPAT format was primarily intended as a standardised format to help buyers compare software products or services. It wasn’t intended for users.

The W3C online tool
The W3C have a an online tool to help you generate ACRs, and some example reports. We don’t use it, but it’s a good starting point.
https://www.w3.org/WAI/planning/statements/

The GDS template
We mostly work in the public sector, which means we have to use the accessibility statement template created by the Government Digital Service (GDS).
https://www.gov.uk/government/publications/sample-accessibility-statement/sample-accessibility-statement-for-a-fictional-public-sector-website

This is more comprehensive than the W3C statement and includes some sections that are only relevant to the public sector - you can leave those out if you’re in the private sector.

One of the nice (but time consuming) things about the GDS template is that it contains two lists of issues. One is written in terms of the user experience and is for the benefit of users. The other is written in terms of WCAG conformance.

Not all issues would necessarily be included in both sections and they may be written very differently. For instance, you may say “Screen reader users may find it difficult to understand and interact with the carousel on the home page” in the UX section, whereas you might say “The carousel control buttons do not have accessible names and their “button” roles are overwritten by role=“presentation” attributes” in the WCAG conformance section.

The UX section can include issues you found that are not WCAG non-conformances, but the WCAG section must not.

Best practices
There are no best practices. Everything depends on your context.
https://context-driven-testing.com/

Automated testing tools
You must not believe anything they tell you. Ever. They report false positives, they miss most issues, they misdiagnose genuine issues and recommend the wrong fixes. And that’s just the best tools!

You must not base your WCAG pass / fail decision on the behaviour of screen readers or other assistive technologies. However, you can mention any adverse behaviours in the UX section of the ACR.

You must always base your WCAG pass / fail decision on inspection of the source code and user interface. You must always be able to trace your decision back to the various standards such as WCAG, ARIA, HTML, CSS etc.

In the case of your pagination links, it sounds like your tool is wrong - I came across a very similar instance recently where Axe invented a “best practice” that’s almost always going to be wrong.

1 Like