Don’t have a lot of time to read this one? Here’s the short version: Ask them for a sample deliverable.

If you’ve read my prior messages, you’ll know that I’m not a fan of accessibility audits. I firmly believe that having a skilled development team come in and fix things is more effective and less expensive than an audit.

But, there are a handful of reasons to get an audit:

1. If onboarding contractors is a long & painful process
2. If setting up all the things needed for local development work is long & painful
3. If you have extremely strict security practices that would prevent access to important parts of the system
4. If you have other contractural obligations limiting who can do the work
5. If your chosen vendor will also be training your developers and supporting them during remediation

But how do you choose a vendor for your audit?

There are a lot of vendors out there selling accessibility audit services. Like any other service, quality can vary significantly and, if you’re not an expert in digital accessibility yourself, it can be pretty hard to choose the right vendor. But, I have one easy way to narrow down your choices: Ask for a sample deliverable.

A good accessibility consultant will be able to share a sample of what you’re paying them for. If they won’t provide a sample to you, that should be the first sign that they should not be up for consideration. They may try to say that their work is covered under NDA, but those NDAs don’t prevent them from providing an anonymized example of a past audit.

Once you get the consultant’s sample, all you need to do is review it and then ask yourself the simplest of questions: “Can I take the information in this deliverable and directly use it to make improvements to the product they audited?”. The answer to this question will depend on your company and the product. Fundamentally, the deliverable will contain a list of the accessibility issues discovered during the audit. You should take a look at each of the issues to determine the following:

• Can you understand what the issue is?
• Can you understand why it is an issue?
• Can you understand how to fix the issue?

In our opinion, the last question is the most important one. An audit deliverable that only lists a bunch of things they found wrong is of little value without detailed instructions describing what needs to change. At minimum you should be given the exact rendered code that has the problem and the exact rendered code that would solve the problem – with the caveat that some problems are too complex for this approach. In cases where the recommended repair is too complex for a code recommendation, you should nonetheless expect detailed enough guidance that could be followed by your developers.

The special case of the VPAT ACR

The Voluntary Product Accessibility Template (VPAT) is a document which can be used by customers when doing their market research prior to procuring an ICT (Information and Communications Technology) product. A completed VPAT is known as an Accessibility Conformance Report (ACR). Public sector and higher education customers will frequently ask for an ACR and, depending on the customer, the ACR will be an important deciding factor in which product they buy. This is why it is very important that a skilled third-party should create your ACR.

An ACR is not a sales document, but rather a disclosure document. It is important that the ACR be honest and clear, as doing so will benefit both the customer and you. As I describe in the link above, a dishonest or unclear ACR can cause an array of problems, ranging from a lost sale to a lawsuit.

When choosing an accessibility vendor to create an ACR for you, this is again a case to ask for a sample. As before, a vendor that won’t show you one should be removed from consideration. There are a number of things to look for, but the most important one for you to look at is the “Remarks and Explanations” column

Use these 4 criteria for determining each of the Remarks and Explanations:

1. When they say that their product supports a specific success criterion, have they accurately conveyed the ways that it supports that success criteria?
2. When they say it partially supports a success criterion do they list specific and easy-to-understand information about what exceptions exist for that success criteria?
3. When they say it does not support the success criteria, do they accurately say why it does not support it and the extent to which it does not support?
4. When they say that a success criterion is not applicable, do they accurately and informatively describe why it is not applicable?

The information need not be excessively documented, but it must be clear enough and detailed enough that the consumer of that information can understand the types of problems exist in the system.

Vendors’ samples should be weighed heaviest in your decision

There are a lot of factors to be considered when trying to find the right vendor for your accessibility needs and there are a lot of differences between what different vendors offer. The most important decision point should be whether you are provided with deliverables that put your organization in a position that facilitates your success in accessibility. The best way to verify that is to simply ask for a sample.