There are an increasing number of tools that can carry out automated accessibility checks. But are they any good?
For quick checks, I have long used Cynthia Says as I find the resulting report informative and easy to understand. Having just re-evaluated Webxact (formerly Bobby) I see that the report has been extended to include other, none accessibility, issues. Now this may be useful for some, but it doesn’t suit my needs.
However, for offline use, I have access to InFocus. Although not particularily easy to use, it is fairly customisable, right down to creating your own test configurations, and removing WAI checkpoints that you feel are no longer relevant.
No automated checker alone can highlight all potential accessibility problems and some may highlight problems where none exist. Human intervention has to be included, as does a certain amount of user testing. Do not think that just because you get a Bobby AAA pass that your site is accessible. It isn’t, or at least, is unlikely to be. All of the user checkpoints and warnings that it flags up should be thoroughly investigated.
Yet that isn’t the end of it. There are other tools out there that may, or may not, be useful. Perhaps the one with the highest commercial profile is Sitemorse. However to evaluate this tool you have to register on the site. It is only later on that you find out that you have signed up to a service that you cannot stop.
There is currently no method available for you to remove any site you have entered into their listings. This means that, once a week, they will test your site whether you like it or not. No indication of this is available on the site, or during, the registration process. In order for site listings to be removed you must email them. This should be an integral part of the system, especially for those that only signed up to evaluate rather than utilise their product.
Then there is the registration form, and subsequent forms, itself.
Shouldn’t a form for registering for an accessibility check be accessible?
They have not used <label> for any of them. WAI checkpoints 12.4, and 10.2, apply. Although these are Priority 2 checkpoints, surely a site dealing with accessibility should display best practice and utilise them!
Sitemorse also publish a league table of known pundits in the accessible world. Off their own bat, Sitemorse test sites belonging to other accessibility experts and organisations. The problem is they use their own automated checking software, and they define a Pass as 100%, with no room for failure. All well and good, but is their software accurate enough to encompass every possibility?
At the moment the answer is a resounding no.
Being an automated tool, it is obviously unable to make a decision about missing alt text for an image, for example, or even if the text provided is correct. Yet they attack any site that fails this report without actually bothering to check the results of the test manually thus greatly increasing the possibility of both false negatives and false positives.
In business today, attacking the opposition is surely not a good thing especially when you you do check your facts first, and I do wonder what Sitemorse think they can gain from this, especially when their own site has numerous accessibility issues.
Further information about Sitemorse can be found, but perhaps the articles on <isolani /> are the most interesting:
For an indepth report on automated checkers, have a look Automated testing – How useful is it?.