Accessibility is an increasingly important aspect of web development. Unfortunately, it’s also a complex aspect of design and development and if you don’t know anything about the standards, it can be daunting to begin working towards a more accessible platform / site. Thankfully, there are tools out there like Pa11y that can really help!
Why Automate Accessibility Testing?
This is a question that’s worth asking. There are a number of answers.
You want the best possible experience on your site / platform for all users.
You are legally / contractually obligated to do so.
You don’t want to get sued.
Obviously given your organization and your own personal perspective, some of these answers may be better than others. But these are the real answers I’ve heard from organizations about why they are doing accessibility testing. Is “because we don’t want to get sued” as altruistic as “we want the best possible experience for all of our users?” No, definitely not. Hopefully the end result though is still a highly accessible website.
What is Pa11y?
Pa11y is a command-line interface which loads web pages and highlights any accessibility issues it finds. Useful for when you want to run a one-off test against a web page ~ https://github.com/pa11y/pa11y.
Much like other test runners, Pa11y executes a set of test cases (based on Accessibility standards like WCAG 2.0 AA) against a set of pages on an application. It scans the markup of that page and reports if the markup passes the test cases or not.
There’s good news and bad news about this.
The good news is that this is awesome! You can catch so many different issues just by fixing the markup on the page. And you should fix the markup on the page.
The bad news is that this is just a drop in a much larger bucket. To truly, legitimately make a website accessible you can’t just fix the markup. You ALSO have to test with keyboards and screen readers. You ALSO have to make intelligent decisions in how you design and build your site to make them user friendly to folks using some of these specialized technologies.
So no, Pa11y isn’t going to do all THAT for you. But again, it does help with the markup which is a massive first step.
Why Do you Need Pa11y?
I recently blogged about the impacts that YOU can have on the accessibility of a Drupal site. Drupal itself is quite accessible out of the box (especially if you are working in Drupal 8 or Drupal 9). Other CMS are in a similar boat! The problem is, as you design your site, build out the front end, and create content, those activities can all have major impact on the accessibility of the site. Every table your users hard code into the body of your content, every image you upload, every video you embed, can all have drastic impact on accessibility.
Enter Pa11y (or other tools). The goal being to test all of the features and functions of your site to ensure that as your site grows and matures the accessibility doesn’t degrade. My goal is always to make sure that we test any permutation of a given feature / component / page so that I know for sure that it is accessible. This isn’t the same as testing all of the pages on a site.
What does Pa11y Need to Run?
To use Pa11y, you need a few things:
Markup
Content
Configured CMS
Webserver / Database Server
Container
The basic take away is that you need a site that has real theme work done with real (or real-looking) content. The best way to accomplish this is by using a module like Default Content to export some real content. Here are a few other recommendations!
As much as possible, rely on Drupal to control the markup of your content / media / blocks / etc. The more your content creators can “change the HTML” via your WYSIWYG editor, the higher the probability that your accessibility scores will vary from page to page. What does this mean in practice? If you want to embed a video, don’t allow content creators to actually “embed it.” Use the Media bundle for Video, use an entity reference to embed the video, etc. This way all of the markup for Videos is consistent throughout the site and if there’s a problem, you can fix it once and have it apply everywhere.
Follow “component based design” principles for your site. This means that if you have a <thing> you need to build, try to build it consistently! Again, if you have a hero banner or facet block, the goal is to have a consistent application of this feature so that you can make it accessible one time, and then every use of it on the site will be consistent.
Use Default Content to create representative content. You don’t need “all” of your content tested if you are consistent in how you apply markup. For instance, if a hero banner “always” renders the same way (no matter who creates it or where it’s placed) then you really only need to test “one” hero banner (because it’s representative of “all” hero banners). On the other hand, if you let content authors change the markup or vary the banner wildly, you lose this benefit (and need to test many more pieces of content to ensure accessibility)
Getting Started
Pa11y is actually pretty easy to setup and run, assuming that you already have NPM setup on your project and you are using a continuous integration process that takes advantage of dependency managers (like NPM and Composer). I have some related material on this if you don’t.
Assuming you DO have these pieces in place, you just need to add Pa11y to your NPM package file! You can see an example of me doing this on the GovCon site here.
The TLDR:
run npm install pa11y-ci
configure Pa11y for the accessibility standard you are trying to meet (e.g. WCAG 2.0AA)
decide which pages you want to test, manually test them with Lighthouse in Chrome (or some other tool). fix any accessibility issues you find for each page.
export the content for this page / these pages using default content
configure Pa11y to run on these pages
include Pa11y in your build script
In Conclusion
I recently resented on this topic at Drupal GovCon 2020. If you are interested in a full presentation (and conversation) then check out the recording!
Accessibility testing is an increasingly important component of web development and if you aren’t doing it on “some” level you are behind the times. It’s definitely something you should put on your 2021 roadmap if it isn’t there already!
Related Content
Sometimes when testing an application your cross contaminate test content with real data (or user contact information). This is bad! Let’s talk about how to avoid this type of situation.