Automated vs. Manual Accessibility Testing
What automated accessibility testing catches that manual testing misses, with a side-by-side comparison and a practical workflow any team can use.
Automated vs. Manual Accessibility Testing
Automated accessibility testing is fast, cheap, and useful. It also isn't enough on its own. A scan can tell you an image has no alt text; it can't tell you whether a screen reader user can actually complete your checkout or signup form. Both automated and manual testing catch real problems, and neither catches everything the other does.
This article breaks down what each type of testing catches and misses, compares them side by side, and offers a practical testing workflow your team can use without a big budget.
Key takeaways
- Automated testing can catch many objective, rule-based issues; it can't judge whether content actually makes sense to a person using assistive technology.
- Manual testing, including keyboard and screen reader use, can reveal whether important tasks work in ways automated checks cannot evaluate.
- A passing automated scan is a useful signal, not proof of accessibility or legal compliance.
- Basic manual checks can complement routine automated testing, while more complex websites or higher-risk situations may still benefit from professional review.
- Retesting after redesigns, CMS changes, or new third-party integrations matters as much as the initial test.
What automated testing catches
Automated tools scan your website's code against a defined rule set and can quickly check a large number of pages. Depending on the tool and page structure, automated testing can flag issues such as:
- Images missing alt text entirely.
- Form fields with no associated label.
- Many color-contrast failures against solid backgrounds.
- Empty links or buttons with no accessible name.
- Missing document language or page title.
- Some heading-structure problems, like skipped heading levels.
What automated testing can't catch
Some accessibility issues require human judgment because an automated check can detect whether something exists without determining whether it works well for the person using it:
- Whether alt text actually describes the image, rather than just existing.
- Whether a form's error messages make sense and help someone fix the problem.
- Whether a custom interface element, like a dropdown menu or a multi-select filter, behaves correctly with a screen reader.
- Whether focus moves logically through a multi-step process.
- Whether the reading order of a complex layout matches how a sighted user would understand it.
- Whether captions are accurate, not just present.
Automated vs. manual testing at a glance
What you need to know | Automated testing | Manual testing |
|---|---|---|
Speed and cost | Fast and low-cost; can scan many pages at once | Slower; needs a person's time, though not necessarily a paid specialist |
Missing labels, alt text, contrast | Can detect many rule-based issues | Can reveal whether a label or alt text actually makes sense |
Keyboard operability | Can flag some missing focus indicators | Tests whether a full task can be completed by keyboard |
Screen reader experience | Cannot reproduce or judge the user's screen reader experience | Can reveal whether content and interactions work with a screen reader |
What it proves | Flags candidate issues to review | Can surface barriers a real person encounters completing a real task |
A practical automated and manual testing workflow
You can start finding meaningful accessibility issues without beginning with a full professional audit. A lightweight, ongoing process can be a practical starting point, particularly for teams with limited accessibility resources.
- Run a quick automated scan first, using a tool like Clym Scanner, to identify potential rule-based issues and get a starting list.
- Pick your highest-stakes pages, such as your checkout or signup flow, primary navigation, and any account or application forms, rather than trying to review the entire site at once.
- Do a keyboard-only pass: unplug your mouse and try to complete the same tasks a visitor would, watching for anything you can't reach or operate.
- Do a basic screen reader spot-check using a free tool, like NVDA on Windows or VoiceOver on Mac, on those same high-stakes pages.
- Prioritize issues that prevent someone from completing an important task, then work through the remaining barriers based on their impact.
- Retest after any redesign, CMS change, or new third-party platform, not just once at the start.
How this connects to your legal obligations
Testing shows where barriers exist; it doesn't, by itself, tell you which laws apply to your organization. Depending on your organization, that could include the ADA, state accessibility requirements, federal funding or contracting requirements, or contractual terms that reference WCAG.
Testing can identify technical accessibility issues against the standard you're using, but it doesn't, by itself, determine whether your organization meets its legal obligations.
Where accessibility widgets fit
Accessibility widgets serve a different purpose from testing tools. They give visitors options to adjust aspects of their experience, such as text presentation, contrast, or navigation preferences, but they don't test the underlying website or replace remediation.
Clym's Accessibility Widget complements testing and remediation rather than replacing either.
Conclusion
Automated and manual testing answer different questions. Automated testing helps identify rule-based issues at scale; manual testing can uncover barriers that require someone to interact with the website using a keyboard, screen reader, or other assistive technology. Using both gives your team a more complete picture than relying on either method alone.
Start with your highest-stakes pages, retest after changes, and build testing into your regular publishing habits rather than treating it as a once-a-year project.
Frequently asked questions
No. Automated testing can catch many objective, rule-based issues, but it can't judge whether content is actually usable, such as whether alt text is accurate or a screen reader user can complete a task. Combining it with manual testing gives a much fuller picture.
No single automated scan can confirm legal compliance. It can show that a specific set of technical checks passed, which is a useful starting point, not a guarantee about your organization's legal obligations.
Not necessarily. A keyboard-only pass and a basic screen reader spot-check with a free tool can be done in-house and can identify issues automated tools may miss. More complex websites or higher-risk situations may still benefit from a professional review.
Retest after any redesign, CMS change, or new third-party integration, such as an e-commerce or event registration tool. Many teams also find it useful to recheck periodically between major changes, since routine content updates can introduce new issues too.
Start with the pages tied to your core outcomes: checkout or signup flows, account or application pages, and any multi-step forms. These tend to combine the custom controls and multi-step flows that automated testing is least equipped to evaluate.
Adam is the Head of Digital Marketing at Clym, where he leverages his diverse expertise in marketing to support businesses with their compliance needs and drive awareness about data privacy and web accessibility. As one of the company’s original team members, Adam has been instrumental in shaping its journey from the very beginning. When he’s not diving into marketing strategies, Adam can be found cheering on his favorite sports teams or enjoying fishing.
