Why an Automated Accessibility Score Isn't Enough

Imagine you run an automated accessibility scan on your homepage. It comes back clean: no missing alt text, no contrast failures, no unlabeled forms. Green across the board. Most vendors would hand you that report and call the job done.

We don't. Our own score page says this out loud, right under the results: an automated scan is not a complete WCAG audit or a conformance determination, because full WCAG 2.2 AA conformance needs human review and testing with assistive technology. We put that disclaimer there before we wrote this article, because it is true, and we would rather be honest than impressive.

This post is the longer version of that disclaimer. It covers what automated testing (axe-core, Lighthouse, WAVE, the good ones) genuinely catches, what it physically cannot judge, and what manual testing actually looks like. If you have never tested a site with only a keyboard and a screen reader, this will be a useful hour of your week.

Automated scanners are rule engines. They run the rendered page through hundreds of checks: is there an alt attribute on every image, does this text meet the 4.5 to 1 contrast ratio, does every form input have a label, are ARIA roles valid, are there duplicate IDs, is the page language declared. They are fast, consistent, and brutally honest about mechanical failures. Running them on every build, in your CI pipeline, is one of the best investments a team can make.

And they catch a meaningful share of problems. Deque's own 2023 study found that axe-core identifies about 57% of digital accessibility issues by volume. Depending on methodology, other analyses land lower, closer to a third. Either way, the honest reading is: automation gets you somewhere between a third and a bit over half of the issues. The rest belongs to people.

One more stat that frames it: in Deque's data, seven WCAG criteria account for over 80% of all issues found, with color contrast alone around 30%. So automated tools are excellent at the most common failures, the ones that show up everywhere. That is exactly why you should run them. It is also why you cannot stop there. The most common issues are not the same as the hardest ones.

Here is the core limit of every automated tool, and it will not change no matter how good the AI gets at pattern matching: a scanner cannot judge meaning, context, or experience.

An alt attribute that says "image" passes every automated check and tells a blind user absolutely nothing. A scanner can verify the attribute exists; it cannot tell you the description is useless. That is a judgment call a person has to make, ideally a person who understands what the image is actually for on that page.

The same gap runs through everything that matters:

• Reading order. A scanner reads the DOM. A person reads the page visually, and a screen reader user hears content in DOM order. If those two orders disagree, the experience is confusing, and no rule engine flags it.
• Focus order. Whether the tab order through a page follows a logical sequence (WCAG 2.4.3) is something only a human tabbing through can assess.
• Error messages. A scanner can check that an error is programmatically associated with a field. It cannot tell you whether the message ("Error 1042") helps anyone recover.
• Whether a custom widget behaves. Tabs, carousels, date pickers, multi-select dropdowns: custom components are where accessibility goes to die, and automated tools see them as ordinary divs unless the ARIA is catastrophically broken.

A huge share of disabled users navigate without a mouse: people with motor impairments, people using switch devices or voice control, power users, and plenty of temporarily injured people. Keyboard support is not a bonus feature. It is a core requirement (WCAG 2.1.1), and testing it takes about ten minutes per page. Here is the routine:

• Tab through everything. Every link, button, form field, and custom control must be reachable with the Tab key. If you can't get to it with a keyboard, it doesn't exist for a keyboard user.
• Check the focus indicator. At every stop, you must be able to see where you are (WCAG 2.4.7, and the newer 2.4.11 focus appearance criteria). Many designers hide the focus outline because it looks ugly. That outline is the only way some users know where they are.
• Try to get trapped. Open a modal, a date picker, a dropdown menu. Tab through it. Can you get out? Keyboard traps (WCAG 2.1.2), where focus enters a component and cannot leave, are one of the most common catastrophic failures in the wild, and automated tools almost never catch them.
• Use the skip link. Every page should have a "skip to main content" link so keyboard users don't have to tab through the entire navigation on every page load. Test that it works and that it lands where it says.
• Close things with Escape. Modals and dropdowns should close with Escape, and focus should return to the element that opened them. This focus management is invisible to scanners and essential to people.

Screen readers are how millions of blind and low-vision users browse the web. The three you need to know: NVDA (free, Windows), JAWS (paid, Windows, the enterprise standard), and VoiceOver (built in to every Mac, iPhone, and iPad). You do not need to become an expert. You need to do a walkthrough and listen for what is wrong. What you are listening for:

• Meaningful announcements. Every button and link should announce a name that makes sense out of context. "Button" is not a name. "Submit application, button" is. (WCAG 4.1.2.)
• Headings as navigation. Screen reader users navigate by headings, jumping from section to section. A page with no headings, or with heading levels that skip from H1 to H4, is a page with no map.
• Form labels and errors. Labels must be announced with their fields, and error messages must be read when they appear, linked to the field they describe. A form that turns a field red without saying why has told the screen reader user nothing.
• Dynamic content. Toasts, loading states, cart updates, form submissions: anything that changes without a page reload must be announced through ARIA live regions or status messages (WCAG 4.1.3). This is one of the most frequently broken things on modern single-page apps, because the visual update works and nobody checked whether it was announced.
• Images and video. Informative images need descriptions that actually describe; decorative images should be hidden from the screen reader entirely. Video needs more than a title: it needs the audio content described.

Accessibility is not only about blindness and motor impairments. For deaf and hard-of-hearing users, and for anyone in a noisy room or a quiet office, captions and transcripts are the access path to your audio and video (WCAG 1.2.2 for captions, 1.2.4 for live captions, 1.2.3 for audio description or a media alternative). Three honest notes on this:

• Auto-generated captions are a starting point, not a deliverable. YouTube and most platforms generate them for free, and they are wrong often enough to matter: names, terminology, and anything said over background noise. Review and correct them, or pay for human captioning on anything important.
• Captions must match the audio. A scanner can check that a caption track exists. It cannot check that the captions say what was actually said, or that they are synchronized. Only a person watching the video can.
• Transcripts help more people than you think. A full text transcript of audio or video content serves deaf-blind users, people who process text better than audio, and anyone searching your content. It is also excellent for SEO, which is a nice bonus for doing the right thing.

This section is the reason this post exists, and we are going to be direct.

There is a class of products, usually called accessibility overlays or widgets, that promises one line of JavaScript will make your site compliant. The disability community has been warning about these for years, the litigation data backs them up, and in 2025 the regulators caught up too. The facts, briefly:

• The FTC acted. In January 2025, the Federal Trade Commission announced that accessiBe, one of the largest overlay vendors, would pay $1 million to settle allegations that it misrepresented its AI-powered widget's ability to make any website WCAG-compliant, and that it had disguised paid endorsements as independent reviews. The order was finalized in April 2025.
• Overlays do not protect you from lawsuits. Per UsableNet's 2025 midyear litigation data, about 22.6% of web accessibility lawsuits in the first half of 2025, roughly 456 cases, targeted websites that had an overlay installed. In 2024, a quarter of the year's cases explicitly named an overlay or widget as a barrier. Sites with widgets get sued at roughly the same rate as sites without them.
• They don't fix the underlying code. An overlay adds a layer on top of your site. It does not repair your HTML, your forms, or your scripts. As disability rights attorney Lainey Feingold put it: one line of code does not make a website accessible to disabled people.

This is where we state plainly how Thanos Labs operates, because we think every vendor in this space should:

• Our score pages are automated scans. They catch the mechanical failures fast and in plain language. They are labeled, on the page, as automated scans, never as conformance determinations. If anyone tells you a single automated scan certifies WCAG conformance, that is a claim we will not make about our own product, and you should not accept it from anyone else's.
• Human review is part of the service. Full WCAG 2.2 AA conformance needs human review and testing with assistive technology. That is not a footnote for us; it is the second half of the work, and it is priced and staffed that way.
• A human approves changes before they reach your visitors. Our accessibility AI agent detects barriers and manages remediation, but nothing it touches goes live on a client's site without a human's approval. Automation proposes; people decide.

Pin this next to your desk. Run the automated scan first to clear the mechanical failures, then do this. Do that once per release on the pages that changed, and you will be ahead of the vast majority of the web. The automated score gets you to the starting line. The manual pass gets you to the finish.

• Tab through the entire page. Everything interactive is reachable, in a logical order.
• The focus indicator is visible at every stop.
• No keyboard traps. Modals, menus, and pickers open, operate, and close with the keyboard. Escape closes; focus returns to the trigger.
• The skip link works and lands on the main content.
• Walk the page with a screen reader (NVDA, JAWS, or VoiceOver). Buttons and links announce meaningful names. Headings form a usable map. Form labels and errors are announced. Dynamic updates are announced through live regions.
• All images have useful alt text or are hidden as decorative.
• Videos have accurate, synchronized captions. Audio and video have transcripts.
• Zoom to 200%. Nothing is cut off or overlapping.