Accessibility testing & remediation of web apps

Since 28 June 2025 the European Accessibility Act (EAA) requires many consumer-facing digital services in the EU (e-commerce, banking, ticketing, e-books, transport) to be accessible, and public bodies have been bound by the Web Accessibility Directive for years. Most teams still find defects late, by audit. Rule engines catch the mechanical part in CI; AI adds judgement for the rest: is this alt text meaningful, does the focus order make sense, and what is the right fix in this component. Result: earlier detection, fixes proposed as pull requests, and expert time reserved for what only humans can verify.

Problem & value

• Today: accessibility is checked by a yearly external audit or by overlay widgets. Findings arrive as PDFs, backlogs grow, developers lack the know-how to fix "missing name for button" or focus traps, and regressions appear with every release.
• What automation can and cannot do: rule engines such as axe-core reliably detect only a portion of WCAG issues; vendor and practitioner sources put this at roughly a third to a half of defects by occurrence, and only a minority of success criteria are fully machine-testable. Manual keyboard, screen-reader and user testing remain required [1][3].
• AI value: (1) triage and explain findings in developer language, (2) generate patches in the codebase, (3) assess semantic criteria that rules cannot (alt text quality, heading logic, label clarity, error messages, reading order from screenshots), (4) drive the page like a user (keyboard-only agents), (5) write regression tests.
• Legal driver: EAA (Directive (EU) 2019/882) applies from 28 June 2025; harmonised standard EN 301 549 references WCAG. Version 4.1.1 (dated 2026-09) aligns with WCAG 2.2 AA; check whether it is already cited in the Official Journal for your case, as v3.2.1 (WCAG 2.1 AA) may still be the formal reference [2][4].

Variants

• SMB web shop / agency: CI scan on every build plus an AI assistant that fixes template-level issues once for all pages.
• Enterprise product team: design-system level remediation (component library), accessibility gates in pull requests, dashboards across hundreds of apps.
• Public sector: monitoring of many sites for the Web Accessibility Directive, accessibility statements, feedback mechanism.
• Mobile apps: same logic with native accessibility trees (iOS/Android); EAA covers apps too.
• Documents (PDF/Office) on the site: see DOC-026. Video captions/audio description: see SPEECH-004.
• Private / hobby sites: free browser tooling (Lighthouse, axe extension) plus an LLM to explain findings.

How it works

1. code repo + staging URL
2. rule-based scan (axe-core / Lighthouse / Pa11y in Playwright, per page and per state)
3. agentic exploration (keyboard-only navigation, forms, dialogs, error states; DOM + a11y tree + screenshots)
4. LLM/VLM judgement on semantic criteria (alt text, labels, headings, contrast in images, reading order)
5. mapping to WCAG 2.2 / EN 301 549 clauses + severity + dedupe by component
6. fix generation (patch against the source component, using repo context)
7. re-scan the patched build
8. pull request with before/after evidence
9. human review (developer + accessibility specialist)
10. merge
11. CI regression tests
12. periodic manual audit with screen readers and users with disabilities

• Detection: deterministic rules produce the ground truth; the LLM must not "declare" conformance.
• Fix agent: coding agent (see CODE-002) with an accessibility tool via MCP, so fixes are verified by re-running the rules.
• Human-in-the-loop: developer reviews every patch; accessibility expert signs off conformance claims and the accessibility statement; AI-generated alt text is reviewed where context matters (informational images, charts).

Data preparation

Data here is mostly code and rendered pages, so preparation is light (roughly 10 to 15 percent of effort). Sources are the Git repository, the design system or component library, staging URLs, the issue tracker and existing audit PDFs. Typical gaps: no stable staging environment, test states (logged in, checkout, error dialogs) that cannot be reached automatically, no synthetic test accounts, and accessibility findings that exist only as scanned or PDF audit reports.

Preparation steps: - Define the 5 to 10 key user journeys and script them in Playwright so the scanner reaches every state, not only the home page. - Create synthetic test accounts and demo data so no real customer data appears in screenshots or DOM dumps. - Convert old audit reports into an issue backlog with WCAG criterion, component and severity; a model can do the extraction (see DOC-014 and DOC-018 for document extraction). - Document component conventions (design tokens, focus styles, naming rules) as short markdown files the agent can read; this is a small knowledge base in the sense of PREP-013. - Check which PDFs and videos on the site need separate treatment (DOC-026, SPEECH-004).

Data-readiness basics are covered in GUIDE-003.

Implementation paths

Which path for whom

• smb: Path B — the fixes must land in your own repository and templates, and an axe check in CI plus a coding agent costs little; overlay or SaaS products rarely fix source code.
• enterprise: Path A — with hundreds of apps, platforms such as Deque axe or Siteimprove provide dashboards, reporting and audit support that are not worth building yourself.
• developer: Path B — add axe-core to the existing Playwright suite and let a coding agent propose patches that it verifies by re-scanning.
• public: Path C — open-source engines (axe-core, Pa11y) and open-weight code models keep code and test accounts on your own infrastructure and avoid procurement hurdles.

Path A — Buy (off-the-shelf)

• Accessibility platforms with AI features: Deque axe DevTools (incl. Axe MCP Server for AI coding agents in the IDE), Level Access, Siteimprove, TPGi, BrowserStack Accessibility, LambdaTest, Evinced; EU-based: Siteimprove (DK) and national BITV/EAA testing specialists.
• Built-in tooling: Lighthouse and Chrome DevTools, Microsoft Accessibility Insights (open source), GitHub Copilot or Cursor with the axe MCP server.
• Caution: "overlay" widgets that promise one-line compliance do not remediate source code and are widely criticised by accessibility experts and have been challenged by regulators in the US; do not rely on them for EAA compliance.
• Choose A when: you want dashboards, reporting and audit support with little engineering effort.

Path B — Build on cloud APIs (recommended)

• Add @axe-core/playwright (or Pa11y / Lighthouse CI) to the existing Playwright/Cypress suite and fail builds on new violations.
• Send violations plus the relevant component source to a code model (Claude, GPT, Gemini, or Mistral Codestral/Devstral as EU provider) to propose patches; use a multimodal model for screenshot-based checks (contrast in images, visual focus indicators, reading order).
• Orchestrate through a coding agent (Claude Code, Copilot coding agent, Codex, Cursor) with axe MCP Server or Playwright MCP so the agent can verify its fix.
• Use EU regions or EU providers when pages contain personal data (logged-in states, test accounts).

Path C — Self-host (open source)

• Engines: axe-core (MPL-2.0), Pa11y, Lighthouse, IBM Equal Access Checker, Accessibility Insights, WAVE API alternatives; Playwright for automation.
• Models: open-weight code and vision models (e.g. Qwen3-Coder, Devstral, Qwen-VL family) via vLLM or Ollama; agent via OpenHands, Aider, Cline or Continue.
• Hardware: HW2 for ~30B-class quantised code models and a small VLM; HW3 for stronger agentic fixes.
• Good fit for public bodies or code that cannot leave the premises.

Path D — Custom training

Rarely needed. Possible for fine-tuning a small VLM on your own labelled screenshots (e.g. own design-system patterns) or learning your component-library fix conventions from past accessibility pull requests. A few hundred reviewed before/after patches are a useful start.

State of the art (2026)

• Rules + LLM hybrid is the dominant pattern: deterministic engines for precision, LLMs for explanation, fix generation and semantic checks. MCP servers expose the accessibility engine to coding agents so that the agent validates its own changes [3].
• WCAG 2.2 (W3C Recommendation, adds e.g. Focus Not Obscured, Target Size (Minimum), Dragging Movements, Accessible Authentication) is the current baseline; several of the new criteria are only partially machine-testable [5].
• Agentic keyboard and screen-reader simulation is improving, but screen-reader behaviour differs between NVDA, JAWS, VoiceOver and TalkBack; LLM simulations are not a substitute.
• Limits and failure modes: hallucinated or generic alt text; "fixes" that add ARIA where native HTML is better (the first rule of ARIA is not to use it when a native element works); patches that silence a rule without helping users; missing dynamic states (modals, toasts, SPA route changes); false conformance claims. Always re-test with assistive technology.

Shared resources & combinations

• Shared stack: llm-code + agent-framework + Playwright with CODE-004 (test generation), CODE-002 (issue-to-PR agents), CODE-003 (AI code review as an accessibility reviewer), CODE-014 (bug triage).
• Vision/design: CODE-008 and CODE-007 generate UI; add an accessibility gate so generated interfaces do not ship inaccessible by default.
• Content: DOC-026 for PDFs/Office files linked from the site; SPEECH-004 for captions of video content.
• Docs: CODE-006 for documenting component accessibility contracts.
• One GPU server (HW2/HW3) serving a code model can serve all of these.

Effort, cost & KPIs

• Path A: days to weeks; licences per user or per scanned page/domain.
• Path B: 2-6 weeks to add CI gates and a fix agent for one product; cost mainly model tokens per violation (small) and developer review time.
• Path C: 1-3 months incl. model serving and agent setup.
• KPIs: open violations by severity, new violations per release (target zero), time to fix, share of fixes accepted without rework, WCAG criteria covered by manual audit, support complaints and feedback-mechanism reports, audit findings per year.

People & change

• Involve early: management (budget, legal duty under the EAA/BFSG), the product or shop owner as process owner, and developers and designers as end-users. Bring in legal/compliance for the accessibility statement and IT for CI access. Ask people with disabilities or an accessibility association to join testing; they are the real customers of the result.
• Works council: depends. The tool analyses code and pages, not people. Co-determination under §87(1) no. 6 BetrVG becomes relevant if pull-request acceptance rates or per-developer metrics are reported; keep dashboards at component or team level. Inform the works council anyway under §90(1) no. 3 BetrVG when work processes change.
• Data protection: use synthetic test accounts; a DPIA is normally not needed, but sign an AVV (Art. 28 GDPR) with model vendors and ask the DPO if logged-in pages are scanned.
• AI literacy: support developers with a short session (Art. 4 AI Act as amended): what the agent can and cannot judge, why alt-text suggestions need review.
• Pilot and communication: start with one team and one template; present it as a way to remove tedious fixes, not to measure people. Typical concern: "the AI declares us compliant" — answer that conformance claims stay with an expert and manual testing.
• Humans in control: every patch is reviewed; no automatic merges.

See GUIDE-008, GUIDE-009 and GUIDE-014.

Risks, regulation & compliance

• EU AI Act: minimal risk; the tool is a developer aid and not an Annex III use. AI literacy (Art. 4) applies to the team. If an AI feature is shipped to end users as a chatbot, Art. 50 transparency applies separately.
• Accessibility law: EAA (Directive 2019/882) with national transposition (e.g. BFSG in Germany, applicable since 28 June 2025; microenterprises providing services are exempt); Web Accessibility Directive (2016/2102) for public bodies; EN 301 549 as harmonised standard. Market surveillance and penalties are national. Automated scans alone do not demonstrate conformance [2][4].
• GDPR: scanned staging or logged-in pages and screenshots may contain personal data; use synthetic test accounts, EU hosting or self-hosting, and a data processing agreement with model vendors.
• Security/IP: coding agents get repository access; use least-privilege tokens and review all patches. Do not paste proprietary code into consumer chat tools.
• Mitigations: deterministic verification of each fix, expert sign-off, user testing with people with disabilities, publish an accurate accessibility statement.

Getting started

1. Run Lighthouse and axe on your 5-10 most important user journeys (checkout, login, forms) and record the baseline.
2. Add @axe-core/playwright to CI so new violations fail the build; start with critical and serious only.
3. Fix template and design-system issues first (colour contrast, focus styles, form labels, landmarks) with an AI coding agent and review each patch.
4. Add a manual keyboard and screen-reader pass (NVDA/VoiceOver) per release; commission an expert audit before claiming conformance.
5. Publish an accessibility statement and a feedback channel, and track the KPIs monthly.

Sources

1. Deque — axe-core (open-source rules engine) — https://github.com/dequelabs/axe-core
2. Directive (EU) 2019/882 (European Accessibility Act) — https://eur-lex.europa.eu/eli/dir/2019/882/oj
3. Deque — Axe MCP Server — https://www.deque.com/axe/mcp-server/
4. ETSI — EN 301 549 V4.1.1 (2026-09) — https://www.etsi.org/deliver/etsi_en/301500_301599/301549/04.01.01_60/en_301549v040101p.pdf
5. W3C — Web Content Accessibility Guidelines (WCAG) 2.2 — https://www.w3.org/TR/WCAG22/
6. Deque — axe DevTools Playwright integration — https://www.deque.com/blog/new-axe-devtools-integration-playwright/
7. Microsoft — Accessibility Insights — https://accessibilityinsights.io/
8. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj