Skip to content

Run axe accessibility scans in e2e tests - #1411

Merged
microbit-matt-hillsdon merged 2 commits into
mainfrom
axe-e2e
Nov 12, 2026
Merged

microbit-matt-hillsdon merged 2 commits into
mainfrom
axe-e2e

Conversation

@microbit-matt-hillsdon

@microbit-matt-hillsdon microbit-matt-hillsdon commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Adds axe scans to the Playwright suite. Scenarios call app.expectNoNewAccessibilityViolations() on the states they reach, and a new block in accessibility.test.ts covers the menus and dialogs that the page object's helpers click straight through. 22 scans in total.

Existing violations are not fixed here. They are recorded in knownViolations in src/e2e/axe.ts as a rule id, a CSS selector and a reason, and a scan fails only on violations that aren't listed. Fixing one means deleting its entry.

Each entry also records its WCAG level, taken from axe's tags for the rule, to help prioritise fixes. Currently known:

Level A

  • 4.1.2: CodeMirror content has no accessible name (main editor and documentation code examples)
  • 4.1.2: Simulator range sensor threshold marks use aria-label on a div with no role
  • 1.1.1: Sidebar header logos and simulator sensor icons are role="img" with no label

Level AA

  • 1.4.3: Editor line numbers are 2.02:1 contrast
  • 1.4.3: Unselected sidebar tab labels are 4.3:1 contrast

axe best practice (not required by WCAG)

  • API documentation entries skip to h4
  • No level-one heading
  • Resize handles and the simulator's sensor controls are outside landmarks

Theme

The known list is recorded against the Foundation's private theme, which is what CI and the deployed editor use. Without the theme package installed (forks, or a checkout without it) the color-contrast rule is skipped, as the open source default colours fail in different places. All other rules still run. This is noted in src/e2e/axe.ts and AGENTS.md.

Notes for review

  • The only non-test change is data-testid="simulator" on the simulator container, so one known-violation entry can cover its sensor controls.
  • The simulator iframe is excluded from scans. It is a separate deployment, and its h1 otherwise hides our missing one.
  • The role-img-alt and heading-order entries are broad (div[role="img"], h4), so they would also hide a new instance of the same mistake.
  • Nothing detects a known entry that no longer matches anything.
  • The scans add about 13s to the Playwright step in CI (3m36s here against 3m23s on main).

Scenarios scan the states they reach and fail on any violation not in the
known list in src/e2e/axe.ts. Existing violations are recorded there rather
than fixed here.

The known list is recorded against the Foundation theme. Without the theme
package the colour contrast rule is skipped, as the open source default
colours fail in different places.
@github-actions

Copy link
Copy Markdown

Preview build will be at
https://review-python.microbit.org/axe-e2e/

@microbit-matt-hillsdon

Copy link
Copy Markdown
Collaborator Author

The simulator iframe is excluded from scans. It is a separate deployment, and its h1 otherwise hides our missing one.

But almost certainly worth adding its own scan now we have e2e there.

Separates conformance failures from axe's best-practice rules, to help
prioritise fixes.
@microbit-matt-hillsdon

Copy link
Copy Markdown
Collaborator Author

CC @microbit-grace as I'm going to do this everywhere in fairly short order.

@microbit-robert microbit-robert left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@microbit-matt-hillsdon

Copy link
Copy Markdown
Collaborator Author

Just holding off merging until I have an ml-trainer version of the same in case I hit issues that suggest a change of approach or it seems shared code is worthwhile.

@microbit-matt-hillsdon
microbit-matt-hillsdon merged commit ae796ad into main Nov 12, 2026
3 checks passed
@microbit-matt-hillsdon
microbit-matt-hillsdon deleted the axe-e2e branch November 12, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants