How Slack Built Accessibility Checks into Its Testing Pipeline

Accessibility isn’t something Slack treats as an afterthought, it’s built into how they ship products. But testing for it isn’t easy. It’s slow, mostly manual, and tough to scale as the product grows.

So Slack asked a practical question: Can we automate accessibility checks without making developers’ lives harder?

Let’s discover how they tried to do exactly that, what worked, what didn’t, and what they learned along the way in this blog today.

The Problem They Were Solving

Slack already takes accessibility seriously, it’s not an afterthought.

They have:

Their own accessibility standards aligned with WCAG
A dedicated accessibility team supporting developers
Regular manual testing with assistive technologies
Feedback from real users, including people with disabilities

They even involve users early in design and prototype stages, which is something many teams skip. So this wasn’t a case of “they weren’t doing enough.”

The real issue was scale.

Most of this work especially testing was still manual. And while manual testing is important (and irreplaceable in many cases), it doesn’t scale well as the product grows and changes quickly.

They started running into a few practical challenges:

Getting faster feedback during development
Reducing the dependency on manual testing for every change
Making accessibility checks easier for developers to adopt

At the same time, they knew automation isn’t perfect. Automated tools can:

Miss nuanced issues (like how a screen reader actually feels to use)
Flag things that aren’t real problems in context

So Slack wasn’t trying to replace human testing. Instead, they saw an opportunity: Use automation as an extra layer of support

The goal was to make accessibility checks easier to run, easier to catch early, and easier for developers to integrate into their existing workflows without adding more friction.

Step 1: Choosing the Right Tool

To automate accessibility checks, Slack chose Axe, a well-known tool that checks apps against WCAG guidelines. It made sense because:

It covers a wide range of accessibility rules
It’s flexible and configurable
It keeps false positives relatively low
It works well with modern testing frameworks

The idea was simple (and ambitious): Run accessibility checks automatically inside existing tests without developers having to think about it.

Attempt 1: React Testing Library + Jest

They first tried adding Axe directly into their component tests using React Testing Library.

The plan:

Wrap the render function
Run accessibility checks automatically every time a component renders

Sounds neat, but it didn’t work out. Problems they hit:

Slack’s custom Jest setup made integration messy
It required developers to write separate accessibility-specific tests
Increased complexity and friction in the workflow

In short: it wasn’t simple enough to scale. So they dropped this approach.

Attempt 2: Deep Integration with Playwright

Next, they moved to Playwright, their end-to-end (E2E) testing framework.

This time, the idea was even more seamless: Automatically run accessibility checks after every user action (clicks, navigation, etc.)

But this ran into a more subtle, technical issue.

The core problem:

Playwright uses something called a Locator, which:

Waits for elements to be ready
Ensures interactions are stable

But here’s the catch: It only guarantees that one element is ready, not the entire page.

Why that’s a problem

Accessibility checks need the full UI to be completely rendered.

If you run them too early:

Some elements might not be loaded yet
Real issues can be missed
Results become unreliable

They also explored using older methods (like manually waiting), but:

These slowed down tests
Caused duplicate checks
Went against how Playwright is designed

So fully “baking” accessibility into every interaction wasn’t reliable.

Why that’s a problem

Accessibility checks need the full UI to be completely rendered.

If you run them too early:

Some elements might not be loaded yet
Real issues can be missed
Results become unreliable

They also explored using older methods (like manually waiting), but:

These slowed down tests
Caused duplicate checks
Went against how Playwright is designed

So fully “baking” accessibility into every interaction wasn’t reliable.

What They Actually Did (The Practical Approach)

Instead of forcing full automation, Slack took a step back and simplified things.

Their approach:

Don’t hide everything behind magic
Run checks at the right time
Keep it easy for developers

Final Flow

Here’s what they ended up with:

A test loads a page or completes a flow
Axe runs an accessibility scan
Known issues are ignored (to reduce noise)
Only important (critical) issues are flagged
Results are saved for review

Trying to fully automate accessibility checks inside frameworks sounds great—but timing and reliability matter more.

Slack realized: It’s better to run checks at the right moment than to run them everywhere blindly.

That small shift made the system far more practical and actually usable.

Making It Work in Practice (Customizations & Workarounds)

After hitting multiple roadblocks trying to fully automate accessibility checks, Slack shifted to a more practical approach.

Instead of forcing everything into the framework, they focused on:

👉 Keeping things simple for developers 👉 Running checks at the right time 👉 Reducing noise in results

They continued using Playwright + Axe, but added smart customizations to make the system actually usable at scale.

Key Improvements They Made

Focus Only on What Matters

Raw accessibility reports can be noisy. So Slack filtered results to:

Only show critical severity issues (based on WCAG)
Remove duplicate violations within a test

This way, developers only see issues that actually need attention.

Ignore Known & Irrelevant Issues

Not every violation is worth flagging every time. They created exclusion lists for:

Known bugs (already tracked)
Rules that don’t apply to Slack’s design

This avoids repeated, noisy alerts and keeps reports clean.

Make It Dead Simple for Developers

Instead of complex setup, they exposed a simple helper:

await slack.utils.a11y.runAxeAndSaveViolations();

One line that will run accessibility checks. They integrated this into their existing Playwright fixture system, so developers could use it without extra setup.

Better Debugging & Reports

Catching issues isn’t enough, developers need to fix them quickly. So Slack improved reporting by adding:

Screenshots of the failing UI
Structured HTML reports (via Playwright reporter)
Clean, readable error messages

Each issue clearly shows what failed, why it failed and where it failed. Thus leading to faster debugging and less confusion.

Where They Run Accessibility Checks

This part turned out to be critical. They didn’t run checks everywhere blindly. Instead, they placed them:

After navigation (new page/view loads)
After UI changes (clicks, flows)
Only when the full page is rendered

And avoided:

Running checks too early
Running checks multiple times on the same UI

This required a solid understanding of app flows not just tools.

How Developers Actually Use It

To keep things flexible, Slack added a feature flag:

A11Y_ENABLE=true

This allows:

On-demand testing → run locally when needed
Scheduled runs → daily automated checks
CI integration → optional checks before merging

Developers aren’t forced but it’s easy to opt in.

Reporting & Issue Tracking

Slack also made sure issues don’t get lost. When violations are found:

Alerts are sent internally
Jira tickets can be auto-created
Issues are tagged as “automated accessibility”

Thus giving them clear ownership and easier tracking.

Auditing the System

They also built internal tools to:

Detect duplicate accessibility checks
Ensure good coverage across flows
Capture screenshots for verification
This process is still partly manual but improving.

What They’re Exploring Next

Slack isn’t stopping here. They’re looking into:

Using AI to suggest where checks should run
AI to analyze accessibility reports
Smaller blocking test suites for critical flows (like keyboard navigation)

Moving toward smarter, not just more, automation.

Key Takeaways

Automation helps but can’t replace human testing
Perfect automation is hard (and sometimes unnecessary)
Timing and placement matter more than full automation
Developer experience is just as important as the tool itself
Start simple → then scale

Slack didn’t chase perfect automation. They focused on something more important: Making accessibility checks simple, reliable, and actually usable. Because in the end, accessibility doesn’t improve just because tools exist, It improves when developers actually use them.

Official blog from Slack: Automated Accessibility Testing at Slack

By now, you must have had a clear idea of, How Slack Built Accessibility Checks into Its Testing Pipeline? In a nutshell, Slack added automated accessibility testing using Axe + Playwright, balancing automation with reliability and developer ease. Instead of full automation, they focused on smart integration, better reporting, and scalable workflows.

Congratulations! You've just advanced another step in your tech journey. Keep progressing!