Content
# Playwright Accessibility Testing Project Setup Guide
This guide provides complete instructions to set up a comprehensive accessibility testing project using Playwright and axe-core. The project includes automated scanning, report generation, and test cases for WCAG compliance.
## Project Overview
This project enables:
- Automated accessibility scanning of websites using axe-core
- Comprehensive WCAG compliance reporting (2.0, 2.1, 2.2)
- Detailed markdown reports with issues, severity, recommendations, and action items
- Playwright test cases for regression prevention
- Integration with CI/CD pipelines
## Prerequisites
- Node.js 18+ installed
- npm or yarn package manager
- Basic knowledge of TypeScript and Playwright
## Step 1: Initialize the Project
```bash
mkdir a11y-demo
cd a11y-demo
npm init -y
```
## Step 2: Install Dependencies
```bash
npm install --save-dev @playwright/test @axe-core/playwright typescript @types/node tsx
npx playwright install chromium
```
## Step 3: Create Project Structure
Create the following directory structure:
```
a11y-demo/
├── audits/ # Generated audit reports
├── scripts/ # Utility scripts
├── tests/ # Playwright test files
├── package.json
├── playwright.config.ts
└── tsconfig.json
```
## Step 4: Configure TypeScript
Create `tsconfig.json`:
```json
{
"compilerOptions": {
"target": "ES2022",
"module": "commonjs",
"lib": ["ES2022"],
"moduleResolution": "node",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"outDir": "./dist"
},
"include": ["tests/**/*", "playwright.config.ts", "scripts/**/*"]
}
```
## Step 5: Configure Playwright
Create `playwright.config.ts`:
```typescript
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: 'html',
use: {
trace: 'on-first-retry',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
],
});
```
## Step 6: Update package.json Scripts
Update the `scripts` section in `package.json`:
```json
{
"scripts": {
"test": "playwright test",
"test:ui": "playwright test --ui",
"test:debug": "playwright test --debug",
"test:headed": "playwright test --headed",
"scan:a11y": "tsx scripts/run-accessibility-scan.ts",
"generate:report": "tsx scripts/generate-audit-report.ts"
}
}
```
## Step 7: Create Accessibility Scan Script
Create `scripts/run-accessibility-scan.ts`:
```typescript
import { chromium } from 'playwright';
import AxeBuilder from '@axe-core/playwright';
import * as fs from 'fs';
import * as path from 'path';
async function runAccessibilityScan() {
const browser = await chromium.launch();
const context = await browser.newContext();
const page = await context.newPage();
try {
console.log('Navigating to https://sv.siman.com/...');
await page.goto('https://sv.siman.com/', { waitUntil: 'networkidle' });
// Wait for dynamic content to load
await page.waitForTimeout(5000);
console.log('Running accessibility scan...');
const results = await new AxeBuilder({ page })
.withTags([
'wcag2a', 'wcag2aa', 'wcag2aaa',
'wcag21a', 'wcag21aa', 'wcag21aaa',
'wcag22a', 'wcag22aa', 'wcag22aaa',
'section508',
'cat.aria', 'cat.color', 'cat.forms', 'cat.keyboard',
'cat.language', 'cat.name-role-value', 'cat.parsing',
'cat.semantics', 'cat.sensory-and-visual-cues',
'cat.structure', 'cat.tables', 'cat.text-alternatives',
'cat.time-and-media'
])
.analyze();
// Ensure audits directory exists
const auditsDir = path.join(__dirname, '..', 'audits');
if (!fs.existsSync(auditsDir)) {
fs.mkdirSync(auditsDir, { recursive: true });
}
// Save raw results as JSON
const timestamp = new Date().toISOString().split('T')[0];
const jsonPath = path.join(auditsDir, `sv-siman-audit-${timestamp}.json`);
fs.writeFileSync(jsonPath, JSON.stringify(results, null, 2));
console.log(`Raw results saved to ${jsonPath}`);
return results;
} finally {
await browser.close();
}
}
runAccessibilityScan()
.then((results) => {
console.log(`\nScan complete! Found ${results.violations.length} violations.`);
process.exit(0);
})
.catch((error) => {
console.error('Error running scan:', error);
process.exit(1);
});
```
**Note:** Update the URL in the script to target your website.
## Step 8: Create Report Generation Script
Create `scripts/generate-audit-report.ts`:
```typescript
import * as fs from 'fs';
import * as path from 'path';
interface AxeResult {
violations: Violation[];
passes: any[];
inapplicable: any[];
incomplete: any[];
timestamp: string;
url: string;
testEngine?: { version: string };
}
interface Violation {
id: string;
impact: 'critical' | 'serious' | 'moderate' | 'minor';
tags: string[];
description: string;
help: string;
helpUrl: string;
nodes: Node[];
}
interface Node {
html: string;
target: string[];
any?: any[];
all?: any[];
none?: any[];
}
function getWCAGLevel(tags: string[]): { version: string; level: string }[] {
const levels: { version: string; level: string }[] = [];
tags.forEach(tag => {
if (tag.startsWith('wcag2')) {
const match = tag.match(/wcag(\d+)([a-z]+)/);
if (match) {
levels.push({ version: `WCAG ${match[1]}.0`, level: match[2].toUpperCase() });
}
} else if (tag.startsWith('wcag')) {
const match = tag.match(/wcag(\d+)([a-z]+)/);
if (match) {
levels.push({ version: `WCAG ${match[1]}.0`, level: match[2].toUpperCase() });
}
}
});
return levels.length > 0 ? levels : [{ version: 'WCAG 2.1', level: 'A' }];
}
function getSeverityPriority(impact: string): number {
const priorities: Record<string, number> = {
'critical': 1,
'serious': 2,
'moderate': 3,
'minor': 4
};
return priorities[impact] || 5;
}
function calculateCompliance(violations: Violation[]): Record<string, { total: number; passed: number; percentage: number }> {
const compliance: Record<string, { total: number; passed: number; percentage: number }> = {};
const wcagLevels = ['wcag2a', 'wcag2aa', 'wcag2aaa', 'wcag21a', 'wcag21aa', 'wcag21aaa', 'wcag22a', 'wcag22aa', 'wcag22aaa'];
wcagLevels.forEach(level => {
const violationsForLevel = violations.filter(v => v.tags.includes(level));
const total = 100;
const passed = Math.max(0, total - violationsForLevel.length);
compliance[level] = {
total,
passed,
percentage: Math.round((passed / total) * 100)
};
});
return compliance;
}
function generateRecommendations(violation: Violation): string {
const recommendations: Record<string, string> = {
'aria-command-name': 'Add an accessible name to ARIA commands using aria-label, aria-labelledby, or visible text content.',
'color-contrast': 'Ensure text has sufficient color contrast ratio (4.5:1 for normal text, 3:1 for large text).',
'image-alt': 'Add descriptive alt text to all images that convey meaning. Use empty alt="" for decorative images.',
'link-name': 'Ensure all links have descriptive text that makes sense out of context.',
'button-name': 'Ensure all buttons have accessible names via text content, aria-label, or aria-labelledby.',
'heading-order': 'Use heading elements (h1-h6) in sequential order without skipping levels.',
'landmark-one-main': 'Ensure the page has one main landmark or use aria-label to distinguish multiple main regions.',
'page-has-heading-one': 'Ensure the page has a level 1 heading that describes the main content.',
'region': 'Ensure all page content is contained within landmarks (main, nav, aside, etc.).',
'html-has-lang': 'Specify a valid language attribute on the html element.'
};
return recommendations[violation.id] || `Review the ${violation.id} rule and follow the guidance at ${violation.helpUrl}`;
}
function generateActionItems(violations: Violation[]): string[] {
const actionItems: string[] = [];
const groupedViolations = new Map<string, Violation[]>();
violations.forEach(v => {
if (!groupedViolations.has(v.id)) {
groupedViolations.set(v.id, []);
}
groupedViolations.get(v.id)!.push(v);
});
groupedViolations.forEach((violations, id) => {
const violation = violations[0];
const count = violations.length;
const effort = violation.impact === 'critical' ? 'High' :
violation.impact === 'serious' ? 'Medium' : 'Low';
actionItems.push(`[${violation.impact.toUpperCase()}] Fix ${violation.id}: ${violation.help} (${count} instance${count > 1 ? 's' : ''}) - Effort: ${effort}`);
});
return actionItems.sort((a, b) => {
const priorityA = getSeverityPriority(violations.find(v => a.includes(v.id))?.impact || 'minor');
const priorityB = getSeverityPriority(violations.find(v => b.includes(v.id))?.impact || 'minor');
return priorityA - priorityB;
});
}
function generateTestCases(violations: Violation[]): string {
const testCases: string[] = [];
const uniqueViolations = new Map<string, Violation>();
violations.forEach(v => {
if (!uniqueViolations.has(v.id)) {
uniqueViolations.set(v.id, v);
}
});
uniqueViolations.forEach((violation, id) => {
const wcagLevels = getWCAGLevel(violation.tags);
const levelStr = wcagLevels.map(l => `${l.version} ${l.level}`).join(', ');
testCases.push(`test('should not have ${violation.id} violations (${levelStr})', async ({ page }) => {`);
testCases.push(` await page.goto('https://sv.siman.com/');`);
testCases.push(` await page.waitForLoadState('networkidle');`);
testCases.push(` `);
testCases.push(` const accessibilityScanResults = await new AxeBuilder({ page })`);
testCases.push(` .withTags(['${violation.tags.filter(t => t.startsWith('wcag')).join("', '")}'])`);
testCases.push(` .analyze();`);
testCases.push(` `);
testCases.push(` const violations = accessibilityScanResults.violations.filter(`);
testCases.push(` v => v.id === '${violation.id}'`);
testCases.push(` );`);
testCases.push(` `);
testCases.push(` expect(violations).toHaveLength(0);`);
testCases.push(`});`);
testCases.push('');
});
return testCases.join('\n');
}
function generateReport(data: AxeResult): string {
const violations = data.violations;
const compliance = calculateCompliance(violations);
const actionItems = generateActionItems(violations);
const testCases = generateTestCases(violations);
const timestamp = new Date(data.timestamp).toLocaleString();
const dateStr = new Date(data.timestamp).toISOString().split('T')[0];
let report = `# Accessibility Audit Report - ${new URL(data.url).hostname}\n\n`;
report += `**Audit Date:** ${timestamp} \n`;
report += `**URL:** ${data.url} \n`;
report += `**Tool:** axe-core ${data.testEngine?.version || 'unknown'}\n\n`;
report += `---\n\n`;
// Executive Summary
report += `## Executive Summary\n\n`;
report += `This accessibility audit identified **${violations.length} violation${violations.length !== 1 ? 's' : ''}** across the homepage.\n\n`;
const criticalCount = violations.filter(v => v.impact === 'critical').length;
const seriousCount = violations.filter(v => v.impact === 'serious').length;
const moderateCount = violations.filter(v => v.impact === 'moderate').length;
const minorCount = violations.filter(v => v.impact === 'minor').length;
report += `### Severity Breakdown\n`;
report += `- **Critical:** ${criticalCount}\n`;
report += `- **Serious:** ${seriousCount}\n`;
report += `- **Moderate:** ${moderateCount}\n`;
report += `- **Minor:** ${minorCount}\n\n`;
// Compliance Levels
report += `## Compliance Levels\n\n`;
report += `### WCAG 2.0 Compliance\n`;
report += `- **Level A:** ${compliance.wcag2a?.percentage || 0}% compliant\n`;
report += `- **Level AA:** ${compliance.wcag2aa?.percentage || 0}% compliant\n`;
report += `- **Level AAA:** ${compliance.wcag2aaa?.percentage || 0}% compliant\n\n`;
report += `### WCAG 2.1 Compliance\n`;
report += `- **Level A:** ${compliance.wcag21a?.percentage || 0}% compliant\n`;
report += `- **Level AA:** ${compliance.wcag21aa?.percentage || 0}% compliant\n`;
report += `- **Level AAA:** ${compliance.wcag21aaa?.percentage || 0}% compliant\n\n`;
report += `### WCAG 2.2 Compliance\n`;
report += `- **Level A:** ${compliance.wcag22a?.percentage || 0}% compliant\n`;
report += `- **Level AA:** ${compliance.wcag22aa?.percentage || 0}% compliant\n`;
report += `- **Level AAA:** ${compliance.wcag22aaa?.percentage || 0}% compliant\n\n`;
// Issues Found
report += `## Issues Found\n\n`;
violations.sort((a, b) => getSeverityPriority(a.impact) - getSeverityPriority(b.impact));
violations.forEach((violation, index) => {
const wcagLevels = getWCAGLevel(violation.tags);
const levelStr = wcagLevels.map(l => `${l.version} ${l.level}`).join(', ');
report += `### ${index + 1}. ${violation.id} (${violation.impact.toUpperCase()})\n\n`;
report += `**Description:** ${violation.description}\n\n`;
report += `**Help:** ${violation.help}\n\n`;
report += `**WCAG Level:** ${levelStr}\n\n`;
report += `**Affected Elements:** ${violation.nodes.length} instance${violation.nodes.length !== 1 ? 's' : ''}\n\n`;
if (violation.nodes.length > 0) {
report += `**Example Elements:**\n\n`;
violation.nodes.slice(0, 3).forEach((node, nodeIndex) => {
report += `${nodeIndex + 1}. Selector: \`${node.target[0] || 'N/A'}\`\n`;
report += ` HTML: \`${node.html.substring(0, 100)}${node.html.length > 100 ? '...' : ''}\`\n\n`;
});
if (violation.nodes.length > 3) {
report += `*... and ${violation.nodes.length - 3} more instance(s)*\n\n`;
}
}
report += `**Recommendation:** ${generateRecommendations(violation)}\n\n`;
report += `**Help URL:** ${violation.helpUrl}\n\n`;
report += `---\n\n`;
});
// Recommendations
report += `## Recommendations\n\n`;
report += `### Priority Fixes (by Severity)\n\n`;
const groupedBySeverity = new Map<string, Violation[]>();
violations.forEach(v => {
if (!groupedBySeverity.has(v.impact)) {
groupedBySeverity.set(v.impact, []);
}
groupedBySeverity.get(v.impact)!.push(v);
});
['critical', 'serious', 'moderate', 'minor'].forEach(severity => {
const violationsForSeverity = groupedBySeverity.get(severity) || [];
if (violationsForSeverity.length > 0) {
report += `#### ${severity.toUpperCase()} Issues\n\n`;
violationsForSeverity.forEach(v => {
report += `- **${v.id}**: ${generateRecommendations(v)}\n`;
});
report += `\n`;
}
});
// Action Items
report += `## Action Items for Developers\n\n`;
report += `Use this checklist to track progress on fixing accessibility issues:\n\n`;
actionItems.forEach((item, index) => {
report += `${index + 1}. [ ] ${item}\n`;
});
report += `\n---\n\n`;
// Test Cases
report += `## Initial Test Cases for Playwright\n\n`;
report += `The following test cases can be added to your Playwright test suite to prevent regression:\n\n`;
report += `\`\`\`typescript\n`;
report += `import { test, expect } from '@playwright/test';\n`;
report += `import AxeBuilder from '@axe-core/playwright';\n\n`;
report += testCases;
report += `\`\`\`\n\n`;
report += `---\n\n`;
report += `*Report generated on ${timestamp}*\n`;
return report;
}
// Main execution
const auditsDir = path.join(__dirname, '..', 'audits');
const files = fs.readdirSync(auditsDir).filter(f => f.endsWith('.json') && f.includes('audit'));
if (files.length === 0) {
console.error('No audit JSON files found. Please run the scan first: npm run scan:a11y');
process.exit(1);
}
// Use the most recent file
const latestFile = files.sort().reverse()[0];
const jsonPath = path.join(auditsDir, latestFile);
const data: AxeResult = JSON.parse(fs.readFileSync(jsonPath, 'utf-8'));
const report = generateReport(data);
const dateStr = new Date(data.timestamp).toISOString().split('T')[0];
const reportPath = path.join(auditsDir, `audit-${dateStr}.md`);
fs.writeFileSync(reportPath, report);
console.log(`\n✅ Report generated successfully: ${reportPath}`);
console.log(`\nSummary:`);
console.log(`- Total violations: ${data.violations.length}`);
console.log(`- Critical: ${data.violations.filter(v => v.impact === 'critical').length}`);
console.log(`- Serious: ${data.violations.filter(v => v.impact === 'serious').length}`);
console.log(`- Moderate: ${data.violations.filter(v => v.impact === 'moderate').length}`);
console.log(`- Minor: ${data.violations.filter(v => v.impact === 'minor').length}`);
```
## Step 9: Create Playwright Test Cases
Create `tests/accessibility.spec.ts`:
```typescript
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test.describe('Accessibility Tests', () => {
test.beforeEach(async ({ page }) => {
await page.goto('https://sv.siman.com/');
await page.waitForLoadState('networkidle');
await page.waitForTimeout(3000);
});
test('should pass all WCAG 2.0 Level A checks', async ({ page }) => {
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag2a'])
.analyze();
expect(accessibilityScanResults.violations).toHaveLength(0);
});
test('should pass all WCAG 2.1 Level A checks', async ({ page }) => {
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag21a'])
.analyze();
expect(accessibilityScanResults.violations).toHaveLength(0);
});
test('should pass all WCAG 2.2 Level A checks', async ({ page }) => {
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag22a'])
.analyze();
expect(accessibilityScanResults.violations).toHaveLength(0);
});
// Add specific violation tests based on your audit results
test('should not have image-alt violations', async ({ page }) => {
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag244', 'wcag412'])
.analyze();
const violations = accessibilityScanResults.violations.filter(
v => v.id === 'image-alt'
);
if (violations.length > 0) {
console.log('Image-alt violations found:');
violations.forEach(v => {
console.log(`- ${v.help}: ${v.nodes.length} instance(s)`);
});
}
expect(violations).toHaveLength(0);
});
});
```
**Note:** Update URLs and add specific test cases based on your audit findings.
## Step 10: Create Audits Directory README
Create `audits/README.md`:
```markdown
# Accessibility Audit Reports
This directory contains accessibility audit reports generated using axe-core and Playwright.
## Running an Audit
```bash
npm run scan:a11y
```
## Generating a Report
```bash
npm run generate:report
```
Reports are saved as markdown files with comprehensive analysis including:
- Executive summary
- Detailed issue descriptions
- Compliance levels (WCAG 2.0, 2.1, 2.2)
- Recommendations
- Action items for developers
- Test cases for Playwright
```
## Step 11: Create .gitignore
Create `.gitignore`:
```
node_modules/
playwright-report/
test-results/
*.log
.DS_Store
```
## Usage
### Run Accessibility Scan
```bash
npm run scan:a11y
```
This will:
1. Navigate to the target website
2. Run comprehensive accessibility scan
3. Save raw results as JSON in `audits/` directory
### Generate Report
```bash
npm run generate:report
```
This will:
1. Read the most recent audit JSON file
2. Generate comprehensive markdown report
3. Save report in `audits/` directory
### Run Tests
```bash
npm test
```
Or with UI:
```bash
npm run test:ui
```
## Customization
### Change Target URL
Update the URL in:
- `scripts/run-accessibility-scan.ts` (line 12)
- `tests/accessibility.spec.ts` (in beforeEach)
### Modify WCAG Tags
Edit the tags array in `scripts/run-accessibility-scan.ts` to focus on specific WCAG levels or categories.
### Add Custom Recommendations
Extend the `generateRecommendations` function in `scripts/generate-audit-report.ts` to add custom recommendations for specific violation types.
## Project Features
✅ Automated accessibility scanning with axe-core
✅ Comprehensive WCAG 2.0, 2.1, and 2.2 compliance checking
✅ Detailed markdown reports with actionable insights
✅ Playwright test cases for regression prevention
✅ Severity-based prioritization of issues
✅ Developer-friendly action items
✅ CI/CD ready test suite
## Resources
- [WCAG Guidelines](https://www.w3.org/WAI/WCAG21/quickref/)
- [axe-core Documentation](https://github.com/dequelabs/axe-core)
- [Playwright Documentation](https://playwright.dev/)
- [Digital A11y Cheat Sheets](https://www.digitala11y.com/accessibility-cheat-sheets/)
## Support
For issues or questions:
1. Check the [axe-core GitHub issues](https://github.com/dequelabs/axe-core/issues)
2. Review [Playwright accessibility testing guide](https://playwright.dev/docs/accessibility-testing)
3. Consult WCAG guidelines for detailed requirements
