For this special kind of testing, we have introduced Custom Reports specifically designed for Accessiblity testing. Please note that additional requirements apply for accessibility reports on top of the requirements for regular bug reports formulated in the Bug Report Requirements and Bug Report Attachments academy articles.
Additional obligations that each accessibility report must adhere to are described in the following chapters. You can find two examples at the end of the article to showcase accessibility reports with applied rules.
To start submitting an Accesibility Report, please click "Submit Report" button at the bottom of the page. Then, select the feature you are testing and Report Type (if available).
Title
As you can see, there are several fields that are used to generate a Final title.
Feature/page: filled automatically based on your selection.
Title: please provide here a short, descriptive title for your bug (Accessibility level + WCAG checkpoint will be automaticaly added)
Final title: auto-generated in a format of [level][WCAG checkpoint] Feature name: title
WCAG checkpoints: please select the corresponding checkpoint(s)
Level A issues | Level AA issues |
|
|
|
|
Issue Type
A typical accessibility report is a violation. Best Practice (if available) would report an issue that is not a direct violation of A11Y criteria, but still something that is often done differently for better accessibility. Please report Best Practice reports only if stated in Test Instructions.
Steps to reproduce
In general
First step: In addition to the call of the test URL, include the name of the tool you used in your first step. Replace the <tool>
placeholder with e.g. NVDA, JAWS, WAVE, etc. in the following template:
Using <tool>, open <test URL>
, so e.g.Using NVDA, open https://www.test.io
Multiple elements: If a checkpoint fails for multiple elements of the same type (e.g. button or link), group these repetitions in one report.
Multiple occurrences: To document multiple identical occurrences of an issue on the same page or on different pages, list them in your actual result under a section called “Other occurrences:”.
Described actions: Since visually impaired or blind people do not use a computer mouse, do not describe mouse actions. Instead, describe keyboard and touchpad actions, such as:
Use the TAB key to navigate to User Account
Press the ENTER key to activate Sign out link
Swipe left to focus hamburger menu
Automated tools: You are welcome to use tools like WAVE to identify problems, but your report, including your steps, must be described from a user's point of view. You may not describe in your steps how you use an automated tool and what it reports, but must rather describe everything as if you had not used any tool but simply based on the WCAG checkpoints.
Color contrast issues
Your steps must be the following (replace placeholders):
Open <test URL>
...
...
Measure the color contrast ratio of <element name>
Problems with the same color set may only be reported once. E.g. a problem with blue foreground on white background and white foreground on blue background are the same color set and cannot be submitted twice.
Actual Result
In the actual result field – besides the typical actual result that you have to enter in regular exploratory tests –, provide which checkpoint failed, e.g.:
Failed WCAG 2.2 checkpoint(s): 1.3.2 Level A
Attachments
What attachment has to be attached depends on the type of accessibility problem:
Color contrast: screenshot
All other problems: screencast
Screenshots: As in regular tests, the issue must be marked on screenshots.
Screencasts: While recording your screencast, please slow down your actions before you reach the element that has the issue. If you record your actions at normal speed, the viewer will otherwise miss the issue. If you perform actions too fast while recording, the TL might ask you for an improved recording.
Optional Fields - for experts only
There is a set of optional fields marked with "for experts only (optional)".
Category – for experts only (optional)
Recommendations – for experts only (optional)
Techniques – for experts only (optional)
We highly recommend to leave the mentioned fields empty unless otherwise stated in Test Instructions.
Keyboard navigation issues
Your screencast must include what keys you pressed on your keyboard during your recording, otherwise viewers cannot comprehend the issue. Enable the on-screen keyboard on your computer, which is available for both Windows and macOS. A virtual keyboard will overlay part of your screen and you simply click on a button to simulate the corresponding key press.
Examples
Screen reader example
Title:
[AA] [2.4.3] Homepage: Focus not trapped in hamburger menu modal
Steps:
Using VoiceOver, open https://test.io/
Swipe right to focus hamburger menu
Double tap to activate it
Swipe through the expanded hamburger menu modal and observe
Actual result:
The focus continues to move through the background content behind the modal, which is not part of the overlaying hamburger menu.
Failed WCAG 2.1 checkpoint(s): 2.4.3 Level AA
Expected result:
The focus should be trapped in hamburger menu modal. Hidden elements from the underlying page should not be focused and announced.
Keyboard navigation example
Title:
[AA] [2.4.7] Demo: Get Demo link does not have visible focus outline
Steps:
Using NVDA, open https://test.io/
Press TAB to focus “Get a Demo” button under “Do you take Quality Personally” section and observe
Actual result:
The “Get a Demo” link in the “Do you take Quality Personally” section does not have a focus outline.
Other occurrences: Social Media icons
Failed WCAG 2.1 checkpoint(s): 2.4.7 Level AA
Expected result:
Actionable elements must have visible focus outlines when they are focused.
Color contrast example
Title:
[AA] [1.4.3] White text on blue background for “Get a demo” link fails color contrast requirements
Steps:
Open https://test.io/
Scroll down to “Get A Demo” link
Measure the color contrast ratio of “Get a demo” link
Actual result:
The “Get a Demo” link fails the color contrast requirements. The blue foreground text (#21BEF4) on the white background (#FFFFF) has a contrast ratio of only 2.15:1.
Failed WCAG 2.1 checkpoint(s): 1.4.3 Level AA
Expected result:
Small text should have a contrast ratio of at least 4.5:1. Large text (14 pt bold or 18 pt and larger) should have a contrast ratio of at least 3:1.