In this article, you will learn how to properly document a bug according to our rules and standards. In order to understand a bug, customers need sufficient information in a certain quality.
After finding the bug submission form, you have to select the right feature. Make sure the first drop-down shows the feature that you want to submit a bug report for. If you cannot find the right feature, go back to the test overview page, go through all features and mark the one as read that you want to submit a bug for.
The whole bug form will now be visible to you and will look like this for the functional bug type:
In the following sections, we will go over each field and explain in detail what you should enter and how.
For the bug type Functional, you will see an extra option called Severity with between one and three buttons: Low, High, and/or Critical. The severity indicates the urgency of your report. It is made up of multiple factors. To select the appropriate severity for your bug, visit our separate article Functional Bugs.
The field Severity will not be displayed when you select bug types other than Functional.
The bug report title must summarize the problem, so that the reader gets the general idea of the bug just by reading the bug title. They should not have to read the whole report to understand what the problem is.
As a summary, titles should be precise and not too long at the same time. It should contain the concerned element (e.g. search field), what functionality is broken, and the origin.
Describe what is happening instead of what is not happening. Your title should never state that something does not work, otherwise the reader has no idea what is actually happening. Titles must not be abstract.
Titles must reflect the real problem. If the bug only occurs under a certain condition, the condition must be included in the bug title. If you are, for example, not able to book a ticket when you state that you are a teenager, this is a relevant information and must be included.
Titles should not include URLs. Instead, name what page you are on, e.g. the Help Center.
A good approach is to put yourself in the shoes of someone who did not test the website or app, who cannot picture what page you were on, what it looks like and what you did. Read your title from that person's perspective and think about if you would understand the bug. If you don't get a good idea of the bug, adjust the bug title and repeat the process.
Exemplary bug titles
Wrong: Search does not work
Correct: Search is not triggered when clicking the search button – only after pressing the enter key
Wrong: Sort method X does not work properly
Correct: When setting sort method X on page Y, products don’t change order. No change is visible.
Problems of both titles: There are many possible scenarios where this title would fit. They are too abstract. The reader does not know what you did and how the page responds. Testers are forced to read through the whole report to understand what the bug is. Other testers cannot distinguish this bug from others, which increases chances that duplicate bug reports gets submitted.
Enter the URL of the web page where the bug occurs into the URL field. Just visit the page before the bug appears and copy & paste the URL from your browser's URL field into the field URL of the bug report form.
The URL must be complete, so that it redirects to the correct destination.
If you test an app, please leave this field blank unless stated differently in the test description or in the test chat.
Steps to reproduce
Bugs have to be reproducible and they need a detailed step-by-step guide on how they can be reproduced. Each step describes your actions. (Note that you don't have to number your steps as this is done automatically by our system.)
The first step must contain the URL if you test a website or the app name if you test a mobile app. Afterwards, enter all steps that you took from there – what buttons you press, what links you follow, and what you enter. Your last step must describe the action that you perform that triggers the bug.
Your steps should be as general as possible. Only if your bug occurs under specific conditions, e.g. only for a specific product overview page, for a specific filter, for a specific input you make, name that condition in your steps. For example, don't explain that you visited a specific product overview page, then a specific product detail page, and added a specific product to the cart if the problem occurs for any product. This will help the reader grasp the idea of your bug and they will not get distracted by irrelevant details.
If you have to be specific, e.g. when you are asked to document what credentials you used to log in to your user account, use names that are actual words and don’t just smash your keyboard. A bad example would be “asdlkfgjasdfkg lajsdhjasd” as this does not look professional to our customers.
- Go to http://www.examplewebsite.com
- Locate the search function in the upper right corner
- Enter any search query, for example, “San Francisco”
- Click on “Search Now” or hit the enter key
Sometimes it makes sense to not only state what action you performed, e.g. what button you clicked on, but to also state what happens afterwards. The right place to put this information is at the end of the same step. You can use the greater-than character “>” to indicate the result of the previous action.
- Click on button “Add to cart” once > The product is added exactly once.
- Click on the button “Add to cart” with a double-click.
In this example, you point out the different behavior by stating the result of the second last step. What happens after the double-clock would then be explained in the Actual Result field.
Describe what you expect to happen after performing your last described step. As always, details are the key. Think about what should have happened if you had not found your bug, so if everything would work correctly.
Exemplary expected result
Wrong: Town name search function works
Correct: After entering the town name “San Francisco”, available hotels in San Francisco should be displayed. If there are no such hotels, a message should be displayed, e. g. “There are no hotels available in your town”.
The actual result is one of the most important fields of a bug report because here you explain what the problem is and all further information that are needed to understand the bug.
What actually happens after following your step-by-step guide should be described in detail. Be very precise. Don't be too general by e.g. saying products remain mostly in the same order after applying sort method X, but make specific examples what products are not in order.
Add any information in this field that are relevant for the bug, e.g. examples, further conditions, or exceptions. Just make sure to structure your information to help the reader understand your thinking process.
Important note: The actual and expected results must never be just the opposite of each other. Your expectation and what actually happens instead differ greatly. Make sure that you enter information that are not similar.
Exemplary actual result
Wrong: Town name search function doesn’t work
Correct: After you click on “Search now”, nothing happens. No search results are displayed on a new browser page and the user doesn’t get any message or notification about what went wrong. The button “Search now” does not show any implemented functionality.
As of now, you can leave that field empty. It's a deprecated feature that we currently don't use. It might be removed in the future or we'll implement it differently.
At least one attachment must be uploaded for every bug report. To learn what kind of attachment must be attached for your bug and what rules apply, please visit our article for requirements for bug report attachments.
It’s important for us and our customer to know which device you experience the bug on. For a website test, click on the browser icon next to the device that you used. For a mobile app test, select the device that has the app installed.
You may only use devices to test the test environment that are listed in this section. Your device, browser, and/or operating system specification must match the device that you show on your attachments.
Make sure you select the right environment during the submission process because this is the only piece of information that cannot be changed afterwards – neither by your nor the team leader. Unfortunately, your report has to be rejected if you selected the wrong environment, but you can resubmit your bug if the test is still running and if no other tester already submitted the bug properly in the meantime.
You would like to test on a device that cannot be added to your profile? Send us a request via the support chat and we will check if your device is relevant for our customers.
Note: When you remove the device that you were invited for on your device list in your tester profile, you can no longer submit reports in that test. The environment section of the bug form will be empty and the form cannot be sent. Deleting a device in your profile cannot be reversed!
Straight rejection rule
Please invest enough time in documenting your bugs because team leaders will not improve your bug reports to reach the required level of quality; it's your job to provide sufficient information in a sufficient quality. If you are no longer a greenhorn tester, team leaders will have to straight-reject bug reports if one or more of the following cases is true for your bug report:
- Several grammar mistakes (not just a typo)
- Repeated improper feature choice
- Overall bug report description not understandable
- Unclear bug report title
- Missing important steps
- Tester clearly did not read or follow test instructions
- Inconsistent evidence – when the discrepancy between steps and screencast is too large.
- Incomplete evidence such as missing attachments including missing crash logs.
If one of your bug reports was rejected because the quality was not sufficient, you are allowed to submit it again with proper quality unless another tester already has already done so in the meantime.