Bug Report Requirements

How do I document a bug report properly and what are Test IO's standards?

Markus avatar
Written by Markus
Updated over a week ago

In this article, you will learn how to properly document your bug according to our rules and standards. In order to understand a bug, customers need sufficient information with high-quality documentation. You will find more detailed information about our rules in each section of this article, but here's a quick summary of our bug report requirements:

  • If you're reporting a functional bug, you must select one of the available Severities before filling the rest of the bug report.

  • The Title must answer the questions of what happened, where the bug happened and when it is triggered, reflecting the real problem and avoiding describing what didn't happen (instead, focus on what actually happened).

  • The URL must be the link copied from your browser of the page where the bug is triggered.

  • The first Step to Reproduce must contain the URL of the landing page or the app name. The other steps should describe the actions you took to trigger the issue, with the last step being the last action taken to trigger the issue (and not "observe").

  • The Actual Result should be formed by one or more sentences explaining what happened after the last action. You can also add results of previous actions if they are necessary to understand the bug. It must not be the same as the title.

  • The Expected Result should contain information about what should have happened instead after you performed the last step to trigger the bug. It must not be a copy of the actual result with minor changes since those fields are designed for different purposes.

  • If an Attachment is required for your report, you must not forget to attach one.

  • Finally, you must select the correct Used Environment and browser (if applicable) used for testing, based on the device you were invited to test with when you accepted the cycle.

Your first step in the bug report form should be selecting the right Feature. If you cannot find the right Feature in the drop-down list, go back to the test overview page, go through all Feature descriptions and mark them as read. After that, you can go back to the bug report form and all the Features will be presented in the drop-down list.

Bug Form

After selecting the Feature, the whole bug form will be visible to you. A form for functional bugs, for example, looks as follows:

You should fill in every field of the bug form with the correct specific information according to our quality standards. More detailed information about every field and its requirements can be found below.

Severity

For Functional bugs only, you will see an extra field called Severity: Low, High, and/or Critical. The severity indicates the urgency of your report and depends on multiple factors. To learn about different severity levels, please visit the following article Functional Bugs.

The field Severity will not be displayed for other bug types.

Title

The bug report title should summarize the problem so that the reader gets a general idea of the bug just by reading the bug title. They should not have to read the whole report to understand what the problem is. Your bug report title should be precise and not too long at the same time. It should contain information: what is the bug, where did the bug happen, and when the bug is triggered? So when you write a title for your bug report, always remember: What? Where? When?

When you write a bug title, describe what is happening instead of what is not happening. Your title should never state that something does not work, otherwise the reader has no idea what is actually happening.

Titles must reflect the real problem. If the bug only occurs under certain conditions, the conditions must be included in the bug title. If you are, for example, not able to book a ticket when you state that you are a teenager, this is relevant information and must be included in your title.

To come up with a descriptive title, put yourself in the shoes of someone who never tested the website/app, who cannot picture what page you are on, what it looks like and what you did. Read your title from that person's perspective to see if you would understand the bug. If you don't get a good idea of the bug, adjust the bug title and repeat the process.

Exemplary bug titles

Wrong: Error shown on the Cart page
Correct: An "Error 500" will be shown on the Cart page after the user clicks on the "Checkout" button

Wrong: The user cannot add a product to the Cart
Correct: "Unexpected Error" was shown on PDP, when the user selected Size and clicked on the "Add To Cart" button.

What is wrong with the examples above: there are many possible scenarios in which these titles would fit because they are too abstract. The reader does not know what actions you perform and how the system responds to them. Hence, the reviewer has to read through the whole report to understand what the bug is and cannot distinguish this bug from others easily.

URL

Visit the page where the bug appears and copy & paste the URL from your browser's URL field into the field URL of the bug report form.

The URL must be a valid one.

Steps to reproduce

Bugs have to be reproducible and they need a detailed step-by-step guide on how they can be reproduced. Each step should describe a separate action.

Note that you don't have to number your steps as this is done automatically by our system.

The first step must contain an indication to access the URL of the landing page provided by the customer in the Access section if you test a website or an indication to open the app (with its name) if you test a mobile app. All further steps should describe your actions from the initial step up to the point when the bug occurs – what buttons you press, what links you follow, and what you enter. Your last step must describe the action that you perform that triggers the bug. Remember that "observing" is not an action taken by the user.

Your steps should be as general as possible. Only if your bug occurs under specific conditions, e.g. only for a specific product overview page, for a specific filter, for a specific input, etc., name this condition in your steps. For example, in your steps, don't describe the specific product overview page you visited, then the specific product that you added to the cart if the problem occurs for any product. This will help the reader grasp the idea of your bug and they will not get distracted by irrelevant details.

Finally, make sure your steps contain the least amount of actions possible to perform. After reading each step, the person reproducing your reported bug should be able to complete them on the website or app. They shouldn't have to check the same step multiple times to remember what needs to be done.

Exemplary steps

  1. Enter any search query in the top-right search bar (e.g. “San Francisco”)

  2. Click on “Search Now” button

  3. Scroll down and click on "Sort by"

  4. Select the option "Sort by price: High to Low"

Exemplary BAD steps

  1. Observe

  2. Search > Sort > High to low

  3. Observe

What is wrong with the examples above: The first step must contain an indication to access the URL of the landing page, not just the URL itself. The third step is not enough detailed and contains too many actions in a single step. The second and fourth steps are redundant and not necessary to understand the bug.

Actual Result

The actual result is one of the most important fields of a bug report because here you explain what the problem is and all further details that are needed to understand the bug.

What actually happens after following your step-by-step guide should be described as detailed as possible. Try to be very precise and don't be too general by e.g. saying products remain mostly in the same order after applying sort method X but instead describe specific examples of the products that are not in the correct order. Add any information in this field that is relevant to the bug, e.g. examples, further conditions, exceptions or results of other important if necessary. Just make sure to structure your information to help the reader understand your thinking process.

Important notes: The actual and the expected results must never be just the opposite of each other. The expectation of what should've happened and what actually happened instead differ greatly.

Similarly, the actual result must not be the same as the report's title. While the title is a summary of the problem, the actual result should be a detailed description of it and should include further details such as scenario information, examples and results obtained while performing the steps to reproduce the bug.

Exemplary actual result

Wrong: Error shown on the Cart page after clicking on the Checkout button.
Correct: When the user added some product to the Cart and tried to proceed to the Checkout page, he will notice that he will be not able to do that. An "Error 500 - Internal Server Error - Sorry, something went wrong" will be shown when the "Checkout" button is clicked in the right side menu.

Wrong: The user cannot add a product to the Cart, an error is shown.
Correct: After the user opened "Test IO - Product 1" detail page, selected Size: 36, and clicked on the "Add To Cart" button, he will experience an "Unexpected Error" error message banner in the top right corner of the PDP, and the product is not added to the Cart.

Expected Result

Describe what you expect to happen after performing your last described step. As always, details are the key. Think about what should have happened if you had not experienced the bug - if everything worked correctly.

Remember the expected result is not the same as the actual result with some slight variations or negative words, but a different field designed so you can explain everything that should've happened after the last step to reproduce the bug was completed.

Exemplary expected result

Wrong: User can proceed to the Checkout
Correct: After adding some product to the Cart, and clicking on the "Checkout" button, the user should be redirected correctly to the Checkout page, where he should be able to add shipping and payment info and place an order.


Wrong: Product should be added successfully to the Cart.
Correct: When selecting Size: 36 for the "Test IO - Product 1", and clicking on the "Add To Cart" button, the product should be added successfully to the Cart. The user should not encounter any errors in this process.

Attachments

To learn what kind of attachment must be attached for your bug and what rules apply, please visit the following article: Requirements for bug report attachments.


Used environment

It’s important for us and our customers to know which device you used when you experienced the bug. When testing a website, click on the browser icon next to the device that you used. When testing a mobile app, select the device that you used for testing and that has the app installed.

You may only use the devices for testing that are listed in this section. Furthermore, you should select only one device or browser when you are reporting the bug, and upload only attachments for it. If you are able to reproduce the bug in other devices or browsers, please mention this in your Actual result.

Selecting the correct environment where the bug occurs is a mandatory action. When you submit your report, please make sure that you select the correct environment. If you accidentally select the wrong environment, you can correct it after you submit it. You can make this correction by changing your environment selection before the team leader reviews the bug report. If the bug report contains the wrong environment, TL will reject it during the bug report review.

You would like to test with a device that is not in the list of available devices in your profile? Simply send us a request via the Support chat and we will add your device to your list provided it is relevant to our customers.

Note: When you remove the device that you were invited for from your device list in your tester profile, you can no longer submit reports in this test. The environment section of the bug form will be empty and the form cannot be sent. Deleting a device in your profile cannot be reversed after you've accepted a test invitation!

Improving your report

After you submitted your report, you can still edit all fields except the bug type selected. You must always submit complete bug reports that are ready to be reviewed and only use the Edit option in case you've made a small typo mistake or wants to rephrase your words to improve the report quality.

Remember placeholders are not allowed, so do not submit incomplete reports to edit them later.


If your bug only happens to any specific input, use terms that would be used by real users and avoid entering random keywords while creating the bug report or the attachments. A bad example would be to use a username while creating an account on the customer interface such as “asdsdfkg_lajsdh” as this makes your report look unprofessional.

Did this answer your question?