Test Process

What does a regular test entail and how to submit a bug?

Markus avatar
Written by Markus
Updated over a week ago

"You can join as many tests as you want at once; however, consider how much time you can dedicate and will need to participate in all of them."

Motivation

Tests are mini-projects by one of our customers that we regularly invite you to. Each test covers a certain scope of (usually) one of their products. By testing the given scope, you will find bugs that you can then report for one of the given features in this test. You get paid for your report if you submit a valid bug and the team leader approves it. Other tasks may also be available in your cycle, but you will mainly be hunting for bugs in Test IO cycles.

Test invitations

When a customer submits a new test, our distribution algorithm checks which testers match the test's requirements regarding their profile information, devices, and individual performance. If you match all criteria, you will receive an invitation by email. You can also find all your pending invitations under “Interesting tests for you” on your dashboard. We recommended adding all your devices to your Profile (such as your Smartphones, Tablets, Computers, Wearables and Streaming devices) to increase the number of invites to cycles.

Accepting or rejecting an invitation

Whether you accept or reject an invitation is entirely up to you. You can join as many tests as you want at once; however, consider how much time you can dedicate and will need to participate in all of them. Keep in mind that we measure your activity in tests.

If you don’t wish to participate in a test, you can reject the invitation. Given the case that you accepted a test and want to quit it later on, you will find this option at the bottom of the right sidebar on the test overview page. The quit option is only available as long as you haven't submitted bug reports and completed other tasks such as Paid Activity Sessions, Reproductions or User Stories in the test and the test is not over yet. Here's an image of the Quit test button


Test scope

All cycles will have several pieces of information that need to be carefully read by you as soon as you join them. It's important that you understand everything that was written down by the customer so you won't wrongly test the product. All information about the test scope and its elements can be found in this article here.

Participating in a test

After joining a test, you will see the test overview page:

Before beginning to test, you have to read through all the instructions and feature descriptions to grasp the scope of the test. You can do so before the test starts to be able to test right away once the test begins. A countdown towards the top of the page indicates how much longer you have to wait until the start or how much time remains before the test ends.

As part of our testing protocols, testers are strictly prohibited from utilizing browser AddOn translators to translate platform content. However, testers are permitted to employ native browser translation services such as Google Chrome Translate or Microsoft Translator in Edge browser.

Furthermore, testers must handle the customer's data professionally, ensuring that information is not exposed to third-party tools during translation processes. This safeguard maintains the confidentiality and integrity of the customer's data throughout the testing cycle.

You must answer the question "Will you participate in this test?" positively if you agreed with the instructions and want to test or negatively if you don't want to participate. You will be required to give a reason for not taking this cycle and we appreciate your honest feedback here. You may or may not be able to select the device, depending on the seats left for each cycle.

Remember, you will only be able to use this device to submit bugs the entire cycle until you become a Bronze Level Tester, so we recommend you be careful with your choice.

Only after the test starts and while the test is running, the test environment becomes visible to you. To start testing, you have to access that test environment first. If a mobile or streaming app is supposed to be tested, install the app on your mobile or streaming device. 

You will need to start a Test Session in order to submit reports and complete the available tasks. Once you've found a bug on the test environment that is in scope, submit it for one of the given features.

Here's how the test overview will look like once you can access the environment, agree with the test instructions and start a Test Session. Notice the Submit Bug button is now active and you can start bug hunting.


Review of your work

In every test, there is a team leader who can help you but their main focus is to check submitted bug reports. If necessary, the team leader will ask for further information when necessary and approve or reject bug reports. It's important that you check from time to time your bug report status to make sure the TL did not send you any requests.

Bug reports approved by the team leader will be forwarded to the customer. Customers will review the forwarded bug reports and will make the final decision – the bug report either gets accepted or rejected. Customers can also send requests to testers, so it's important that you keep an eye on your email notifications even when the cycle is over.

Furthermore, if you don't agree with a team leader's decision, you can leave a comment in your bug report and ask them for further information about it. If you still don't agree with their response or if they end up taking too long to answer your request, you can also open a dispute

Where to find more information about the test process?

We’re always creating content to help you understand the testing process on our Discord server Test IO Community or our Social Media channels. Whenever you are not sure about something in our cycles, the Team Leader will usually be your best source of information and you can use the chat feature to contact them.

Finally, you can listen to the podcast below for more information about ❝Understanding Tests Setup and Contradicting Instructions❞.

Did this answer your question?