Invitation System Update

Here’s what’s new —

Device-specific Invitations:

We will now send out invitations for a specific device or OS you have listed in your profile.
This means if we have a test where some requested devices are uncommon, you will be invited for one of those devices. For example, we might invite you to test on a Samsung Galaxy S6. When you accept this invitation, you must use this device to test, even if there are other devices in the test you also might own.

This also applies to operating systems. If we invite you to test using a certain OS, you can use any device you have operating on this OS, but you must test on the OS for which you are invited. Reproductions can be done with any device in scope.

Tester Segmentation:

The system will be classifying testers based on performance. If you are really good at rapid tests, you may be marked as a “rapid test tester”. What impact does this have? We reserve extra invitations for testers who are parts of these performance-based segments

If you perform better in certain test types, you will have a better chance at being invited to these test types.
If you are not part of one of these specialist segments, you will still receive invitations to those tests.
Greenhorn testers (less than 50 bugs reported) have their own segment and will not be rated on their performance initially. As soon as you have reported 51 or more bugs our system will classify you based on all bugs you reported. So make sure you’re always doing your best work, even as a Greenhorn tester.

Test Distribution:

We consider test distribution updates the most important change.
Tests will be distributed equally among testers within one segment.
If you test as well as another tester, and your device coverage is the same, both of you should get about the same number of invites.
Many of you have told us lately that you are not getting as many tests as other testers. This issue should now be fixed. If you think this is not the case, please let us know!

The Leaderboards:

Points you score for reporting bugs and other activities do not impact the invitations you receive anymore.
We will keep the leaderboards active so you can compare yourself with other testers, if you want to.
But remember, rejected bugs will influence your performance rating and you are less likely to be in a top segment if you get a lot of bugs rejected.

Coming up —

Tester Profile:

Currently, you cannot see your own performance indicators. This is something we would like to change in the future.
To do so we are in the planning phase of developing a “profile page” where you can see things like “number of bugs posted,” “bug acceptance rate,” and other performance indicators.

One thought on “Invitation System Update”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s