Introducing the new test IO tester profile

We asked what you, our community wanted to see introduced to test IO in order to improve your experience. Consequently, we are very excited to announce one of the most requested features. Our new tester profile page allows you to view key metrics relating to your testing with us, as well as allowing you to review how you compare to other members of the community.

In order to view all details on your tester profile page, you need to meet a number of criteria (listed below) relating to your activity in the last 90-days. We want to keep your profile relevant and allow you to see your improvement in real time, which is why we decided to calculate your scores based on the last 90-days.

  • You need to have accepted at least 5 test invitations.
  • You need to have at least 10 bugs or 40 reproductions submitted.

Your testing contributions will only be considered in this calculation once the test is fully finalized. This can be up to 10 days after the end of the test depending on the customer activity level.

Now that you have access to all this new information, you need to understand what it really means and what you can do to change it. There are two different scores which create the basis for the page, the quality score and the reliability score.

Quality score

Just in case you haven’t already realized it, we value quality extremely highly, and as a result we have included a score so that you can track yours. It is comprised of the following elements:

  • Approval rate: If your bugs are approved by the team leader then you’re clearly providing high-quality bugs and we want to acknowledge your work.
  • Accuracy: When your bugs are rejected by the team leaders for being out of scope, or because you have not followed the instructions, it is a sign that your testing has not accurately met the test specification. As a result, these types of rejections will reduce your quality score more than the other rejection reasons, so be sure to read the instructions carefully.
  • Originality: Reporting new and original bugs is far more valuable and skillful than reporting the same things again and again. We want to try and give extra credit for finding those bugs.

Reliability score

This score is a reflection of how reliable you are when it comes to your testing with us and is comprised of a number of elements.

  • Contribution rate: In simple terms, we want to display how much you contribute to a test, in terms of bugs and reproductions per test. So even if you don’t test that often but contribute a lot when you do, this will reflect how valuable your work is. Only team leader accepted bugs and reproductions are considered for this.
  • Request rate: Sometimes we need to contact you for more information related to your bugs and we rely on you to respond in a timely manner. This element of the score simply considers how often you manage to reply to customer and TL requests within the time limit.
  • Activity: We love an active tester, but we realize that modern lives are busy and we pride ourselves on providing you with flexible work. As a result, we decided to review your activity over the entire 90-day window in terms of tests accepted, then taking an average to give a fair representation of your testing. So if you have a busy week, or take a well earned holiday, it will still only have a small impact on your activity rate.

Activity Tracker

In order to help you work as efficiently as possible, you will now easily be able to see when you are most active in terms of the bugs and reproductions you submit. The activity tracker works on a colour scale, with darker shades of blue reflecting days with higher activity. When you select any one day from the calendar view you can see your total number of contributions on that day.

Our customers want to say thank you

We are also introducing more options for our customers to tell you how much they appreciate your work. You will see two new numbers on the tester profile; the number of 5 starred tests and the number of liked bugs. This data relates to your entire career with test IO and is not limited to the last 90 days.

  • Liked bugs are exactly as they sound, bugs which the customer particularly liked. It might be that you found a really hidden bug, uncovered a major issue or simply provided a really good bug report. Whichever is true, they just wanted to let you know how great your bug is.
  • 5 starred test cycle ratings are tests where customers have rated the overall test 5 stars. This is really a tester team score, but it will allow you to see how often you contribute as an active team member in a 5 star test.

You will also find a couple of lifetime metrics in this section displaying the number of approved bugs you have reported, as well as the total number of test cycles you have contributed to. We plan to add a number of other metrics to this section over time to give you the best possible overview of your valuable work.

This is only our first version of the tester profile and we already have a number of ideas of what we can enhance, however, as ever it is your feedback we value the most. Let us know what you would like to see, what you don’t like or just what you think of your shiny new tester profile.