All posts by Matt_testIO

Introducing the new test IO tester profile

We asked what you, our community wanted to see introduced to test IO in order to improve your experience. Consequently, we are very excited to announce one of the most requested features. Our new tester profile page allows you to view key metrics relating to your testing with us, as well as allowing you to review how you compare to other members of the community.

In order to view all details on your tester profile page, you need to meet a number of criteria (listed below) relating to your activity in the last 90-days. We want to keep your profile relevant and allow you to see your improvement in real time, which is why we decided to calculate your scores based on the last 90-days.

  • You need to have accepted at least 5 test invitations.
  • You need to have at least 10 bugs or 40 reproductions submitted.

Your testing contributions will only be considered in this calculation once the test is fully finalized. This can be up to 10 days after the end of the test depending on the customer activity level.

Now that you have access to all this new information, you need to understand what it really means and what you can do to change it. There are two different scores which create the basis for the page, the quality score and the reliability score.

Quality score

Just in case you haven’t already realized it, we value quality extremely highly, and as a result we have included a score so that you can track yours. It is comprised of the following elements:

  • Approval rate: If your bugs are approved by the team leader then you’re clearly providing high-quality bugs and we want to acknowledge your work.
  • Accuracy: When your bugs are rejected by the team leaders for being out of scope, or because you have not followed the instructions, it is a sign that your testing has not accurately met the test specification. As a result, these types of rejections will reduce your quality score more than the other rejection reasons, so be sure to read the instructions carefully.
  • Originality: Reporting new and original bugs is far more valuable and skillful than reporting the same things again and again. We want to try and give extra credit for finding those bugs.

Reliability score

This score is a reflection of how reliable you are when it comes to your testing with us and is comprised of a number of elements.

  • Contribution rate: In simple terms, we want to display how much you contribute to a test, in terms of bugs and reproductions per test. So even if you don’t test that often but contribute a lot when you do, this will reflect how valuable your work is. Only team leader accepted bugs and reproductions are considered for this.
  • Request rate: Sometimes we need to contact you for more information related to your bugs and we rely on you to respond in a timely manner. This element of the score simply considers how often you manage to reply to customer and TL requests within the time limit.
  • Activity: We love an active tester, but we realize that modern lives are busy and we pride ourselves on providing you with flexible work. As a result, we decided to review your activity over the entire 90-day window in terms of tests accepted, then taking an average to give a fair representation of your testing. So if you have a busy week, or take a well earned holiday, it will still only have a small impact on your activity rate.

Activity Tracker

In order to help you work as efficiently as possible, you will now easily be able to see when you are most active in terms of the bugs and reproductions you submit. The activity tracker works on a colour scale, with darker shades of blue reflecting days with higher activity. When you select any one day from the calendar view you can see your total number of contributions on that day.

Our customers want to say thank you

We are also introducing more options for our customers to tell you how much they appreciate your work. You will see two new numbers on the tester profile; the number of 5 starred tests and the number of liked bugs. This data relates to your entire career with test IO and is not limited to the last 90 days.

  • Liked bugs are exactly as they sound, bugs which the customer particularly liked. It might be that you found a really hidden bug, uncovered a major issue or simply provided a really good bug report. Whichever is true, they just wanted to let you know how great your bug is.
  • 5 starred test cycle ratings are tests where customers have rated the overall test 5 stars. This is really a tester team score, but it will allow you to see how often you contribute as an active team member in a 5 star test.

You will also find a couple of lifetime metrics in this section displaying the number of approved bugs you have reported, as well as the total number of test cycles you have contributed to. We plan to add a number of other metrics to this section over time to give you the best possible overview of your valuable work.

This is only our first version of the tester profile and we already have a number of ideas of what we can enhance, however, as ever it is your feedback we value the most. Let us know what you would like to see, what you don’t like or just what you think of your shiny new tester profile.

test IO tester ms_frodo shares his 5 top tips to crowd testing success!

1st tip: Thoroughly read test instructions before starting testing
Often some testers start testing, without reading all the test instructions. As a result many bugs raised by them are rejected with one of the following reasons: the most popular “bug is out of scope” and two less popular “this is not a bug” or “this bug is known”. This tip will allow you to escape rejected bugs, as result, keeping your tester ranking and not wasting time for nothing.

2nd tip: Increase quality of your bug report
When you start filling out the bug report, make sure that you include all important information needed to reproduce your bug. Improve your own English level (if it is not already very good) for writing bug reports grammatically correct and understandable for reviewer and customer. Use rule of three ‘W’ (What, Where, When) in your bug title. This tip will allow you to write quality bugs, as result again escaping rejected bugs. And one more plus, the reviewer and customer will enjoy working with you and your bug reports.

3rd tip: Ensure that your bug isn’t a duplicate
Before raising a bug, check the bug list and ensure that your bug isn’t a duplicate. Sometimes I firstly raise the bug and immediately after I check the bug list to ensure that my bug isn’t a duplicate. Most rejected bugs are rejected for the reason “this bug is a duplicate..”. This tip is very simple, but many testers forget about it and as a result, they have many rejected bugs.

4th tip: Check the full functionality of the app during testing
When you start to execute testing of the app, ensure that you checked all features inside this app. Sometimes, high or critical bugs are located in the most prominent places, I mean features like “upload button doesn’t work” or “unable to sign up via facebook network”. It’s not necessary to go to the deep into the app, in order to find some serious bug. You need to test thoroughly all app features to ensure they work correctly and find the most hidden bugs.

5th tip: Go hand in hand with the test platform
What do I mean? I mean that you should always read and review latest news and updates of the testing platform. Usually testing platforms update their rules of testing and your job can suffer, if you are not aware of the latest updates of your testing platform. Also, if you are a novice it’ll be useful for you to read articles, which are present on the test platform, because there you can learn a lot of useful things, which will help you in your job.

Keep in mind these rules and they will certainly help to improve your work.

Relevant bugs are valuable bugs!

Put yourself in the shoes of a developer or product owner, you have run a crowd test and there are lots of bugs for you to work on and fix. Then you start to work through the bugs and realise that some of them are just not relevant to you, this may be due to the environment, the behaviour to trigger or multiple other reasons. The result, the irrelevant bugs are rejected.

So what can you do to ensure you are reporting relevant bugs?

  1.  Step back into the shoes of the developer and try to understand what they are looking for. The test description and features should give you a very good idea of what is relevant.
  2. Consider the target market of the app or website. If the app is designed for the Russian market then chances are nobody will be interested in a bug relating to a content issue when you change your phone settings to Italian.
  3. If you have to follow steps a real user would never follow in typical usage, then there is no real value provided in your report and it would be an edge case. Edge case bugs are very likely to be rejected. If the bug is never going to impact a customer’s users, they most likely don’t care about it.

 

 

Insider insights: 5 Golden rules to get your bugs accepted.

It’s the one question we are asked more than any other, “How can I get more of my bugs accepted?” Luckily we have experience of nearly half a million bugs reports and hundreds of customers, so we have a pretty good idea of exactly what is required! We believe there are 5 golden rules to follow to maximise your bug acceptance and we thought it was about time we shared them with you.

Rule #1: READ THE TEST INSTRUCTIONS!

It might sound obvious but without reading the feature descriptions, instructions and test chat fully, you will not know exactly what the customer is looking for. Even if you find bugs, they may not be the ones they are currently interested in, so check what they’re asking for in the test.

Rule #2: REPRODUCE YOUR BUG

If you can’t reproduce your bug there is very little chance the customer will be able to and nobody is interested in bugs which only happen once!

Rule #3: ONLY RELEVANT BUGS

Something we see often are reports where something clearly goes wrong but the behaviour to trigger this was pretty strange and does not reflect the target user behaviour. Put yourself in the customer’s shoes, you are only going to be interested in bugs which your actual customers are likely to encounter, not ones that happen when you do some random sequence of events. On this point it is always worth considering who the target audience for the product will be.

Rule #4: MAKE SURE IT’S NOT A DUPLICATE

For most tests there will be a known bug list of existing bugs so firstly make sure your bug is not on the list, also don’t forget to check the bugs already reported by other testers in the current test. When you are typing your bug title you will be shown possible similar bugs, so make sure you keep a close eye on this as well as being aware of what is on the list in general.

Rule #5: HIGH QUALITY

It’s another fairly obvious one but if your report and evidence is not clear and well written, the customer may not understand your bug immediately and as a result, reject it. You can find guidance on quality here.

Follow these 5 golden rules to high acceptance rates of your bugs.