Skip to main content
All CollectionsGetting StartedTest IO vs Other Testing Platforms
Comparison of Testlio and Test IO Platforms for Testers
Comparison of Testlio and Test IO Platforms for Testers

Quickly learn about the differences between Testlio and Test IO platforms

Charlie avatar
Written by Charlie
Updated over a week ago


Software testing plays a vital role in ensuring the quality and reliability of software products. Crowdsourced testing platforms like Testlio and Test IO allow freelance testers to contribute their skills to various testing projects.

This article compares the differences between the Testlio and Test IO platforms, specifically focusing on their test processes, bug types, severity of functional bugs, attachments, payouts, and test communication.



Test IO

Test Processes

Testlio follows a structured test process linked to workspaces. Testers are assigned to specific workspaces to conduct a wide variety of testing, such as exploratory testing, payment testing, functional regression testing, localisation testing, accessibility testing, usability testing, live stream testing, and instrumentation SDK testing, in which they perform test cases from top to bottom.

If the test case step fails, testers must submit bugs or issues; if the bug was already submitted, they should execute a reproduction.

Testers can find duplicated bugs when submitting an issue by looking at other testers' reports or searching within the ❝Issue browser❞. If the title contains a minimum of 30 characters, testers will automatically get suggestions of similar issues already reported.

The Testing Team selects testers based on their profiles and device information to match the workspace requirements and invites them via email in two stages. First, the invitation is to the workspace; the second stage is to a run where the time, device and schedule to test is given.

Depending on the workspace, testers may or may not get invitations to a run; everything depends on the workspace's needs.

Testers must reply to the run invitation to be invited to the run. Letting invitations expire is considered a severe issue.

After finishing the assigned time to test, testers need to answer a feedback survey.

After raising an issue, the TL can approve it, request additional information, or
close it. Closing an issue is a rejection and will harm the tester's scores.

Hence, testers wait for others to raise issues and then make a reproduction because submitting an issue might worsen the tester's rating/score if it’s closed (rejected); in contrast, if the bug is accepted, it doesn’t bring any benefits.

Test IO employs an agile test process that involves test planning, execution, and continuous feedback. Testers participate in test cycles that match their language, devices and location. We emphasise real-time and manual exploratory testing, enabling testers to discover and report bugs on the fly.

To start contributing to a test cycle, testers need to start test sessions in advance and acknowledge that the instructions of the features in scope were read and understood.

To identify duplicates, when submitting a bug, the ❝Similar bugs❞ list on the right-hand side of the bug form will show you the list of already submitted bugs in the current test, and you can also search and filter the bugs and Known Bugs list to do so.

If a bug report gets rejected, the tester's scores are negatively affected based on the type of rejection received.

Testers can dispute bugs rejected by team leaders. Once a report is disputed, it gets locked for review by the Bug Dispute Team.

Placeholder reports are highly forbidden.

Based on the devices registered in the Tester's Profile and other information, the system selects one of the devices and invites you to participate in the test. Depending on the seats left for each cycle, you may be unable to select a device other than the one the system chose. You cannot switch to another device once you have been invited to a device. Therefore, you can only submit bugs found on this device.

Bug form

When required, the Testlio bug form or issue form presents the following structure:

  1. Severity

  2. Feature

  3. Labels

  4. Title

  5. Environment

    1. Device and OS

    2. Testable App version

    3. Network

    4. Location

    5. Reproducibility rate

  6. Steps to reproduce

  7. Expected result

  8. Actual result

  9. Suggested Solution (For Accessibility Testing)

  10. Attachment Description

  11. Attachment

The severity, feature and labels are dropdown lists to select options from; the other sections, minus the attachment, are templates testers must complete manually. This applies to every failed step, submitted bug or executed reproduction.

Every issue must have a prefix of the section where the issue occurs before the title if otherwise specified.

Test IO's bug form, on the other hand, simplifies tester documentation with the following structure:

  1. Bug Title

  2. Type of bug with severity for functional bugs

  3. URL field (where the bug happens)

  4. Steps which are the actions performed

  5. Actual result

  6. Expected result.

  7. Attachments

You don't have to follow a specific format to construct a title. However, you must respond to what happened, where the bug occurred and when it triggered.

On our bug form, you don't have to add the step number; our form already provides it. You can drag them around and rearrange them at will.

Except for the browser while website testing, the device's information is retrieved from the device you selected when accepting to participate in a test cycle, which cannot be changed afterwards.

Types of Bugs

Testlio generally classifies issues as:

  • Functional

  • Visual

Functional bus required two attachments for documentation.

Localisation testing requires verification of the UI issue and the content itself.

Test IO categorises bugs as:

  • Functional

  • Visual

  • Content

  • Usability

  • Test Case bugs.

We don't perform security tests.

Here are more specifications. Content bugs are related to every type of information, not only text (e.g., translations; typos are not reported), so missing images and buttons are content bugs rather than visual bugs.

Functional bugs considered performance, like endless loading, can be regarded as functional bugs if they directly affect end-users, for instance, accessing content or progress over a task.

On the other hand, during performance test cycles, which are run as per demand, endless loading is a performance issue, and they are reported as such; therefore, measuring the internet speed and .har files is part of the attachments needed to attach to the report.

The Severity of Functional Bugs

Testlio bug severity for functional bugs is:

  • High

  • Medium

  • Low

Testers decide which severity applies better to the issue based on a brief explanation of each severity.

At Test IO, we offer specific scenarios to guide you through the correct selection of the severity; these are only three:

  • Low

  • High

  • Critical

The scenarios we provide will help analyse the issue in different ways, such as considering the functionality compromised and the bug's potential impact on end-users. For instance, showstopper bugs are critical, while if there is an easy and intuitive workaround like reloading the page, the correct severity will be low.

The Attachments in Bug Report

Testlio allows testers to attach relevant files to their bug reports, such as screenshots, screencast, and log files (crash, console, and sometimes Charles log files).

Log files are mandatory, and functional bugs must provide two different attachments.

These attachments must have a meaningful name to help the developer who downloads it diagnose the issue.

While a screenshot or a screencast can easily document visual bugs, functional bugs require two attachments per submitted issue.


  • The video recording must not contain any personal or sensitive data, including data from other workspaces.

  • The video has to be in .mp4 format and under 10MB

  • The minimum resolution of 720p

  • No background sound recorded if not needed

  • Only include important reproducing steps

  • For iOS 11 and above, the iPhone's own screen recorder should be used

All issues must have a screenshot of the page (even if the tester has attached a screencast), and a text file with the HTML code snippet of the issue must be added. Sometimes, the information in the ❝Attachments in Bug Report❞ section is enough.

Test IO testers can include attachments, such as screenshots, screencasts, and crash logs.


  • Highlight

  • JPG or PNG format.

  • We recommend not using hand drawing but shapes like squares, rectangles and arrows.

  • Capture the complete screen.

  • Unnecessary tabs or applications shouldn't be seen, but more importantly, Test IO customers' information must not be visible (e.g. notifications with a test cycle ID and customer's name)

  • The orientation shown in the screenshot should be the one user while testing.


  • No noise

  • Entire screen

  • The length for bug reports is 60 seconds, while for User Stories and reproductions, it is 15 seconds.

  • The steps recorded must only be the last navigational step, the action that triggers the bug, and the bug itself.

  • Only mp4 format

  • Tap/touches/clicks must be visible on desktop and Android devices.

For performance testing, .har files are required.

Test Communication

Testlio fosters communication between testers and test leads via the platform's Chat or the RocketChat channel for the specific workspace; however, testers are added to this Chat only if they've passed a certification test or it's required for the workspace.

Email is used to send testers improvement requests.

Test IO emphasises real-time communication, enabling testers to collaborate and discuss issues through test chat, bug report comments, Discord server, and email platforms between testers, team leaders (TL), Customer Success Managers (CSMs), and customers. Testers can ask questions, seek clarifications, and receive instant feedback from the abovementioned stakeholders, as these can request information from testers' tasks.

Bug payout

Testlio pays testers by the hour, not by the bug.

Test IO payout depends on the type of task performed. There are tasks directly related to Test Cycles. In contrast, others, like Reproductions, Bug Fix Confirmation and Bug Report Confirmation, can be performed without joining the test cycle the report belongs to.

However, in a test cycle, the payout depends on the type and severity of the bug, as well as device specifications.

Paid Tasks & Bonuses

At Testlio, freelance testers get paid according to the hours assigned to execute specific tasks within a run in a particular project.

At Test IO, freelance testers can perform these paid tasks and receive the following bonuses:

  • Bug reporting

  • Bug reproduction

  • User Stories

  • Test cases

  • Bug fix confirmation

  • Bug Report confirmation

  • Participation in special projects bonus

  • Paid Activity Sessions

  • Purchase Reports

  • Test Feedback (paid in some rare cases)

  • Bug Like Bonus

Instead of waiting for the client to accept or reject your work at Test IO, you're being paid as soon as the Team Leader takes your bug or execution.

In conclusion, each platform has its unique approach to test processes, bug reporting, bug severity, communication, and payment structures.

Testlio and Test IO differ in several aspects. Testlio follows a structured test process, assigning testers to workspaces for various testing types and allowing for bug submissions and disputes. Test IO employs an agile process with real-time exploratory testing, requiring testers to acknowledge instructions for any test cycle they're eligible to join based on their languages, devices and location. Bug forms in Testlio require detailed information that testers must provide, while Test IO simplifies the bug form. Testers can attach various files in Testlio, with mandatory log files for functional bugs, whereas Test IO offers general attachment guidelines for screenshots and screencasts. Bug severity varies between three levels in Testlio, determined by testers, while Test IO provides specific scenarios for bug severity selection. Communication differs, with Testlio using chats and emails, while Test IO emphasises real-time communication through various channels. Payment structures also vary, as Testlio pays testers hourly, and Test IO's payout depends on the many task types testers can perform on our platform and bug type and severity.

Testers are the backbone of software quality and end-user satisfaction, and their tireless efforts ensure that software is reliable and seamless across all platforms.

Did this answer your question?