Streaming Devices are small dedicated gadgets built so users can stream their favourite content such as series, movies, music and all types of Live shows. Since they are dedicated to this task, they'll usually come with a Remote Control along with other accessories and features to improve the viewer experience. Most streaming devices available on the market run an operating system developed by their team which allows users to install and remove apps as you do with your mobile or your desktop devices.
Our customers will look forward to testing the apps they developed for such devices if you have one. This article is here to help you understand the best practices when you are testing using streaming devices and how to avoid common mistakes.
How can I be invited to an exploratory cycle for streaming devices' apps?
You are not allowed to test any app using emulated devices, so you must own a real-world one in order to start testing. Once you have it, you must add the device to your profile to receive invites for the cycles. You can learn more about how to add a new device to your Test IO account here.
Note: You must be careful to select the exact device version on the list and the OS to be invited to cycles. If the list does not contain your device or if the most updated OS version is not available to be selected, please contact our team so we can look into it.
Best Practices
We gathered some important information so you can understand better the particularities of testing with streaming devices.
Make sure you are testing in a calm and silent environment. The audio while streaming is really important and a lot of bugs can be found if you pay attention to it. Also, remember to record the sound if you find any bug related to it.
Trick play while streaming is your best friend to find bugs. That means the actions you usually do when you are watching a video (such as pausing, playing, changing the subtitles, fast forwarding, rewinding, scrubbing the progress bar and other available actions depending on your device). It's always important to mimic regular user behaviour to find the most relevant bugs for our customers.
Avoid forced behaviours while testing using streaming devices. Fast-paced clicking on the remote buttons for example is not a regular action that users are expected to do. This action will end up triggering unexpected behaviours on the device that are usually not relevant to our customers.
Always have a reliable VPN service to use while testing. Most of the time, customers will have to restrict their access to the available content on the apps to certain locations due to legal/contract reasons. So even if you are testing a staging app, a VPN to a certain country might be required to see the content correctly on the customer apps. So before starting the test session, it's good to search for a good VPN service that works with your device.
Add all of your available streaming devices to your profile. The more devices you have added to your profile means your chances to get an invitation to test using one of them will be higher. So if you have easy access to one of the streaming devices (either if it's yours or if it belongs to another house member who authorizes you to use it for testing purposes), you can always add it to your profile.
But remember, sharing the customer information with others is not allowed in any of the Test IO cycles, so please only borrow a device if you can test it in a safe environment.
Always remove the testing app and credentials (if given) from your device once you've finished testing. Sometimes our customers will provide you premium accounts to test the device as a regular paid user would. But the credentials are meant to be used only while testing and should not be used for personal purposes. If you are caught using the app outside the test window or sharing the credentials with other people, you will be banned from our platform.
Keep in mind the test scope and feature descriptions. Streaming app developers might be interested in exploratory testing covering the entire app or only specific features and areas. Such information will usually be clear in the test description, but sometimes it can be hard to identify the correct areas that are within the scope of the cycle. If you find any problems or if you are not sure about something, please ask for the TL's help using the Chat tab on your cycle instead of making assumptions.
The content of the app (the video assets) may not be controlled by our customers. Sometimes, streaming app developers are only focusing on the app itself and the films/tv series/shows available list is controlled by another team (or even other companies). In such situations, the customer devs might not be able to fix any wrong content (e.g. missing a series season or a movie with an incorrect description). So to avoid such bug reports, the Content bug severity may be disabled in your cycle. However, even if the Content bugs are allowed, we recommend you check the instructions to see if there are any notes about which Content bugs can be reported.
Examples of bugs that are usually relevant to our customers
To help and inspire you to find bugs, here are some examples of issues that our customers would be interested in:
The subtitles are not in sync with the video or don't work at all when available
An error is shown when a video is played
The app crashes when you perform any regular action (e.g. pausing a video, pressing the back button on the remote, fast forwarding the playback)
Video is not added to the favorite list (if applicable)
Video is not added to the "recently played" list (if applicable)
Ads won't play correctly (if applicable)