Functional bugs are related to the functionality of a piece of software. Examples: A button doesn’t submit the form, the search doesn’t react to the user input and the app crashes after interacting with an element on it. Every time you perform an action and the website/app doesn’t respond as it was expected, it might be a functional issue. Our limited information about our customers' products and lack of knowledge of their implementations make it difficult to determine whether an observed behaviour is intentional or actually a bug. Making educated guesses based on experience and analyzing the product behaviour by testing different scenarios can help you to come up with answers.
How to determine if app behaviour is a functional bug:
Try to figure out if a feature is designed a particular way or if it is actually broken. Test it by itself and in combination with other features to spot potential differences.
Think about what the customer’s intentions might have been and consider that the product might just work how it has been implemented.
Find evidence that something is not working as it should. Always support your claims by providing written down evidence and quality attachments.
Example: A webshop functionality works differently than in other webshops you know. That doesn’t mean the functionality is broken. The client can implement their product however they want.
Example: You claim a form field is not validated and it’s a bug. Is there any indication that they intended the field to be validated? Provide evidence by showing that the field is validated in some cases but not in others. If you don’t provide evidence, it’s an unproven claim.
A visual or content problem becomes a functional problem when it hinders a functionality and thus should be reported as a functional bug.
Does a piece of functionality consistently works the same way in different scenarios and without obvious problems? Then it’s probably intended (not a bug) and you just suggest a change (usability suggestion).
Severity Assessment
When judging the functional severity level of a bug, several factors have to be considered: the problem's functional impact, the extent of the problem, the available workarounds, if there are potential and notable losses of sales, and comparing this bug to other bugs of the same severity.
A straightforward approach is to look at the bug's functional impact. Think about how severe it is that a function is not available. Subordinate functionality will not block users from pursuing their goals – other than broken main functionality. Ask yourself how relevant that piece of functionality is in the context of the whole product.
The question of how many people, products, or items are affected by a functional problem is a determining factor for the extent of a problem. Does, for example, the “Add to Cart” button not react on all product detail pages of a webshop or only on a particular one? Is a small group of users concerned by the problem, or everyone?
Consider if you can reach your goal via an alternative route or option or if a piece of the functionality remains unavailable. When you intuitively and easily find a way around a bug, this so-called workaround enables you to still reach your goal. A bug with a simple workaround receives a lower severity level than a similar bug with a complex workaround. Finally, when there is no workaround for broken main functionality, it is a showstopper.
Estimating a potential loss of sales is a secondary approach as you often can only assume how people might react to a bug. Nevertheless, take into account how high a potential loss is. It makes a world of difference if the price of a product differs by cents or hundreds of dollars.
After all, you can also compare your bug to those that the same team leader already approved in this test in order to find out if your severity level is appropriate.
We have three severity levels for functional bugs:
LOW:
Minimal impact on the usage of the product.
The product shows unintended behaviour, but the general usage is not affected.
Few users, products, or items are concerned.
A feature/piece of functionality is broken or unavailable, but an easy workaround solves the problem.
HIGH:
Serious impact on the usage of the product, but the main functionality is intact.
A large number of users, products, or items is concerned.
Non-trivial functionality is broken or unavailable and no workaround exists.
Important functionality is broken or unavailable but a workaround exists (hence not a showstopper).
CRITICAL:
The bug prevents the core functionality of the app or website.
A showstopper prevents the user from continuing with the main process, e.g. the checkout process.
Common assessments
We have a list of cases that have fixed severity levels. The above assessment scheme does not apply to cases on that list. As the list will be updated over time, please check it regularly.
Edge case bugs
Edge case bugs occur when a piece of functionality is used in an unusual way. The functionality is not broken when used with typical data and typical user actions. Here are some examples:
Instant actions, such as minimizing an app after clicking on a button
Repeatedly doing the same thing, e.g. opening and closing menus
Any bug that only occurs after an uncommon set of actions
Each case must be evaluated separately. Edge case bugs that are relevant for our customers are forwarded as Low bugs. Irrelevant cases, which account for most edge case bugs, will be rejected.
Forced bugs
Forcing a bug by nontypical behaviour or special conditions is generally out of scope as such bugs are not relevant to our customers. Nontypical behaviour does not reflect normal user behaviour. Examples of nontypical behaviour or special conditions:
Tapping on multiple elements at the same time
Random button pressing
Fast-paced clicking on a button multiple times
Reducing the size of your window to non-typical sizes
Full RAM or internal memory leading to unexpected behaviour
Using unofficial, beta, or modified OS versions
Related articles:
Bug Report Requirements – our quality standards
Crash Logs