LLM Testing at Test IO represents a specific process which encompasses several different stages:

LLM Test Invitations are sent to testers who passed the LLM Testing Course and who match other requirements (location, gender, age, device combo, etc).

Remember: The LLM Test Invitation looks like any other Test Invitation from our platform. The only differences are:

Unlike Manual Functional Testing where you test the application functionality (working as expected), Manual LLM Testing requires you to focus only on the quality and appropriateness of the LLM output. Depending on the LLM test type, your testing should be focused on:

In LLM Exploratory Testing test cycles you will need to craft your messages for LLM and validate the response quality and appropriateness while in the LLM Regression Testing, you will evaluate the responses on the messages available in the Test Cases.

While in the Manual Functional Testing, you submit bugs, in the Manual LLM Testing, you need to submit Reports. Reports in LLM Testing could be:

The difference between LLM Bug Reports and LLM Validation Reports is in the response outcome. If the LLM is showing behavior that should not be present, you should report a bug. In cases when LLM performs as expected, you should submit a Validation Report.