Problems

A problem is one root cause. Qalion files every failure under a problem, so you fix causes, not individual red results.

What a problem is

When a run ends with failures, they are grouped here by the test case and the step they failed on, and kept across runs and environments. A problem belongs to the whole project. Problems is available to Owners and Admins.

How failures are grouped

Only Failed and Error results are grouped, shortly after each run ends.

Rules first. A failure joins a problem with the same test case, the same failed step and the same kind of failure (failed, timeout, agent stuck, site unreachable, cancelled, platform). The environment, the browser and the agent's wording don't matter.

AI for the rest. A failure the rules can't match is given to an AI model, which either matches it to an open problem or creates a new one with a title, description, steps to reproduce, category and confidence. When it is not sure of a match, it creates a separate problem marked "Possible duplicate of".

A title you rename yourself is never overwritten. Categories are Product bug, Test script, Test data, Environment, Flaky test and Platform; confidence is low, medium or high.

States

State	Meaning
New	The starting state.
Investigating	Someone is on it.
Known	Accepted for now. Needs a reason; optionally until a date.
Fixed	Fixed. If it fails again, it reopens.
Won't fix	Closed for good. It never reopens.

Known failures are tagged in runs and left out of new failures and notifications, but still counted in pass rates. After the Until date, a new failure reopens the problem.
A Fixed problem that fails again goes back to New with a Regressed badge.
Tabs: Open (New and Investigating) and All.
Filter by Environment, Browser and Category, or search titles and error text.

The problem panel

State, Regressed badge, Owner and linked Jira issue.
Environments with counts, and Affected browsers (detected automatically; you can edit them).
Category, confidence, first and last seen, occurrences, and a 14-day trend.
Failing step with Expected and Observed and the screenshots, Steps to reproduce, Affected test cases and Runs.

The Problems page with two open problems, and the panel of the first one: state New, owner unassigned, environments, affected browsers, first and last seen, occurrences, and the failing step with its expected and observed results and the screenshot — A problem open in its panel: where it happens, how often, and the failing step with the agent's screenshot.

AI analysis

Click Analyze with AI to investigate a problem. The AI reads the steps, the screenshots, the console and network logs and the last passing run, and cites what it found. It can take a minute.

The result gives a summary, a Likely cause, a Suggested fix, the Evidence it relied on, up to five Next steps, and a Jira draft you can copy or file with Create Jira ticket.

New problems created by the AI are analysed automatically, up to 20 per project per day. Clicks by a person are never capped.
One analysis per problem is shared with the team. If nothing changed, the saved analysis is shown again. New failures since then show "New failures since — analyze again".
Analyses use credits, and the cost of each one is shown with it.

Actions and Jira

A row's menu: Open, Rename, Copy link, Copy details, Download PDF report, and the five states.
Bulk: Change state, Assign owner, Merge (keeps one problem and moves the others' failures, flags, Jira issue and analysis into it) and Report to Jira.
Keyboard: j and k to move, F to focus, Esc to close.
Jira actions need Jira issue reporting set up under Integrations.

Was this page helpful?

Coverage

Account & security