Test runs

A run is one launch of tests. The AI agent carries out each execution in a real browser, and verdicts and evidence arrive while you watch.

Ways to start a run

From	How
A test plan	Run in the plan's launch bar, or Run on the plans list
A quick run	Run in the case editor, or Run selected on ticked cases. Pick profiles and an environment, then Run. The run is named "Quick run · …".
A schedule	The plan's schedule starts it automatically
Qalion AI	Qalion AI proposes a run on a card, and it starts when you approve

Runs are numbered per project (RUN-1000, RUN-1001…). Each run contains executions: one per case, browser profile and dataset row.

Statuses and verdicts

Executions

Verdict	Meaning
Passed	The agent found every expected result.
Failed	The agent judged the app wrong here.
Error	The agent never gave a verdict, so this is not a failure of your app. A reason says why.
Skipped	Not executed, because the run was stopped.
Queued, Running	Waiting for a browser session, or in progress.

Runs

A run is Pending or Queued, then Running. When it ends it is Aborted if someone stopped it, Error if any execution errored, Failed if any execution failed, and Completed otherwise.

Error reasons

Reason	What happened
Timed out	The execution went over the project's execution time limit.
Agent stuck	The agent couldn't interact with an element, for example on a given step.
Site unreachable	The agent couldn't reach the site.
Cancelled	The run was stopped before the execution finished.
Platform issue	A problem on Qalion's side. Re-run it.

Errors are not verdicts: pass rates leave them out.

The run page

The run page updates live. If the live connection drops, it shows "Live updates paused — refreshing every few seconds." The header shows the run's cost ("… spent", with the number of model calls).

Switch between views of the run:

All executions, with filters and sorting (failures first, slowest…).
Failures by root cause: failures grouped by problem.
Changes vs a previous run: what is new and what got fixed.
Matrix: cases × browser profiles.
Activity: comments from your team.

The RUN-1001 run page: plan, environment, duration and cost in the header, a progress bar with 2 failed and 10 passed, and the list of executions with their browser, steps, cost and start time — A finished run: 10 passed, 2 failed. Failures are listed first, each with the step that failed.

Evidence

Click an execution to open its evidence panel. It has five tabs:

Tab	What it shows
Steps	Each step with its verdict, Expected and Observed, and a screenshot after the step. While the execution runs, a live frame ("Live · …").
Agent	The agent's action log: what it clicked, typed and checked.
Console	The browser console, with a count of warnings.
Network	The requests the page made.
Video	A recording of the execution.

The evidence of a failed execution in full view: the failure reason at step 3, the screenshot the agent took showing the same picture on every product, the step thumbnails with their verdicts, and the Steps tab with Expected and Observed for each step — Evidence of a failed execution: the agent saw the same picture on every product, so it failed step 3.

From an execution you can also Report to Jira, mark it as flaky to quarantine it (see Insights) or mark its problem as a known issue (see Problems).

Re-run

Qalion doesn't retry failed executions automatically. You re-run them yourself, inside the same run, once it has finished:

The Re-run button in the run header (for example Re-run 2 failures). Its arrow menu offers Re-run failed, Re-run errors or Re-run all.
On one execution: Re-run this, or Re-run on a fresh worker.
Each re-run is a new attempt. The latest attempt counts; earlier ones stay available as evidence.

Stop a run

Click Stop run on the run page, or Cancel run on the runs list. The run ends Aborted. Executions still queued or running end as Skipped, with the reason Cancelled. Finished results are kept.

The runs list

Tabs: All runs, Needs attention, My runs, Scheduled and CI.
Search runs, plans and people. Filter by Plan, Environment, Status, Date, Trigger, Reason and Target.
Run a plan takes you to the plans. Compare runs compares two runs.
Bulk actions on ticked runs: Re-run failures, Compare (two runs), Export JUnit, Export CSV and Delete.

The Test runs page: runs, pass rate, needs attention and median duration at the top, the All runs, Needs attention, My runs, Scheduled and CI tabs, and two runs of the Release 2.4 regression plan with their outcome bars and problems — The runs list, with the problems each run hit.

Warning

Deleting a run is permanent. Runs don't go to the Trash.

Downloads and exports

Where	What you get
Run page → Download	PDF report (verdicts, failures, screenshots), JUnit XML (for CI dashboards), Evidence bundle (.zip with screenshots, logs and videos; videos over the size limit are left out)
One execution	Download this execution as PDF
Runs list, bulk	JUnit XML and CSV for the ticked runs

Was this page helpful?

Test plans

How the AI agent runs tests