Testing¶

Midair runs test suites that are built on the Test Environment (TE). It starts the suite on a node, it brings the log back, and it turns the log into a result tree.

The Testing section of the sidebar has four screens:

Runs: The test runs and the test templates.
Templates: The templates only.
Test suites: The test suite records and the suite generator.
Distributions: The test environment builds. See Distributions.

Templates¶

A template describes what to run. Open Templates, and add a record.

Name: The name that the lists show.
Workspace: The workspace that the template belongs to. The workspace supplies the repositories, the run command and the node. See Workspaces.
Distribution: The test environment build that the run is built against.
Run parameters: Text that Midair passes to the suite runner, for example --tester-run=my-ts/wifi. A run can replace it.
Suite branch: A branch, a tag or a commit of the test suite. An empty field means the branch of the workspace. A run can replace it.
Notes: Free text.
Planned runs: The schedule. See Planned runs.

Save stores the template. Save and run stores it and starts a run.

Runs¶

A start makes a new run record, and the template stays as it was. The run carries the configuration that it started with, so a later edit of the template does not change what the run did.

A run record shows two tabs:

Run: The run parameters of this run.
Import: A raw log file to attach. Midair processes an imported log in the same way as a log that it collected itself. Use this to read a log that another system produced.

Revision overrides¶

A run can pin each of the three checkouts of the workspace without a change to the workspace configuration:

the test suite;
the code under test;
the test environment.

Each override accepts a branch, a tag or a commit. Midair checks out a tag or a commit in the detached state.

This is what an assistant or a build job uses to test a branch: start a run that names the branch, read the failures, fix them, and start the next run. See MCP endpoint.

The run screen¶

The run screen shows the state, the log and the results.

Midair reads the log while the run is in progress. It polls the run every second at the start, and it slows down to every ten seconds after about three minutes.

Result tree¶

When the run ends, Midair converts the raw TE log into text. It uses the command from the workspace for that. It then reads the MI records in the text.

From those records it builds the result tree:

a package and a session add one level of nesting;
a test is a leaf;
each entry carries an obtained verdict and an expected verdict;
the printouts between the records stay attached to the test that produced them.

A test passes when the obtained verdict is the expected verdict. The failures list holds every test that did not pass, and every test whose verdict differs from the expected one. Each entry names the path of the test, its parameters, the two verdicts, the error lines, and the identifier of its node in the tree.

Log pages¶

A full TE log is too large to read at once. Midair serves it in pages. A page request bounds the log by record number, or by time in the HH:MM:SS.mmm form, or by a node of the result tree. Each answer carries a cursor for the next page.

AI review¶

Midair can send the processed log to an AI provider and keep two results with the run:

a timeline of what happened during the run;
a review, which is a short engineering assessment.

The review is empty until an analysis of the log has run. Start it from the run screen, or with the API.

The run screen also accepts a question about the log. Midair sends the question with the log and keeps the answer on the run record.

Set the provider under Settings, then Integrations, then AI. See AI providers.

Bublik¶

Midair can publish a finished run to a Bublik log server. Configure the connection under Settings, then Integrations, then Bublik. Midair copies the raw log, the text log and the metadata to the incoming directory of the log server. It then records the Bublik run identifier on the run.

Planned runs¶

A template can start itself. A rule is a set of weekdays and a set of times:

[
  {"enabled": true, "days": ["mon","tue","wed","thu","fri"],
   "times": ["02:00"], "label": "Nightly"},
  {"enabled": true, "days": [], "times": ["09:00","18:00"]}
]

An empty list of days means every day. The rules use the local clock of the server, and Midair decides once per minute whether a rule fires. A restart or a slow tick cannot produce a duplicate run.

A rule holds a label, and a disabled rule stays in the list without firing.

The rules cannot express a repetition inside one hour, and they cannot express a day of the month.

Test suites¶

The Test suites screen holds the records that the suite generator works on. A record names two repositories:

Source data: An optional read-only repository. The generator reads it as the source of the data for the update.
Suite repository: The test suite itself. The generator clones it, changes it in place, and pushes the changes back to the branch that the record names.

The Run screen of a test suite asks for a spec prompt and for a code generation tool. The prompt describes the change to apply to the suite. Midair sends it to Claude Code or to the Codex CLI, together with the context of the TE framework. The screen shows the output of the tool while it runs.

The tool that you select here replaces the global AI provider for this generation.

An attack vector drives the same generator on a loop: it passes a spec of its own, runs the result, reads the verdicts, and writes the next spec from what the round could not settle. The suite’s own spec prompt is left alone.