Skip to main content

​
Coverage

Coverage in Roboticks is requirement coverage, not source-line coverage. The question is “is every requirement confirmed by a passing test on this SHA?”, not “how many source lines did the test suite touch?”. Both are useful; only the first is what an auditor asks about.

​
The state machine

Every requirement is in exactly one of five states per SHA: State is computed deterministically — same inputs, same SHA, same answer. The audit pack includes a re-computation script that proves it.

​
Calculation inputs

For a given SHA, the inputs are:
  1. The requirement set in scope (the current roboticks/requirements.yaml or the snapshot for a release SHA).
  2. The RequirementLink rows in scope (the union of source-code-derived, manual, and accepted LLM links).
  3. The JUnit XML output of the test run on this SHA.
  4. The prior coverage state on the parent branch’s last successful run (for regression detection).
There are no inputs from outside these four buckets — coverage is reproducible from artifacts.

​
Examples

​
Single requirement, single test

​
Single requirement, two tests

​
Requirement that has never been confirmed

regression is reserved for requirements that were once confirmed. A first-time partial is just partial — no regression alarm.

​
Regression detection

Regression detection runs against the parent branch’s last successful Roboticks run, defined as:
  • For a PR, the base branch.
  • For a push to a release branch, the prior commit on the same branch.
  • For the first run on a new branch, the inherited state from the branch’s fork point.
If a requirement was confirmed in that reference run and is partial or has a failing confirming test in the current run, it becomes regression. This is the strongest negative signal Roboticks emits — it shows up red in the Check Run and the matrix, and is called out separately in the summary.

​
Hierarchy rollup

Parent requirements (those with descendants) get their state from their children: Verification methods other than test (i.e. inspection, analysis, demonstration) are excluded from rollup — they don’t drag a parent down to gap just because Roboticks can’t verify them by test. See Hierarchy.

​
Multi-repo rollup

A Roboticks project can host requirements that are confirmed by tests in multiple repos. The rollup is the same — a requirement is confirmed if any confirming test in any linked repo passes on the current SHA of its repo. A push to one repo only re-runs its tests; results from the other repos are read from their last-known-good run. See Multi-repo projects for the freshness rules.

​
Coverage percentage

The dashboard summary shows a per-requirement-type percentage:
The denominator includes only requirements with verification_method: test. Inspection / analysis / demonstration requirements are reported separately as out-of-band verification. A weighted variant — ”% of ASIL-D requirements confirmed” — drives some auditor-facing dashboards. See Standards overview for the integration.

​
Live vs snapshot coverage

The matrix has a snapshot selector. The numbers it shows depend on which mode: This is what makes the evidence pack reliable — the snapshot’s coverage numbers don’t drift between when you ship and when the auditor reads them three years later.

​
Where coverage updates trigger

​
Re-computing coverage

For investigation, the CLI can recompute coverage from artifacts:
This is a read-only operation — it doesn’t mutate the database — and is mostly useful for sanity-checking the platform against a custom expectation in an audit.

​
Next

Matrix

The UI that renders these states.

Gaps

What to do with a gap.

Change-impact

Affected-requirements detection per PR.

Snapshotting

Per-release coverage immutability.