Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.roboticks.io/llms.txt

Use this file to discover all available pages before exploring further.

​
Change-Impact Analysis

Change-impact analysis is the layer that answers “which requirements does this PR touch?”. It runs on every PR, drives the affected-requirements table in the Check Run summary, and is the early-warning system for a regression before the failing test even runs.
Change-impact runs before the test job dispatches. The Check Run starts populated with the affected-requirements table while the runner is still spinning up — reviewers don’t have to wait for the full run to start triaging risk.

​
How it works

Three deterministic steps:
  1. File → Test — for each changed file, find every test that imports it (Python import graph, C++ #include graph, plus ROS2 package dependencies from package.xml).
  2. Test → Requirement — for each affected test, look up its RequirementLink rows (the @confirms and manual links).
  3. Affected requirement set — the union of requirements reachable via the above.
Plus one optional, flagged step:
  1. LLM-inferred extra links (opt-in) — for files that don’t have direct test coverage, ask Claude to suggest which requirements they could plausibly affect, based on the diff and the requirement texts. Every suggestion is flagged confidence: <0..1> and is never silently added to coverage — it’s surfaced in the Check Run as a “possibly affected, please review” section.

​
In the Check Run

The affected-requirements table is the heart of the Check Run summary: 
### Affected requirements

| Req | Title | Before | After | Test |
|---|---|---|---|---|
| REQ-014 | E-stop releases brake after reset | ✅ confirmed | 🔴 regression | `tests/test_estop.py::test_brake_release` |
| REQ-027 | Battery low triggers RTH | ⚪ unconfirmed | ✅ confirmed | `tests/test_battery.py::test_rth_trigger` |

### Possibly affected (LLM-inferred, please review)

| Req | Title | Confidence | Reason |
|---|---|---|---|
| REQ-101 | Perception latency p99 < 50 ms | 0.62 | Files `src/perception/pipeline.cpp` changed; no test covers it directly |
The top table is deterministic — every link there has a concrete @confirms or manual link backing it. The bottom table is inference — it exists to prompt a reviewer to either add a confirming test or add a manual link, not to count as coverage. See Check Runs for the full Check Run anatomy.

​
What “affected” means

A requirement is affected by a PR if:
  1. A test that @confirms it imports a file that the PR changes (direct).
  2. Or any test in the transitive import closure of changed files confirms it (transitive).
  3. Or — opt-in — the LLM judges that the diff plausibly affects it (LLM-inferred, flagged).
The first two are graph traversals. The third is a Bedrock Claude call with the diff, the file paths, and the candidate requirement set as context. We bound the prompt size by pre-filtering candidates with embedding similarity on the diff text vs requirement texts.

​
When LLM-inference helps

LLM-inferred links are useful when:
  • Your repo has legacy code that pre-dates @confirms adoption. The graph traversal misses it; the LLM can point at plausible requirements based on file content.
  • The PR changes a file that is currently not covered by any test. The deterministic path returns nothing; the LLM points at requirements the file might be relevant to, so a reviewer knows to write a test before merge.
  • The PR introduces a new safety-critical module with tests not yet linked. The LLM identifies the requirement family it likely belongs to.
When you accept an LLM-inferred suggestion in the dashboard, it’s promoted to a manual link with source: llm recorded in the audit trail. The accepted link now counts toward coverage on subsequent runs.

​
When LLM-inference is wrong

Often enough that you should never let it auto-merge. Common failure modes:
  • Surface-similar wording. A diff that mentions “velocity” might be linked to a velocity-related requirement that has nothing to do with the change.
  • Refactor-only diffs. Pure renames or formatter passes get flagged as touching everything; the heuristic doesn’t always strip these out.
  • Over-broad parent requirements. “AMR operates safely” matches almost everything.
We surface confidence. Sort the “possibly affected” table by confidence descending and start there; the rest is usually noise.

​
Opt-in and config

LLM-inference is on by default for paid tiers, off for Free tier. Toggle in Settings → Project → Change-impact. You can also set:
SettingDefaultEffect
min_confidence0.5Suggestions below this threshold are dropped silently.
max_suggestions10Cap on suggestions per PR — keep the Check Run readable.
include_parentsfalseWhether to include rollup parents of directly-affected requirements in the table.
inference_scopeallall, safety-only, or tagged:critical — restrict inference to a subset.
For high-stakes safety repos, many teams set inference_scope: safety-only and min_confidence: 0.7 — the suggestions are sparser but more trustworthy.

​
Impact on the test job

Change-impact does not prune the test run. We still run the full configured test suite on every PR — pruning is risky in a verification context and the time saved isn’t worth the chance of missing a regression in an unflagged-affected test. What change-impact does prune is the Check Run summary — only affected requirements show in the table, with the rest available behind a “show all” toggle. This keeps reviewer attention on the things that actually moved.

​
Re-running change-impact

When you push a new commit to the PR, change-impact re-runs against the new diff. The summary updates in place. The full history is in the run log. To re-run change-impact alone (without re-running tests): 
rbtk pr reanalyze --pr 142
Useful when you’ve edited requirement texts and want the LLM-inferred suggestions to pick up the new wording.

​
Audit trail

Change-impact reports are appended to the PR’s audit record. The evidence pack for the release that includes the PR records:
  • The diff that triggered change-impact.
  • The affected-requirement set (deterministic).
  • The LLM-inferred set, if any, and whether any suggestions were accepted.
Auditors care about this — they want to see that “when this code changed, the team knew which requirements were on the hook”.

​
Next

Check Runs

Where the affected-requirements table lands.

Coverage

How states transition when an affected requirement fails.

Gaps

Closing the gaps that change-impact surfaces.

Multi-repo

Affected-requirement detection across repos.