Skip to main content

​
CI/CD recipes

For GitHub-hosted repos, the GitHub App is the primary integration — install it once and every PR automatically gets a Roboticks Check Run. The CLI is the fallback (or the secondary surface for advanced workflows like running synthetic tests on a schedule). For non-GitHub CI systems, the CLI is the primary surface. The recipes below are the minimum working YAML/Groovy for each platform.

​
GitHub Actions

For most users, the GitHub App is enough — no workflow YAML required. See GitHub App → Installation. Use the CLI in a workflow when you need to:
  • Run on a schedule (synthetic tests)
  • Push tests for an air-gapped runner pool (see Air-gapped mode)
  • Trigger evidence-pack generation from a release workflow
# .github/workflows/release-evidence.yml
name: Release evidence pack
on:
  release:
    types: [published]

jobs:
  evidence:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - run: pipx install roboticks-cli

      # Trade GH OIDC for a CLI token — no long-lived secret needed
      - run: rbtk auth oidc-from-github

      - name: Run release test suite
        run: rbtk test run --push ./ --label release=${{ github.event.release.tag_name }}

      - name: Pull evidence pack
        run: |
          rbtk test evidence-pack latest \
            --release ${{ github.event.release.tag_name }} \
            --format all --verify \
            --out ./evidence/

      - uses: actions/upload-artifact@v4
        with:
          name: evidence-${{ github.event.release.tag_name }}
          path: ./evidence/
          retention-days: 90

​
GitLab CI

# .gitlab-ci.yml
stages: [test, evidence]

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

cache:
  paths: [.cache/pip]

.rbtk_base: &rbtk_base
  image: python:3.12
  before_script:
    - pip install roboticks-cli
  variables:
    # Mint at Settings → CI/CD → Variables (masked, protected)
    ROBOTICKS_API_KEY: $ROBOTICKS_API_KEY

test:
  <<: *rbtk_base
  stage: test
  script:
    - rbtk test run --push ./ --label branch=$CI_COMMIT_REF_NAME
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "main"'

evidence:
  <<: *rbtk_base
  stage: evidence
  script:
    - rbtk test evidence-pack latest --release $CI_COMMIT_TAG --format all --verify --out evidence/
  artifacts:
    paths: [evidence/]
    expire_in: 1 year
  rules:
    - if: '$CI_COMMIT_TAG'
GitLab CI integration is supported via the CLI. A native GitLab App equivalent to the GitHub App is on the post-MVP roadmap.

​
Jenkins

// Jenkinsfile
pipeline {
  agent { docker { image 'python:3.12' } }

  environment {
    ROBOTICKS_API_KEY = credentials('roboticks-api-key')
  }

  stages {
    stage('Install CLI') {
      steps { sh 'pip install roboticks-cli' }
    }

    stage('Test') {
      steps {
        sh 'rbtk test run --push ./ --label build=$BUILD_NUMBER --output json > result.json'
      }
      post {
        always {
          archiveArtifacts artifacts: 'result.json', fingerprint: true
        }
      }
    }

    stage('Evidence pack') {
      when { buildingTag() }
      steps {
        sh '''
          rbtk test evidence-pack latest \
            --release $TAG_NAME \
            --format all --verify \
            --out evidence/
        '''
        archiveArtifacts artifacts: 'evidence/**', fingerprint: true
      }
    }
  }
}

​
CircleCI

# .circleci/config.yml
version: 2.1

jobs:
  test:
    docker:
      - image: cimg/python:3.12
    steps:
      - checkout
      - run: pip install roboticks-cli
      - run:
          name: Roboticks test run
          command: rbtk test run --push ./ --label branch=$CIRCLE_BRANCH
          environment:
            ROBOTICKS_API_KEY: $ROBOTICKS_API_KEY

  evidence:
    docker:
      - image: cimg/python:3.12
    steps:
      - checkout
      - run: pip install roboticks-cli
      - run:
          name: Pull release evidence
          command: |
            rbtk test evidence-pack latest \
              --release $CIRCLE_TAG \
              --format all --verify \
              --out evidence/
      - store_artifacts:
          path: evidence/
          destination: evidence

workflows:
  default:
    jobs:
      - test
      - evidence:
          requires: [test]
          filters:
            tags: { only: /^v.*/ }
            branches: { ignore: /.*/ }

​
BuildKite

# .buildkite/pipeline.yml
steps:
  - label: ":robot: Roboticks test"
    command: |
      pip install roboticks-cli
      rbtk test run --push ./ --label branch=$BUILDKITE_BRANCH
    env:
      ROBOTICKS_API_KEY: ${ROBOTICKS_API_KEY}
    agents:
      queue: docker

  - label: ":scroll: Evidence pack"
    if: build.tag != null
    command: |
      pip install roboticks-cli
      rbtk test evidence-pack latest \
        --release ${BUILDKITE_TAG} \
        --format all --verify \
        --out evidence/
    artifact_paths:
      - "evidence/**/*"
    env:
      ROBOTICKS_API_KEY: ${ROBOTICKS_API_KEY}

​
Common patterns

​
Fail the build on test failure

Default behaviour. rbtk test run --wait exits non-zero (code 7) if any test fails. No extra wiring needed.

​
Don’t wait — fire and forget

RUN_ID=$(rbtk test run --push ./ --no-wait --output json | jq -r .id)
echo "Run dispatched: $RUN_ID"
# Inspect later from the dashboard or another job

​
Annotate the run with build metadata

rbtk test run --push ./ \
  --label branch=$CI_BRANCH \
  --label commit=$CI_SHA \
  --label pipeline=$CI_PIPELINE_ID
Labels are searchable in rbtk test list --label ... and surface in the dashboard’s run-history view.

​
Conditional release evidence

if [[ -n "$CI_TAG" ]]; then
  rbtk test evidence-pack latest --release "$CI_TAG" --out evidence/
fi

​
BYO ingest

If you run your own CI (your own test framework, your own container, your own runner) you can post results straight to the platform without going through a hosted or self-hosted Roboticks runner. This is the BYO ingest path. It is free on every tier — no BYO connector slot is consumed. The wire format is JUnit-with-confirms. Generate it from any pytest/colcon/launch_testing/gtest run (the SDK does this automatically), or hand-author the <property name="roboticks.confirms" ...> element if you’re plumbing a non-Roboticks test framework. 
# Push a JUnit-with-confirms XML for a synthetic test run
rbtk test ingest \
  --junit ./test_results/junit.xml \
  --branch "$CI_BRANCH" \
  --commit "$CI_SHA" \
  --label source=external-ci

# Attach MCAP bags (per-test-case) and screenshots
rbtk test ingest \
  --junit ./test_results/junit.xml \
  --attach "tests/test_estop.py::test_estop_halts_motion:mcap=./mcaps/test_estop.mcap" \
  --attach "tests/test_estop.py::test_estop_halts_motion:attachments=./screenshots/before.png"

# Or post SARIF 2.1.0 findings alongside (rides on the same free ingest)
rbtk findings upload ./sarif/codeql.sarif --tool codeql
rbtk test ingest uploads the JUnit to the per-project endpoint, parses out roboticks.confirms / roboticks.nodeid / roboticks.attach.* properties, and creates a sealed TestRun whose status reflects the JUnit outcome. Plan gate is the standard test_runs:write scope; there is no token or storage burn beyond the hot-storage already accounted for by the org’s plan. What is not free:
  • The vendor-specific catalog connectors (Jama, Polarion, codeBeamer, DOORS, LDRA, Polyspace, Coverity) at $149/connector/month — see BYO connectors.
  • AI features the ingest results feed into (triage, flakiness, sim-vs-real) — those bill in ai_tokens.

​
Secret hygiene

PlatformStorage
GitHub ActionsUse OIDC. No long-lived secret needed. Fallback: repo secret.
GitLab CISettings → CI/CD → Variables (masked, protected)
JenkinsCredentials Plugin, credentials('...') binding
CircleCIProject Settings → Environment Variables
BuildKiteAgent env, secrets manager, or the BuildKite secrets plugin
Never echo ROBOTICKS_API_KEY in build logs. Use platform-native masking; mark the variable as protected/masked at minimum.

​
Troubleshooting

Set ROBOTICKS_CI=true or run with --quiet. The CLI auto-detects non-TTY but some CI runners present a fake TTY.
The workflow’s sub claim isn’t in the policy at Settings → GitHub OIDC. Add the ref/environment, or tighten the workflow’s run conditions.
Make sure the test run completed before you ask for the pack. rbtk test run --wait (the default) blocks until completion; without --wait, you need to poll rbtk test results <id> until the status is terminal.

​
Next

GitHub App

The primary GitHub integration.

Test commands

Full reference for rbtk test.