Skip to main content

​
Packaging Tests

This guide explains how to package your tests for upload to the Roboticks platform using the package_tests.sh script.

​
Prerequisites

Before packaging, ensure you have:
  1. Test files in a tests/ directory (or custom location)
  2. roboticks-tests submodule initialized (for the test framework)
# Initialize submodule if needed
git submodule update --init submodules/roboticks-tests

​
Using package_tests.sh

The tools/package_tests.sh script packages your tests with the test framework:

​
Basic Usage

./tools/package_tests.sh
This creates build/test-package.tar.gz with default settings.

​
With Options

./tools/package_tests.sh \
    --name my-integration-tests \
    --version 1.0.0 \
    --tests-dir tests/ \
    --output build/my-tests.tar.gz

​
Available Options

OptionDescriptionDefault
--name NAMEPackage namemy-tests
--version VERSIONVersion string1.0.0
--tests-dir DIRTests directorytests/
--output FILEOutput tarball pathbuild/test-package.tar.gz
--helpShow help message-

​
Package Contents

The generated tarball contains: 
test-package.tar.gz
|-- test-framework/                    # From roboticks-tests repository
|   |-- __init__.py
|   |-- runner.py                      # Test orchestrator (entry point)
|   |-- zmq_monitor.py                 # ZMQ message capture
|   +-- pytest_roboticks_progress.py   # Progress reporting plugin
|-- tests/                             # Your test files
|   |-- conftest.py
|   +-- test_*.py
+-- VERSION                            # Package metadata

​
VERSION File

The VERSION file contains metadata: 
name=my-integration-tests
version=1.0.0
created=2024-01-15T10:30:00Z
commit=abc123f
branch=main

​
Uploading to Platform

After packaging, upload using the CLI: 
roboticks test-package push \
    --file build/test-package.tar.gz \
    --name my-integration-tests \
    --version 1.0.0 \
    --description "Integration tests for HelloWorld module"

​
CLI Options

OptionDescriptionRequired
--file FILEPath to tarballYes
--name NAMEPackage nameYes
--version VERSIONVersion stringYes
--description DESCDescriptionNo

​
Directory Structure Requirements

For the test framework to work correctly, your tests must follow this structure when extracted: 
/opt/roboticks/
|-- test-framework/
|   +-- runner.py              # REQUIRED for test-framework mode
|-- tests/
|   +-- test_*.py              # Your test files
+-- ...
The package_tests.sh script handles this automatically. It copies files from the roboticks-tests submodule into test-framework/ and your tests into tests/, creating the correct structure. You don’t need to manually organize these directories.
The test runner detects test-framework mode by checking for test-framework/runner.py. If you’re packaging manually, ensure this file is included.

​
Custom Packaging

If you need more control, you can create packages manually:

​
Manual Packaging

# Create temp directory
PKG_DIR=$(mktemp -d)

# Copy test framework
cp -r submodules/roboticks-tests/src/roboticks_tests/* "$PKG_DIR/test-framework/"

# Copy your tests
cp -r tests/* "$PKG_DIR/tests/"

# Create tarball
cd "$PKG_DIR"
tar -czf /path/to/output.tar.gz .

​
Without Test Framework (Direct Mode)

If you don’t need the test framework orchestrator, you can package just your tests: 
# Package only tests - runner won't use test-framework mode
tar -czf test-package.tar.gz tests/
When test-framework/runner.py is absent, the test runner uses a default execution script inside the Docker container:
  1. Test runner starts the Docker container with a default entrypoint script
  2. The script starts the device manager process
  3. Device manager discovers compositions automatically from the deployment directory using its config
  4. The script executes ROBOTICKS_TEST_COMMAND (e.g., pytest tests/ -v)
  5. The script stops the device manager when tests complete
In direct mode, the device manager handles composition discovery and lifecycle itself based on its configuration. This is simpler but offers less customization than the Python-based runner.py orchestrator.Use direct mode when:
  • You have simple tests that don’t need custom orchestration
  • The device manager’s built-in composition discovery is sufficient
  • You prefer the default script to handle the device manager lifecycle

​
Versioning Strategy

We recommend semantic versioning for test packages:
  • MAJOR - Breaking changes to test structure or dependencies
  • MINOR - New tests or features
  • PATCH - Bug fixes to existing tests
Example: 
# Initial release
./tools/package_tests.sh --version 1.0.0

# Added new test file
./tools/package_tests.sh --version 1.1.0

# Fixed flaky test
./tools/package_tests.sh --version 1.1.1

​
CI/CD Integration

​
GitHub Actions Example

name: Package and Upload Tests

on:
  push:
    branches: [main]
    paths:
      - 'tests/**'

jobs:
  package:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: true

      - name: Package tests
        run: |
          ./tools/package_tests.sh \
            --name integration-tests \
            --version ${{ github.sha }}

      - name: Upload to Roboticks
        env:
          ROBOTICKS_API_KEY: ${{ secrets.ROBOTICKS_API_KEY }}
        run: |
          roboticks test-package push \
            --file build/test-package.tar.gz \
            --name integration-tests \
            --version ${{ github.sha }}

​
Troubleshooting

Ensure the roboticks-tests submodule is initialized:
git submodule update --init submodules/roboticks-tests
Verify your tests directory contains test_*.py files:
ls tests/test_*.py
Check your API key and organization permissions:
roboticks auth status
Verify test-framework/runner.py exists in the extracted package:
tar -tzf build/test-package.tar.gz | grep runner.py

​
Next Steps

Writing Tests

Learn how to write effective tests.

Self-Hosted Runners

Set up runners to execute your tests.