Capsules and Configurations Design

Overview

Capsules and Configurations are core components of the Roboticks deployment pipeline: Compositions (Docker) → Capsules (Apps) → Configurations (Settings)

Capsules: Versioned application packages (.tar.gz, .zip) that run inside Docker compositions
Configurations: Site/device-specific settings tied to capsules
Deployment Packages: Complete deployments linking composition + capsule + configuration

Use Cases

Development Workflow: Developers upload new capsule versions with different features/fixes
Multi-Environment: Same capsule with different configurations (dev/staging/prod)
Fleet Management: Deploy specific capsule+config combinations to device fleets
Simulation: Run capsules in compositions for testing before device deployment

Architecture

Database Schema

Capsules Table

class Capsule(Base):
    id: int (PK)
    project_id: int (FK to projects)
    name: str (required, unique per project)
    version: str (required, semver format)
    description: str (optional)
    capsule_type: enum ['SW', 'FW', 'OS']

    # File storage
    s3_path: str (unique)
    file_size_bytes: int
    checksum_sha256: str
    file_format: enum ['.tar.gz', '.zip', '.tar', '.tgz']

    # Metadata
    metadata_json: JSON (custom fields)
    dependencies: JSON (array of {name, version_constraint})

    # Versioning
    is_latest: bool (only one per name)
    version_major: int
    version_minor: int
    version_patch: int

    # Validation/Testing
    validation_status: enum ['pending', 'passed', 'failed']
    validation_notes: str

    # Timestamps
    created_at: datetime
    updated_at: datetime

    # Relationships
    configurations: List[Configuration]
    deployment_packages: List[DeploymentPackage]

Unique Constraints:

(project_id, name, version) - No duplicate versions
(project_id, s3_path) - Unique storage paths

Indexes:

(project_id, name, is_latest) - Fast lookup of latest version
(project_id, capsule_type) - Filter by type
(validation_status) - Find unvalidated capsules

Configurations Table

class Configuration(Base):
    id: int (PK)
    project_id: int (FK to projects)
    capsule_id: int (FK to capsules)

    name: str (required, unique per project)
    description: str (optional)
    environment: str (e.g., 'dev', 'staging', 'prod')

    # Versioning
    version: str (semver)
    compatible_capsule_versions: str (semver constraint, e.g., "^1.2.0")

    # File storage
    s3_path: str (unique)
    file_size_bytes: int
    checksum_sha256: str
    file_format: enum ['.json', '.yaml', '.ini', '.env', '.toml']

    # Inheritance
    parent_config_id: int (FK to configurations, nullable)
    override_fields: JSON (fields that override parent)

    # Configuration content
    config_data: JSON (parsed configuration)

    # Validation
    validation_status: enum ['pending', 'passed', 'failed']
    validation_schema: JSON (JSON schema for validation)

    # Timestamps
    created_at: datetime
    updated_at: datetime

    # Relationships
    capsule: Capsule
    parent_config: Configuration (self-reference)
    deployment_packages: List[DeploymentPackage]

Unique Constraints:

(project_id, name) - Unique config names per project
(project_id, s3_path) - Unique storage paths

Indexes:

(project_id, capsule_id) - Find configs for a capsule
(project_id, environment) - Filter by environment
(parent_config_id) - Find child configs

DeploymentPackages Table

class DeploymentPackage(Base):
    id: int (PK)
    project_id: int (FK to projects)

    name: str (required)
    description: str (optional)

    # Components
    composition_tag: str (Docker image tag)
    capsule_id: int (FK to capsules)
    configuration_id: int (FK to configurations, nullable)

    # Deployment metadata
    deployment_status: enum ['draft', 'active', 'deprecated']
    deployed_at: datetime (nullable)
    deprecated_at: datetime (nullable)

    # Deployment targets
    target_devices: JSON (array of device IDs or fleet filters)

    # Timestamps
    created_at: datetime
    updated_at: datetime

    # Relationships
    capsule: Capsule
    configuration: Configuration

Indexes:

(project_id, deployment_status) - Find active deployments
(capsule_id) - Find deployments using a capsule
(configuration_id) - Find deployments using a config

S3 Storage Structure

s3://roboticks-storage/
└── capsules/
    └── {org_slug}/
        └── {project_slug}/
            └── {capsule_name}/
                ├── {version}/
                │   ├── roboticks-capsule.tar.gz
                │   └── metadata.json
                └── latest/  (symlink or copy)
                    ├── roboticks-capsule.tar.gz
                    └── metadata.json

└── configurations/
    └── {org_slug}/
        └── {project_slug}/
            └── {config_name}/
                ├── {version}/
                │   └── config.yaml
                └── latest/
                    └── config.yaml

API Authentication

Project Secret Authentication:

Each project has a project_secret (generated on creation)
Capsule/config uploads use X-Project-Secret header
Download/list operations use standard JWT tokens

Semantic Versioning

Version Format: MAJOR.MINOR.PATCH Wildcard Support:

1.2.3 - Exact version
1.2.* - Any patch version in 1.2
1.* - Any minor/patch in major 1
^1.2.0 - Compatible versions (>= 1.2.0, < 2.0.0)
~1.2.0 - Approximately equivalent (>= 1.2.0, < 1.3.0)

Enhanced Features

1. Capsule Dependencies

{
  "dependencies": [
    {"name": "vision-core", "version": "^2.1.0"},
    {"name": "sensor-driver", "version": "1.5.*"}
  ]
}

2. Configuration Inheritance

# base-config.yaml (parent)
database:
  host: localhost
  port: 5432

# prod-config.yaml (child)
# Inherits database settings, overrides host
database:
  host: prod.example.com

3. Validation Tracking

Upload triggers validation
Store validation status (pending/passed/failed)
Block deployments of failed capsules/configs

4. Deployment History

Track which capsule+config was deployed when
Support rollback to previous deployment
Audit trail of all deployments

API Endpoints

Capsules

Upload Capsule (with presigned URL):

POST /api/v1/projects/{org_slug}/{project_slug}/capsules/upload-url
Headers: X-Project-Secret: {secret}
Body: {
  "name": "my-app",
  "version": "1.2.3",
  "description": "My app v1.2.3",
  "capsule_type": "SW",
  "file_format": ".tar.gz",
  "file_size_bytes": 1234567,
  "metadata": {...},
  "dependencies": [...]
}
Response: {
  "capsule_id": 123,
  "upload_url": "https://s3.../presigned-url",
  "expires_in": 3600
}

Confirm Upload:

POST /api/v1/projects/{org_slug}/{project_slug}/capsules/{capsule_id}/confirm
Headers: X-Project-Secret: {secret}
Body: {
  "checksum_sha256": "abc123..."
}

List Capsules:

GET /api/v1/projects/{org_slug}/{project_slug}/capsules
Query: ?name={name}&type={SW|FW|OS}&latest_only=true
Response: [{
  "id": 123,
  "name": "my-app",
  "version": "1.2.3",
  "capsule_type": "SW",
  "file_size_bytes": 1234567,
  "is_latest": true,
  "validation_status": "passed",
  "created_at": "2025-11-13T10:00:00Z"
}]

Get Capsule Details:

GET /api/v1/projects/{org_slug}/{project_slug}/capsules/{capsule_id}
Response: {
  "id": 123,
  "name": "my-app",
  "version": "1.2.3",
  "all_versions": ["1.2.3", "1.2.2", "1.2.1"],
  "dependencies": [...],
  "metadata": {...},
  "download_url": "https://s3.../presigned-download-url"
}

Delete Capsule Version:

DELETE /api/v1/projects/{org_slug}/{project_slug}/capsules/{capsule_id}

Configurations

Upload Configuration:

POST /api/v1/projects/{org_slug}/{project_slug}/configurations/upload-url
Headers: X-Project-Secret: {secret}
Body: {
  "name": "prod-config",
  "capsule_id": 123,
  "version": "1.0.0",
  "environment": "production",
  "compatible_capsule_versions": "^1.2.0",
  "file_format": ".yaml",
  "parent_config_id": null
}

List Configurations:

GET /api/v1/projects/{org_slug}/{project_slug}/configurations
Query: ?capsule_id={id}&environment={env}

Get Configuration Details:

GET /api/v1/projects/{org_slug}/{project_slug}/configurations/{config_id}

Deployment Packages

Create Deployment Package:

POST /api/v1/projects/{org_slug}/{project_slug}/deployments
Body: {
  "name": "prod-deployment-v1",
  "composition_tag": "latest",
  "capsule_id": 123,
  "configuration_id": 456,
  "target_devices": ["device-1", "device-2"]
}

List Deployment Packages:

GET /api/v1/projects/{org_slug}/{project_slug}/deployments

Implementation Phases

Phase 1: Backend - Database Models & Migration ✅ COMPLETED

✅ Created CapsuleArchive model (versioned app binaries with dependencies, validation)
✅ Created Configuration model (settings with inheritance, capsule linking)
✅ Renamed Capsule → Deployment (deployment tracking and state management)
✅ Updated all relationships (Project, FleetDevice, Session)
✅ Created comprehensive Alembic migration (20251113_refactor_deployment_pipeline.py)

Phase 2: Backend - API Implementation (Current Priority)

⏳ S3 service for presigned URLs and storage management
⏳ Project secret generation and authentication middleware
⏳ Pydantic schemas (CapsuleArchiveCreate, ConfigurationCreate, etc.)
⏳ CapsuleArchive API endpoints (upload-url, confirm, list, get, delete)
⏳ Configuration API endpoints with inheritance resolution
⏳ Dependency resolution logic for capsule archives

Phase 2: Backend - Configurations

Configuration database model
Configuration API endpoints
Inheritance resolution logic
Validation schema support

Phase 3: Frontend - Capsules & Configurations

Capsules page (list, upload, delete)
Capsule details modal (all versions, metadata)
Configurations page
Upload wizards with drag-and-drop

Phase 4: Deployment Packages

DeploymentPackage model
Deployment API endpoints
Frontend Deployments page
Device fleet integration

Phase 5: Stats & Limits Integration

Add capsule_storage_bytes to ProjectStats
Add configuration_storage_bytes to ProjectStats
Update hourly-stats-updater Lambda
Update frontend UsageLimitsWidget
Enforce storage limits on upload

Storage Limits

Per Organization Tier:

Free: 1 GB total storage (Docker + Capsules + Configs)
Professional: 50 GB
Enterprise: 500 GB

Enforcement:

Check total storage before accepting uploads
Return 413 Payload Too Large if exceeds limit
Show storage breakdown in dashboard

Security Considerations

Project Secret: Generated on project creation, stored hashed in database
Presigned URLs: Short expiration (1 hour for uploads, 15 min for downloads)
Checksums: Verify SHA256 after upload to detect tampering/corruption
Access Control: Only project members can list/download capsules/configs
Rate Limiting: Limit upload frequency to prevent abuse

Future Enhancements

Capsule Scanning: Security vulnerability scanning for uploaded binaries
Configuration Encryption: Encrypt sensitive configs at rest
Rollback Automation: Automatic rollback on deployment failure
Version Comparison: Diff tool to compare capsule/config versions
Dependency Resolution: Auto-download required dependency capsules
Build Integration: CI/CD webhooks to auto-upload capsules on successful builds

Getting Started

Core Concepts

Features

SDK Documentation

Capsules

Capsules and Configurations Design

Overview

Use Cases

Architecture

Database Schema

Capsules Table

Configurations Table

DeploymentPackages Table

S3 Storage Structure

API Authentication

Semantic Versioning

Enhanced Features

1. Capsule Dependencies

2. Configuration Inheritance

3. Validation Tracking

4. Deployment History

API Endpoints

Capsules

Configurations

Deployment Packages

Implementation Phases

Phase 1: Backend - Database Models & Migration ✅ COMPLETED

Phase 2: Backend - API Implementation (Current Priority)

Phase 2: Backend - Configurations

Phase 3: Frontend - Capsules & Configurations

Phase 4: Deployment Packages

Phase 5: Stats & Limits Integration

Storage Limits

Security Considerations

Future Enhancements

Getting Started

Core Concepts

Features

SDK Documentation

​Capsules and Configurations Design

​Overview

​Use Cases

​Architecture

​Database Schema

​Capsules Table

​Configurations Table

​DeploymentPackages Table

​S3 Storage Structure

​API Authentication

​Semantic Versioning

​Enhanced Features

​1. Capsule Dependencies

​2. Configuration Inheritance

​3. Validation Tracking

​4. Deployment History

​API Endpoints

​Capsules

​Configurations

​Deployment Packages

​Implementation Phases

​Phase 1: Backend - Database Models & Migration ✅ COMPLETED

​Phase 2: Backend - API Implementation (Current Priority)

​Phase 2: Backend - Configurations

​Phase 3: Frontend - Capsules & Configurations

​Phase 4: Deployment Packages

​Phase 5: Stats & Limits Integration

​Storage Limits

​Security Considerations

​Future Enhancements

Capsules and Configurations Design

Overview

Use Cases

Architecture

Database Schema

Capsules Table

Configurations Table

DeploymentPackages Table

S3 Storage Structure

API Authentication

Semantic Versioning

Enhanced Features

1. Capsule Dependencies

2. Configuration Inheritance

3. Validation Tracking

4. Deployment History

API Endpoints

Capsules

Configurations

Deployment Packages

Implementation Phases

Phase 1: Backend - Database Models & Migration ✅ COMPLETED

Phase 2: Backend - API Implementation (Current Priority)

Phase 2: Backend - Configurations

Phase 3: Frontend - Capsules & Configurations

Phase 4: Deployment Packages

Phase 5: Stats & Limits Integration

Storage Limits

Security Considerations

Future Enhancements