​

Reverse Tunnels - Deployment Guide

​

Overview

​

Architecture

Browser
  ↓
Route53 DNS (*.fleet.roboticks.io)
  ↓
CloudFront Distribution (TLS termination)
  ↓
API Gateway HTTP API
  ↓
Lambda Proxy Function
  ↓
DynamoDB Cache ← Cache Warmer (PostgreSQL sync)
  ↓
AWS IoT Secure Tunnel
  ↓
Device Local Proxy Agent
  ↓
Device Web Service (localhost:8080)

​

Deployment Steps

​

1. Prerequisites

• ✅ AWS CLI configured with appropriate credentials
• ✅ CDK CLI installed (npm install -g aws-cdk)
• ✅ Route53 hosted zone for roboticks.io (already exists)
• ✅ Backend database with reverse_tunnels table

​

2. Deploy Infrastructure

cd /Users/mujacic/roboticks/infrastructure

# Install dependencies
npm install

# Bootstrap CDK (first time only)
cdk bootstrap

# Synthesize CloudFormation template
cdk synth

# Deploy the stack
cdk deploy --require-approval never

# View outputs
aws cloudformation describe-stacks \
  --stack-name RoboticksStack \
  --query "Stacks[0].Outputs" \
  --output table

​

3. Verify Deployment

aws cloudfront list-distributions \
  --query "DistributionList.Items[?Comment=='Roboticks Reverse Tunnel Distribution'].{Id:Id,Domain:DomainName,Status:Status}"

aws apigatewayv2 get-apis \
  --query "Items[?Name=='roboticks-reverse-tunnel-api'].{ApiId:ApiId,Endpoint:ApiEndpoint}"

aws lambda list-functions \
  --query "Functions[?starts_with(FunctionName, 'RoboticksStack-ReverseTunnel')].{Name:FunctionName,Runtime:Runtime,Status:State}"

aws dynamodb describe-table \
  --table-name roboticks-reverse-tunnel-cache \
  --query "Table.{Name:TableName,Status:TableStatus,ItemCount:ItemCount}"

# Get hosted zone ID for roboticks.io
ZONE_ID=$(aws route53 list-hosted-zones \
  --query "HostedZones[?Name=='roboticks.io.'].Id" \
  --output text | cut -d'/' -f3)

# List fleet subdomain records
aws route53 list-resource-record-sets \
  --hosted-zone-id $ZONE_ID \
  --query "ResourceRecordSets[?contains(Name, 'fleet')]"

​

4. Test DNS Resolution

# Test wildcard DNS (replace with actual CloudFront domain from outputs)
dig device-test.fleet.roboticks.io

# Should resolve to CloudFront distribution
# Look for CNAME or A record pointing to CloudFront

​

5. Test Lambda Proxy

# Get API URL from CDK outputs
API_URL=$(aws cloudformation describe-stacks \
  --stack-name RoboticksStack \
  --query "Stacks[0].Outputs[?OutputKey=='TunnelApiUrl'].OutputValue" \
  --output text)

# Test proxy Lambda (will return 404 if no tunnel exists)
curl -v $API_URL \
  -H "Host: device-test.fleet.roboticks.io"

# Expected response (if no tunnel configured):
# {"error": "Tunnel not found or not active"}

​

6. Test Cache Warmer

# Invoke cache warmer manually
aws lambda invoke \
  --function-name RoboticksStack-TunnelCacheWarmer \
  --payload '{}' \
  response.json

# Check response
cat response.json

# Expected output:
# {
#   "statusCode": 200,
#   "body": "{\"message\":\"Cache warmer completed successfully\",\"tunnels_found\":0,\"tunnels_synced\":0}"
# }

​

Post-Deployment Configuration

​

1. Create Test Reverse Tunnel

# Via API (replace with real device DSN and JWT token)
curl -X POST https://api.roboticks.io/api/v1/reverse-tunnels/devices/{dsn}/reverse-tunnels \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Web Dashboard",
    "local_port": 8080,
    "protocol": "http",
    "require_auth": true
  }'

​

2. Wait for Cache Sync

aws lambda invoke \
  --function-name RoboticksStack-TunnelCacheWarmer \
  --payload '{}' \
  response.json

​

3. Test Public URL Access

# Access device tunnel (replace with actual device DSN)
curl https://device-abc123.fleet.roboticks.io

# Expected response (tunnel status):
# {
#   "message": "Reverse tunnel is active",
#   "dsn": "abc123",
#   "public_url": "device-abc123.fleet.roboticks.io",
#   "tunnel_id": "aws-tunnel-id-...",
#   "tunnel_status": "OPEN"
# }

​

Monitoring

​

CloudWatch Logs

aws logs tail /aws/lambda/RoboticksStack-ReverseTunnelProxy --follow

aws logs tail /aws/lambda/RoboticksStack-TunnelCacheWarmer --follow

​

CloudWatch Metrics

# Lambda proxy invocations
aws cloudwatch get-metric-statistics \
  --namespace AWS/Lambda \
  --metric-name Invocations \
  --dimensions Name=FunctionName,Value=RoboticksStack-ReverseTunnelProxy \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 300 \
  --statistics Sum

# Lambda proxy errors
aws cloudwatch get-metric-statistics \
  --namespace AWS/Lambda \
  --metric-name Errors \
  --dimensions Name=FunctionName,Value=RoboticksStack-ReverseTunnelProxy \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 300 \
  --statistics Sum

​

Troubleshooting

​

Issue: DNS Not Resolving

1. Check Route53 records exist:
   Copy

   aws route53 list-resource-record-sets --hosted-zone-id $ZONE_ID \
     --query "ResourceRecordSets[?contains(Name, 'fleet')]"
   

2. Wait for DNS propagation (can take up to 48 hours, usually < 5 minutes)
3. Clear local DNS cache: sudo dscacheutil -flushcache (macOS)

​

Issue: SSL Certificate Errors

1. Check certificate status:
   Copy

   aws acm describe-certificate \
     --certificate-arn $(aws cloudformation describe-stacks \
       --stack-name RoboticksStack \
       --query "Stacks[0].Outputs[?OutputKey=='FleetCertificateArn'].OutputValue" \
       --output text)
   

2. Ensure status is ISSUED
3. If PENDING_VALIDATION, check Route53 validation records were created
4. Wait for certificate issuance (can take 30 minutes)

​

Issue: Lambda Proxy Returns 500

1. Check Lambda logs:
   Copy

   aws logs tail /aws/lambda/RoboticksStack-ReverseTunnelProxy --follow
   

2. Common causes:
   • Database connection timeout (check VPC security groups)
   • DynamoDB table doesn’t exist (check table name in environment)
   • Missing IAM permissions (check Lambda execution role)

​

Issue: Tunnel Not Found

1. Check tunnel exists in database:
   Copy

   SELECT * FROM reverse_tunnels WHERE public_url LIKE '%fleet.roboticks.io';
   

2. Check tunnel is enabled and not expired:
   Copy

   SELECT * FROM reverse_tunnels
   WHERE is_enabled = true
     AND tunnel_id IS NOT NULL
     AND expires_at > NOW();
   

3. Manually invoke cache warmer to sync to DynamoDB:
   Copy

   aws lambda invoke \
     --function-name RoboticksStack-TunnelCacheWarmer \
     response.json
   

4. Check DynamoDB cache:
   Copy

   aws dynamodb scan --table-name roboticks-reverse-tunnel-cache
   

​

Issue: Cache Warmer Fails

1. Check Lambda logs for error details
2. Common causes:
   • Database connection failure (check VPC/security groups)
   • Missing database credentials (check Secrets Manager)
   • DynamoDB write permission denied (check IAM role)
3. Test database connectivity from Lambda:
   Copy

   aws lambda invoke \
     --function-name RoboticksStack-TunnelCacheWarmer \
     --payload '{}' \
     response.json && cat response.json
   

​

Rollback

# Destroy the entire stack (WARNING: This deletes all resources)
cdk destroy

# Or manually delete individual resources via AWS Console

1. Backup DynamoDB table data if needed
2. Note that Route53 records will be deleted
3. CloudFront distribution deletion can take 15-30 minutes

​

Cost Optimization

​

Current Configuration

• DynamoDB: PAY_PER_REQUEST (serverless)
• Lambda: 512 MB proxy, 256 MB cache warmer
• CloudFront: PRICE_CLASS_100 (North America + Europe)
• API Gateway: HTTP API (cheaper than REST API)

​

Estimated Costs (100 devices, 1000 req/device/day)

• Lambda: ~$10/month
• DynamoDB: ~$1/month
• CloudFront: ~$5/month
• API Gateway: ~$3/month
• Route53: $0.50/month
• ACM Certificate: Free
• Total: ~$20/month

​

Cost Reduction Options

1. Reduce Lambda Memory: Lower proxy Lambda to 256 MB if traffic is light
2. Increase Cache TTL: Reduce cache warmer frequency to every 10-15 minutes
3. CloudFront Price Class: Use PRICE_CLASS_ALL for global coverage (more expensive)
4. Reserved Concurrency: Add reserved concurrency to Lambda for predictable costs

​

Security Considerations

​

Current Security Measures

• ✅ TLS encryption (CloudFront + ACM certificate)
• ✅ VPC isolation for Lambda functions
• ✅ IAM least-privilege permissions
• ✅ DynamoDB encryption at rest (default)
• ✅ CloudFront access logging enabled
• ✅ AWS IoT certificate-based device authentication

​

Recommended Enhancements

1. WAF Rules: Add AWS WAF rules to CloudFront distribution
2. Rate Limiting: Add API Gateway throttling per device
3. Authentication: Implement JWT validation in Lambda proxy
4. Monitoring: Add CloudWatch alarms for anomalous traffic
5. Audit Logging: Enable CloudTrail for API calls

​

Next Steps

​

Infrastructure: ✅ Complete

​

Implementation Remaining:

1. Lambda Proxy Logic (~12-16 hours):
   • WebSocket-to-WebSocket bidirectional forwarding
   • HTTP-to-IoT-Tunnel request proxying
   • AWS IoT Secure Tunneling Local Proxy protocol
2. Device SDK (~4 hours):
   • Reverse tunnel manager in roboticks-sdk
   • Device startup tunnel configuration fetch
3. Monitoring (~2 hours):
   • CloudWatch alarms
   • Metrics dashboard
4. Testing (~4 hours):
   • End-to-end testing with real devices

​

Support

• Check REVERSE_TUNNELS_INFRASTRUCTURE.md
• Check REVERSE_TUNNELS_IMPLEMENTATION_GUIDE.md
• Review CloudWatch logs for errors
• Check AWS documentation for specific services

​

References

• AWS CDK Documentation
• AWS IoT Secure Tunneling
• CloudFront Documentation
• API Gateway HTTP APIs
• Route53 Documentation