WHOOSH/BACKBEAT-prototype/contracts/README.md

BACKBEAT Contracts Package

The authoritative contract definitions and validation tools for BACKBEAT distributed orchestration across the CHORUS 2.0.0 ecosystem.

🎯 Overview

BACKBEAT provides synchronized distributed execution through three core message interfaces:

• INT-A (BeatFrame): 🥁 Rhythm coordination from Pulse → All Services
• INT-B (StatusClaim): 📊 Agent status reporting from Agents → Reverb
• INT-C (BarReport): 📈 Periodic summaries from Reverb → All Services

This contracts package ensures all CHORUS 2.0.0 projects can reliably integrate with BACKBEAT through:

✅ JSON Schema Validation - Semver-versioned schemas for all interfaces
✅ Conformance Testing - Comprehensive test suites with valid/invalid examples
✅ CI Integration - Drop-in validation for any CI pipeline
✅ Documentation - Complete integration guides and best practices

🚀 Quick Start

1. Validate Your Messages

# Clone the contracts repository
git clone https://github.com/chorus-services/backbeat.git
cd backbeat/contracts

# Build the validation tool
cd tests/integration && make build

# Validate your BACKBEAT messages
./backbeat-validate --schemas ../../schemas --dir /path/to/your/messages --exit-code

2. Add to CI Pipeline

GitHub Actions

- name: Validate BACKBEAT Contracts
  run: |
    git clone https://github.com/chorus-services/backbeat.git
    cd backbeat/contracts/tests/integration
    make build
    ./backbeat-validate --schemas ../../schemas --dir ${{ github.workspace }}/messages --exit-code

GitLab CI

validate-backbeat:
  script:
    - git clone https://github.com/chorus-services/backbeat.git
    - cd backbeat/contracts/tests/integration && make build
    - ./backbeat-validate --schemas ../../schemas --dir messages --exit-code

3. Integrate with Your Project

Add to your Makefile:

validate-backbeat:
	@git clone https://github.com/chorus-services/backbeat.git .backbeat 2>/dev/null || true
	@cd .backbeat/contracts/tests/integration && make build
	@.backbeat/contracts/tests/integration/backbeat-validate --schemas .backbeat/contracts/schemas --dir messages --exit-code

📁 Package Structure

contracts/
├── schemas/                    # JSON Schema definitions
│   ├── beatframe-v1.schema.json      # INT-A: Pulse → All Services
│   ├── statusclaim-v1.schema.json    # INT-B: Agents → Reverb  
│   └── barreport-v1.schema.json      # INT-C: Reverb → All Services
├── tests/
│   ├── conformance_test.go           # Go conformance test suite
│   ├── examples/                     # Valid/invalid message examples
│   │   ├── beatframe-valid.json
│   │   ├── beatframe-invalid.json
│   │   ├── statusclaim-valid.json
│   │   ├── statusclaim-invalid.json
│   │   ├── barreport-valid.json
│   │   └── barreport-invalid.json
│   └── integration/                  # CI integration helpers
│       ├── validator.go              # Message validation library
│       ├── ci_helper.go              # CI integration utilities
│       ├── cmd/backbeat-validate/    # CLI validation tool
│       └── Makefile                  # Build and test automation
├── docs/
│   ├── integration-guide.md          # How to BACKBEAT-enable services
│   ├── schema-evolution.md           # Versioning and compatibility
│   └── tempo-guide.md                # Beat timing recommendations  
└── README.md                         # This file

🔧 Core Interfaces

INT-A: BeatFrame (Pulse → All Services)

Synchronization messages broadcast every beat:

{
  "type": "backbeat.beatframe.v1",
  "cluster_id": "chorus-prod",
  "beat_index": 1337,
  "downbeat": false,
  "phase": "execute",
  "hlc": "7ffd:0001:abcd",
  "deadline_at": "2025-09-05T12:30:00Z",
  "tempo_bpm": 2.0,
  "window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5"
}

Key Fields:

• beat_index: Monotonic counter since cluster start
• phase: "plan", "execute", or "review"
• tempo_bpm: Current beats per minute (default: 2.0 = 30-second beats)
• deadline_at: When this phase must complete

INT-B: StatusClaim (Agents → Reverb)

Agent status reports during beat execution:

{
  "type": "backbeat.statusclaim.v1",
  "agent_id": "search-indexer:worker-03", 
  "task_id": "index-batch:20250905-120",
  "beat_index": 1337,
  "state": "executing",
  "beats_left": 3,
  "progress": 0.65,
  "notes": "processing batch 120/200",
  "hlc": "7ffd:0001:beef"
}

Key Fields:

• state: "idle", "planning", "executing", "reviewing", "completed", "failed", "blocked", "helping"
• beats_left: Estimated beats to completion
• progress: Completion percentage (0.0 - 1.0)

INT-C: BarReport (Reverb → All Services)

Periodic cluster health summaries:

{
  "type": "backbeat.barreport.v1",
  "window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5",
  "from_beat": 240,
  "to_beat": 359, 
  "agents_reporting": 978,
  "on_time_reviews": 942,
  "help_promises_fulfilled": 87,
  "secret_rotations_ok": true,
  "tempo_drift_ms": 7.3,
  "issues": []
}

Key Fields:

• agents_reporting: Total active agents in window
• on_time_reviews: Agents completing review phase on time
• tempo_drift_ms: Timing drift (positive = behind, negative = ahead)

🛠️ Usage Examples

Validate Single Message

# Validate from file
./backbeat-validate --schemas ../schemas --file message.json

# Validate from stdin
echo '{"type":"backbeat.beatframe.v1",...}' | ./backbeat-validate --schemas ../schemas --message -

# Get JSON output for programmatic use
./backbeat-validate --schemas ../schemas --file message.json --json

Validate Directory

# Validate all JSON files in directory
./backbeat-validate --schemas ../schemas --dir messages/

# Quiet mode (only errors)
./backbeat-validate --schemas ../schemas --dir messages/ --quiet

# Exit with error code on validation failures
./backbeat-validate --schemas ../schemas --dir messages/ --exit-code

Go Integration

import "github.com/chorus-services/backbeat/contracts/tests/integration"

// Create validator
validator, err := integration.NewMessageValidator("./schemas")
if err != nil {
    log.Fatal(err)
}

// Validate message
result, err := validator.ValidateMessageString(`{"type":"backbeat.beatframe.v1",...}`)
if err != nil {
    log.Fatal(err)
}

if !result.Valid {
    log.Errorf("Validation failed: %v", result.Errors)
}

📊 Tempo Recommendations

Use Case	Tempo (BPM)	Beat Duration	Example Services
Development	0.1 - 0.5	2-10 minutes	Testing, debugging
Batch Processing	0.5 - 2.0	30s - 2 minutes	ETL, data warehouses
Standard Services	2.0 - 10.0	6-30 seconds	APIs, web apps
Responsive Apps	10.0 - 60.0	1-6 seconds	Dashboards, monitoring
High-Frequency	60+	<1 second	Trading, IoT processing

Default: 2.0 BPM (30-second beats) works well for most CHORUS services.

📋 Integration Checklist

• Message Validation: Add schema validation to your CI pipeline
• BeatFrame Handler: Implement INT-A message consumption
• StatusClaim Publisher: Implement INT-B message publishing (if you have agents)
• BarReport Consumer: Implement INT-C message consumption (optional)
• Tempo Selection: Choose appropriate BPM for your workload
• Error Handling: Handle validation failures and timing issues
• Monitoring: Track beat processing latency and deadline misses
• Load Testing: Verify performance at production tempo

🔄 Schema Versioning

Schemas follow Semantic Versioning:

• MAJOR (1.0.0 → 2.0.0): Breaking changes requiring code updates
• MINOR (1.0.0 → 1.1.0): Backward-compatible additions
• PATCH (1.0.0 → 1.0.1): Documentation and example updates

Current versions:

• BeatFrame: v1.0.0 (backbeat.beatframe.v1)
• StatusClaim: v1.0.0 (backbeat.statusclaim.v1)
• BarReport: v1.0.0 (backbeat.barreport.v1)

See schema-evolution.md for migration strategies.

🧪 Running Tests

# Run all tests
make test

# Test schemas are valid JSON
make test-schemas

# Test example messages
make test-examples  

# Run Go integration tests
make test-integration

# Validate built-in examples
make validate-examples

🏗️ Building

# Build CLI validation tool
make build

# Install Go dependencies
make deps

# Format code
make fmt

# Run linter
make lint

# Generate CI configuration examples
make examples

📚 Documentation

• Integration Guide: Complete guide for CHORUS 2.0.0 projects
• Schema Evolution: Versioning and compatibility management
• Tempo Guide: Beat timing and performance optimization

🤝 Contributing

1. Fork this repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Add tests for your changes
4. Run make test to ensure everything passes
5. Commit your changes: git commit -m 'Add amazing feature'
6. Push to the branch: git push origin feature/amazing-feature
7. Open a Pull Request

Schema Changes

• Minor changes (new optional fields): Create PR with updated schema
• Major changes (breaking): Discuss in issue first, follow migration process
• All changes: Update examples and tests accordingly

🔍 Troubleshooting

Common Validation Errors

Error	Cause	Fix
type field is required	Missing type field	Add correct message type
hlc must match pattern	Invalid HLC format	Use XXXX:XXXX:XXXX hex format
window_id must be exactly 32 hex characters	Wrong window ID	Use 32-character hex string
phase must be one of: plan, execute, review	Invalid phase	Use exact phase names
tempo_bpm must be at least 0.1	Tempo too low	Use tempo ≥ 0.1 BPM

Performance Issues

• Beat processing too slow: Reduce tempo or optimize code
• High CPU usage: Consider lower tempo or horizontal scaling
• Network saturation: Reduce message frequency or size
• Memory leaks: Ensure proper cleanup in beat handlers

Getting Help

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Check the docs/ directory
• Examples: See tests/examples/ for message samples

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.

🎵 About BACKBEAT

BACKBEAT provides the rhythmic heartbeat that synchronizes distributed systems across CHORUS 2.0.0. Just as musicians use a metronome to stay in time, BACKBEAT keeps your services coordinated and responsive.

Key Benefits:

• 🎯 Predictable Timing: Know exactly when coordination happens
• 🔄 Graceful Coordination: Services sync without tight coupling
• 📊 Health Visibility: Real-time insight into cluster performance
• 🛡️ Fault Tolerance: Detect and recover from failures quickly
• ⚡ Scalable: Works from development (0.1 BPM) to high-frequency (1000+ BPM)

---

Made with ❤️ by the CHORUS 2.0.0 team

"In rhythm there is coordination, in coordination there is reliability."