BACKBEAT/contracts/README.md

# BACKBEAT Contracts Package

[![Build Status](https://github.com/chorus-services/backbeat/actions/workflows/contracts.yml/badge.svg)](https://github.com/chorus-services/backbeat/actions/workflows/contracts.yml)
[![Schema Version](https://img.shields.io/badge/schema-v1.0.0-blue)](schemas/)
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)

The authoritative contract definitions and validation tools for BACKBEAT distributed orchestration across the CHORUS 2.0.0 ecosystem.

## 🎯 Overview

BACKBEAT provides synchronized distributed execution through three core message interfaces:

- **INT-A (BeatFrame)**: 🥁 Rhythm coordination from Pulse → All Services  
- **INT-B (StatusClaim)**: 📊 Agent status reporting from Agents → Reverb
- **INT-C (BarReport)**: 📈 Periodic summaries from Reverb → All Services

This contracts package ensures all CHORUS 2.0.0 projects can reliably integrate with BACKBEAT through:

✅ **JSON Schema Validation** - Semver-versioned schemas for all interfaces  
✅ **Conformance Testing** - Comprehensive test suites with valid/invalid examples  
✅ **CI Integration** - Drop-in validation for any CI pipeline  
✅ **Documentation** - Complete integration guides and best practices  

## 🚀 Quick Start

### 1. Validate Your Messages

```bash
# Clone the contracts repository
git clone https://github.com/chorus-services/backbeat.git
cd backbeat/contracts

# Build the validation tool
cd tests/integration && make build

# Validate your BACKBEAT messages
./backbeat-validate --schemas ../../schemas --dir /path/to/your/messages --exit-code
```

### 2. Add to CI Pipeline

#### GitHub Actions
```yaml
- name: Validate BACKBEAT Contracts
  run: |
    git clone https://github.com/chorus-services/backbeat.git
    cd backbeat/contracts/tests/integration
    make build
    ./backbeat-validate --schemas ../../schemas --dir ${{ github.workspace }}/messages --exit-code
```

#### GitLab CI
```yaml
validate-backbeat:
  script:
    - git clone https://github.com/chorus-services/backbeat.git
    - cd backbeat/contracts/tests/integration && make build
    - ./backbeat-validate --schemas ../../schemas --dir messages --exit-code
```

### 3. Integrate with Your Project

Add to your `Makefile`:
```makefile
validate-backbeat:
	@git clone https://github.com/chorus-services/backbeat.git .backbeat 2>/dev/null || true
	@cd .backbeat/contracts/tests/integration && make build
	@.backbeat/contracts/tests/integration/backbeat-validate --schemas .backbeat/contracts/schemas --dir messages --exit-code
```

## 📁 Package Structure

```
contracts/
├── schemas/                    # JSON Schema definitions
│   ├── beatframe-v1.schema.json      # INT-A: Pulse → All Services
│   ├── statusclaim-v1.schema.json    # INT-B: Agents → Reverb  
│   └── barreport-v1.schema.json      # INT-C: Reverb → All Services
├── tests/
│   ├── conformance_test.go           # Go conformance test suite
│   ├── examples/                     # Valid/invalid message examples
│   │   ├── beatframe-valid.json
│   │   ├── beatframe-invalid.json
│   │   ├── statusclaim-valid.json
│   │   ├── statusclaim-invalid.json
│   │   ├── barreport-valid.json
│   │   └── barreport-invalid.json
│   └── integration/                  # CI integration helpers
│       ├── validator.go              # Message validation library
│       ├── ci_helper.go              # CI integration utilities
│       ├── cmd/backbeat-validate/    # CLI validation tool
│       └── Makefile                  # Build and test automation
├── docs/
│   ├── integration-guide.md          # How to BACKBEAT-enable services
│   ├── schema-evolution.md           # Versioning and compatibility
│   └── tempo-guide.md                # Beat timing recommendations  
└── README.md                         # This file
```

## 🔧 Core Interfaces

### INT-A: BeatFrame (Pulse → All Services)

Synchronization messages broadcast every beat:

```json
{
  "type": "backbeat.beatframe.v1",
  "cluster_id": "chorus-prod",
  "beat_index": 1337,
  "downbeat": false,
  "phase": "execute",
  "hlc": "7ffd:0001:abcd",
  "deadline_at": "2025-09-05T12:30:00Z",
  "tempo_bpm": 2.0,
  "window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5"
}
```

**Key Fields:**
- `beat_index`: Monotonic counter since cluster start
- `phase`: `"plan"`, `"execute"`, or `"review"`
- `tempo_bpm`: Current beats per minute (default: 2.0 = 30-second beats)
- `deadline_at`: When this phase must complete

### INT-B: StatusClaim (Agents → Reverb)

Agent status reports during beat execution:

```json
{
  "type": "backbeat.statusclaim.v1",
  "agent_id": "search-indexer:worker-03", 
  "task_id": "index-batch:20250905-120",
  "beat_index": 1337,
  "state": "executing",
  "beats_left": 3,
  "progress": 0.65,
  "notes": "processing batch 120/200",
  "hlc": "7ffd:0001:beef"
}
```

**Key Fields:**
- `state`: `"idle"`, `"planning"`, `"executing"`, `"reviewing"`, `"completed"`, `"failed"`, `"blocked"`, `"helping"`
- `beats_left`: Estimated beats to completion
- `progress`: Completion percentage (0.0 - 1.0)

### INT-C: BarReport (Reverb → All Services)

Periodic cluster health summaries:

```json
{
  "type": "backbeat.barreport.v1",
  "window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5",
  "from_beat": 240,
  "to_beat": 359, 
  "agents_reporting": 978,
  "on_time_reviews": 942,
  "help_promises_fulfilled": 87,
  "secret_rotations_ok": true,
  "tempo_drift_ms": 7.3,
  "issues": []
}
```

**Key Fields:**
- `agents_reporting`: Total active agents in window
- `on_time_reviews`: Agents completing review phase on time  
- `tempo_drift_ms`: Timing drift (positive = behind, negative = ahead)

## 🛠️ Usage Examples

### Validate Single Message

```bash
# Validate from file
./backbeat-validate --schemas ../schemas --file message.json

# Validate from stdin
echo '{"type":"backbeat.beatframe.v1",...}' | ./backbeat-validate --schemas ../schemas --message -

# Get JSON output for programmatic use
./backbeat-validate --schemas ../schemas --file message.json --json
```

### Validate Directory

```bash
# Validate all JSON files in directory
./backbeat-validate --schemas ../schemas --dir messages/

# Quiet mode (only errors)
./backbeat-validate --schemas ../schemas --dir messages/ --quiet

# Exit with error code on validation failures
./backbeat-validate --schemas ../schemas --dir messages/ --exit-code
```

### Go Integration

```go
import "github.com/chorus-services/backbeat/contracts/tests/integration"

// Create validator
validator, err := integration.NewMessageValidator("./schemas")
if err != nil {
    log.Fatal(err)
}

// Validate message
result, err := validator.ValidateMessageString(`{"type":"backbeat.beatframe.v1",...}`)
if err != nil {
    log.Fatal(err)
}

if !result.Valid {
    log.Errorf("Validation failed: %v", result.Errors)
}
```

## 📊 Tempo Recommendations

| Use Case | Tempo (BPM) | Beat Duration | Example Services |
|----------|-------------|---------------|------------------|
| **Development** | 0.1 - 0.5 | 2-10 minutes | Testing, debugging |
| **Batch Processing** | 0.5 - 2.0 | 30s - 2 minutes | ETL, data warehouses |
| **Standard Services** | 2.0 - 10.0 | 6-30 seconds | APIs, web apps |
| **Responsive Apps** | 10.0 - 60.0 | 1-6 seconds | Dashboards, monitoring |
| **High-Frequency** | 60+ | <1 second | Trading, IoT processing |

**Default**: 2.0 BPM (30-second beats) works well for most CHORUS services.

## 📋 Integration Checklist

- [ ] **Message Validation**: Add schema validation to your CI pipeline
- [ ] **BeatFrame Handler**: Implement INT-A message consumption
- [ ] **StatusClaim Publisher**: Implement INT-B message publishing (if you have agents)
- [ ] **BarReport Consumer**: Implement INT-C message consumption (optional)
- [ ] **Tempo Selection**: Choose appropriate BPM for your workload
- [ ] **Error Handling**: Handle validation failures and timing issues
- [ ] **Monitoring**: Track beat processing latency and deadline misses
- [ ] **Load Testing**: Verify performance at production tempo

## 🔄 Schema Versioning

Schemas follow [Semantic Versioning](https://semver.org/):

- **MAJOR** (1.0.0 → 2.0.0): Breaking changes requiring code updates
- **MINOR** (1.0.0 → 1.1.0): Backward-compatible additions
- **PATCH** (1.0.0 → 1.0.1): Documentation and example updates

Current versions:
- **BeatFrame**: v1.0.0 (`backbeat.beatframe.v1`)
- **StatusClaim**: v1.0.0 (`backbeat.statusclaim.v1`) 
- **BarReport**: v1.0.0 (`backbeat.barreport.v1`)

See [schema-evolution.md](docs/schema-evolution.md) for migration strategies.

## 🧪 Running Tests

```bash
# Run all tests
make test

# Test schemas are valid JSON
make test-schemas

# Test example messages
make test-examples  

# Run Go integration tests
make test-integration

# Validate built-in examples
make validate-examples
```

## 🏗️ Building

```bash
# Build CLI validation tool
make build

# Install Go dependencies
make deps

# Format code
make fmt

# Run linter
make lint

# Generate CI configuration examples
make examples
```

## 📚 Documentation

- **[Integration Guide](docs/integration-guide.md)**: Complete guide for CHORUS 2.0.0 projects
- **[Schema Evolution](docs/schema-evolution.md)**: Versioning and compatibility management  
- **[Tempo Guide](docs/tempo-guide.md)**: Beat timing and performance optimization

## 🤝 Contributing

1. **Fork** this repository
2. **Create** a feature branch: `git checkout -b feature/amazing-feature`
3. **Add** tests for your changes
4. **Run** `make test` to ensure everything passes
5. **Commit** your changes: `git commit -m 'Add amazing feature'`
6. **Push** to the branch: `git push origin feature/amazing-feature`  
7. **Open** a Pull Request

### Schema Changes

- **Minor changes** (new optional fields): Create PR with updated schema
- **Major changes** (breaking): Discuss in issue first, follow migration process
- **All changes**: Update examples and tests accordingly

## 🔍 Troubleshooting

### Common Validation Errors

| Error | Cause | Fix |
|-------|-------|-----|
| `type field is required` | Missing `type` field | Add correct message type |
| `hlc must match pattern` | Invalid HLC format | Use `XXXX:XXXX:XXXX` hex format |  
| `window_id must be exactly 32 hex characters` | Wrong window ID | Use 32-character hex string |
| `phase must be one of: plan, execute, review` | Invalid phase | Use exact phase names |
| `tempo_bpm must be at least 0.1` | Tempo too low | Use tempo ≥ 0.1 BPM |

### Performance Issues

- **Beat processing too slow**: Reduce tempo or optimize code
- **High CPU usage**: Consider lower tempo or horizontal scaling
- **Network saturation**: Reduce message frequency or size
- **Memory leaks**: Ensure proper cleanup in beat handlers

### Getting Help

- **Issues**: [GitHub Issues](https://github.com/chorus-services/backbeat/issues)
- **Discussions**: [GitHub Discussions](https://github.com/chorus-services/backbeat/discussions)
- **Documentation**: Check the [docs/](docs/) directory
- **Examples**: See [tests/examples/](tests/examples/) for message samples

## 📜 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🎵 About BACKBEAT

BACKBEAT provides the rhythmic heartbeat that synchronizes distributed systems across CHORUS 2.0.0. Just as musicians use a metronome to stay in time, BACKBEAT keeps your services coordinated and responsive.

**Key Benefits:**
- 🎯 **Predictable Timing**: Know exactly when coordination happens
- 🔄 **Graceful Coordination**: Services sync without tight coupling  
- 📊 **Health Visibility**: Real-time insight into cluster performance
- 🛡️ **Fault Tolerance**: Detect and recover from failures quickly
- ⚡ **Scalable**: Works from development (0.1 BPM) to high-frequency (1000+ BPM)

---

**Made with ❤️ by the CHORUS 2.0.0 team**

*"In rhythm there is coordination, in coordination there is reliability."*