366 lines
12 KiB
Markdown
366 lines
12 KiB
Markdown
# BACKBEAT Contracts Package
|
||
|
||
[](https://github.com/chorus-services/backbeat/actions/workflows/contracts.yml)
|
||
[](schemas/)
|
||
[](LICENSE)
|
||
|
||
The authoritative contract definitions and validation tools for BACKBEAT distributed orchestration across the CHORUS 2.0.0 ecosystem.
|
||
|
||
## 🎯 Overview
|
||
|
||
BACKBEAT provides synchronized distributed execution through three core message interfaces:
|
||
|
||
- **INT-A (BeatFrame)**: 🥁 Rhythm coordination from Pulse → All Services
|
||
- **INT-B (StatusClaim)**: 📊 Agent status reporting from Agents → Reverb
|
||
- **INT-C (BarReport)**: 📈 Periodic summaries from Reverb → All Services
|
||
|
||
This contracts package ensures all CHORUS 2.0.0 projects can reliably integrate with BACKBEAT through:
|
||
|
||
✅ **JSON Schema Validation** - Semver-versioned schemas for all interfaces
|
||
✅ **Conformance Testing** - Comprehensive test suites with valid/invalid examples
|
||
✅ **CI Integration** - Drop-in validation for any CI pipeline
|
||
✅ **Documentation** - Complete integration guides and best practices
|
||
|
||
## 🚀 Quick Start
|
||
|
||
### 1. Validate Your Messages
|
||
|
||
```bash
|
||
# Clone the contracts repository
|
||
git clone https://github.com/chorus-services/backbeat.git
|
||
cd backbeat/contracts
|
||
|
||
# Build the validation tool
|
||
cd tests/integration && make build
|
||
|
||
# Validate your BACKBEAT messages
|
||
./backbeat-validate --schemas ../../schemas --dir /path/to/your/messages --exit-code
|
||
```
|
||
|
||
### 2. Add to CI Pipeline
|
||
|
||
#### GitHub Actions
|
||
```yaml
|
||
- name: Validate BACKBEAT Contracts
|
||
run: |
|
||
git clone https://github.com/chorus-services/backbeat.git
|
||
cd backbeat/contracts/tests/integration
|
||
make build
|
||
./backbeat-validate --schemas ../../schemas --dir ${{ github.workspace }}/messages --exit-code
|
||
```
|
||
|
||
#### GitLab CI
|
||
```yaml
|
||
validate-backbeat:
|
||
script:
|
||
- git clone https://github.com/chorus-services/backbeat.git
|
||
- cd backbeat/contracts/tests/integration && make build
|
||
- ./backbeat-validate --schemas ../../schemas --dir messages --exit-code
|
||
```
|
||
|
||
### 3. Integrate with Your Project
|
||
|
||
Add to your `Makefile`:
|
||
```makefile
|
||
validate-backbeat:
|
||
@git clone https://github.com/chorus-services/backbeat.git .backbeat 2>/dev/null || true
|
||
@cd .backbeat/contracts/tests/integration && make build
|
||
@.backbeat/contracts/tests/integration/backbeat-validate --schemas .backbeat/contracts/schemas --dir messages --exit-code
|
||
```
|
||
|
||
## 📁 Package Structure
|
||
|
||
```
|
||
contracts/
|
||
├── schemas/ # JSON Schema definitions
|
||
│ ├── beatframe-v1.schema.json # INT-A: Pulse → All Services
|
||
│ ├── statusclaim-v1.schema.json # INT-B: Agents → Reverb
|
||
│ └── barreport-v1.schema.json # INT-C: Reverb → All Services
|
||
├── tests/
|
||
│ ├── conformance_test.go # Go conformance test suite
|
||
│ ├── examples/ # Valid/invalid message examples
|
||
│ │ ├── beatframe-valid.json
|
||
│ │ ├── beatframe-invalid.json
|
||
│ │ ├── statusclaim-valid.json
|
||
│ │ ├── statusclaim-invalid.json
|
||
│ │ ├── barreport-valid.json
|
||
│ │ └── barreport-invalid.json
|
||
│ └── integration/ # CI integration helpers
|
||
│ ├── validator.go # Message validation library
|
||
│ ├── ci_helper.go # CI integration utilities
|
||
│ ├── cmd/backbeat-validate/ # CLI validation tool
|
||
│ └── Makefile # Build and test automation
|
||
├── docs/
|
||
│ ├── integration-guide.md # How to BACKBEAT-enable services
|
||
│ ├── schema-evolution.md # Versioning and compatibility
|
||
│ └── tempo-guide.md # Beat timing recommendations
|
||
└── README.md # This file
|
||
```
|
||
|
||
## 🔧 Core Interfaces
|
||
|
||
### INT-A: BeatFrame (Pulse → All Services)
|
||
|
||
Synchronization messages broadcast every beat:
|
||
|
||
```json
|
||
{
|
||
"type": "backbeat.beatframe.v1",
|
||
"cluster_id": "chorus-prod",
|
||
"beat_index": 1337,
|
||
"downbeat": false,
|
||
"phase": "execute",
|
||
"hlc": "7ffd:0001:abcd",
|
||
"deadline_at": "2025-09-05T12:30:00Z",
|
||
"tempo_bpm": 2.0,
|
||
"window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5"
|
||
}
|
||
```
|
||
|
||
**Key Fields:**
|
||
- `beat_index`: Monotonic counter since cluster start
|
||
- `phase`: `"plan"`, `"execute"`, or `"review"`
|
||
- `tempo_bpm`: Current beats per minute (default: 2.0 = 30-second beats)
|
||
- `deadline_at`: When this phase must complete
|
||
|
||
### INT-B: StatusClaim (Agents → Reverb)
|
||
|
||
Agent status reports during beat execution:
|
||
|
||
```json
|
||
{
|
||
"type": "backbeat.statusclaim.v1",
|
||
"agent_id": "search-indexer:worker-03",
|
||
"task_id": "index-batch:20250905-120",
|
||
"beat_index": 1337,
|
||
"state": "executing",
|
||
"beats_left": 3,
|
||
"progress": 0.65,
|
||
"notes": "processing batch 120/200",
|
||
"hlc": "7ffd:0001:beef"
|
||
}
|
||
```
|
||
|
||
**Key Fields:**
|
||
- `state`: `"idle"`, `"planning"`, `"executing"`, `"reviewing"`, `"completed"`, `"failed"`, `"blocked"`, `"helping"`
|
||
- `beats_left`: Estimated beats to completion
|
||
- `progress`: Completion percentage (0.0 - 1.0)
|
||
|
||
### INT-C: BarReport (Reverb → All Services)
|
||
|
||
Periodic cluster health summaries:
|
||
|
||
```json
|
||
{
|
||
"type": "backbeat.barreport.v1",
|
||
"window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5",
|
||
"from_beat": 240,
|
||
"to_beat": 359,
|
||
"agents_reporting": 978,
|
||
"on_time_reviews": 942,
|
||
"help_promises_fulfilled": 87,
|
||
"secret_rotations_ok": true,
|
||
"tempo_drift_ms": 7.3,
|
||
"issues": []
|
||
}
|
||
```
|
||
|
||
**Key Fields:**
|
||
- `agents_reporting`: Total active agents in window
|
||
- `on_time_reviews`: Agents completing review phase on time
|
||
- `tempo_drift_ms`: Timing drift (positive = behind, negative = ahead)
|
||
|
||
## 🛠️ Usage Examples
|
||
|
||
### Validate Single Message
|
||
|
||
```bash
|
||
# Validate from file
|
||
./backbeat-validate --schemas ../schemas --file message.json
|
||
|
||
# Validate from stdin
|
||
echo '{"type":"backbeat.beatframe.v1",...}' | ./backbeat-validate --schemas ../schemas --message -
|
||
|
||
# Get JSON output for programmatic use
|
||
./backbeat-validate --schemas ../schemas --file message.json --json
|
||
```
|
||
|
||
### Validate Directory
|
||
|
||
```bash
|
||
# Validate all JSON files in directory
|
||
./backbeat-validate --schemas ../schemas --dir messages/
|
||
|
||
# Quiet mode (only errors)
|
||
./backbeat-validate --schemas ../schemas --dir messages/ --quiet
|
||
|
||
# Exit with error code on validation failures
|
||
./backbeat-validate --schemas ../schemas --dir messages/ --exit-code
|
||
```
|
||
|
||
### Go Integration
|
||
|
||
```go
|
||
import "github.com/chorus-services/backbeat/contracts/tests/integration"
|
||
|
||
// Create validator
|
||
validator, err := integration.NewMessageValidator("./schemas")
|
||
if err != nil {
|
||
log.Fatal(err)
|
||
}
|
||
|
||
// Validate message
|
||
result, err := validator.ValidateMessageString(`{"type":"backbeat.beatframe.v1",...}`)
|
||
if err != nil {
|
||
log.Fatal(err)
|
||
}
|
||
|
||
if !result.Valid {
|
||
log.Errorf("Validation failed: %v", result.Errors)
|
||
}
|
||
```
|
||
|
||
## 📊 Tempo Recommendations
|
||
|
||
| Use Case | Tempo (BPM) | Beat Duration | Example Services |
|
||
|----------|-------------|---------------|------------------|
|
||
| **Development** | 0.1 - 0.5 | 2-10 minutes | Testing, debugging |
|
||
| **Batch Processing** | 0.5 - 2.0 | 30s - 2 minutes | ETL, data warehouses |
|
||
| **Standard Services** | 2.0 - 10.0 | 6-30 seconds | APIs, web apps |
|
||
| **Responsive Apps** | 10.0 - 60.0 | 1-6 seconds | Dashboards, monitoring |
|
||
| **High-Frequency** | 60+ | <1 second | Trading, IoT processing |
|
||
|
||
**Default**: 2.0 BPM (30-second beats) works well for most CHORUS services.
|
||
|
||
## 📋 Integration Checklist
|
||
|
||
- [ ] **Message Validation**: Add schema validation to your CI pipeline
|
||
- [ ] **BeatFrame Handler**: Implement INT-A message consumption
|
||
- [ ] **StatusClaim Publisher**: Implement INT-B message publishing (if you have agents)
|
||
- [ ] **BarReport Consumer**: Implement INT-C message consumption (optional)
|
||
- [ ] **Tempo Selection**: Choose appropriate BPM for your workload
|
||
- [ ] **Error Handling**: Handle validation failures and timing issues
|
||
- [ ] **Monitoring**: Track beat processing latency and deadline misses
|
||
- [ ] **Load Testing**: Verify performance at production tempo
|
||
|
||
## 🔄 Schema Versioning
|
||
|
||
Schemas follow [Semantic Versioning](https://semver.org/):
|
||
|
||
- **MAJOR** (1.0.0 → 2.0.0): Breaking changes requiring code updates
|
||
- **MINOR** (1.0.0 → 1.1.0): Backward-compatible additions
|
||
- **PATCH** (1.0.0 → 1.0.1): Documentation and example updates
|
||
|
||
Current versions:
|
||
- **BeatFrame**: v1.0.0 (`backbeat.beatframe.v1`)
|
||
- **StatusClaim**: v1.0.0 (`backbeat.statusclaim.v1`)
|
||
- **BarReport**: v1.0.0 (`backbeat.barreport.v1`)
|
||
|
||
See [schema-evolution.md](docs/schema-evolution.md) for migration strategies.
|
||
|
||
## 🧪 Running Tests
|
||
|
||
```bash
|
||
# Run all tests
|
||
make test
|
||
|
||
# Test schemas are valid JSON
|
||
make test-schemas
|
||
|
||
# Test example messages
|
||
make test-examples
|
||
|
||
# Run Go integration tests
|
||
make test-integration
|
||
|
||
# Validate built-in examples
|
||
make validate-examples
|
||
```
|
||
|
||
## 🏗️ Building
|
||
|
||
```bash
|
||
# Build CLI validation tool
|
||
make build
|
||
|
||
# Install Go dependencies
|
||
make deps
|
||
|
||
# Format code
|
||
make fmt
|
||
|
||
# Run linter
|
||
make lint
|
||
|
||
# Generate CI configuration examples
|
||
make examples
|
||
```
|
||
|
||
## 📚 Documentation
|
||
|
||
- **[Integration Guide](docs/integration-guide.md)**: Complete guide for CHORUS 2.0.0 projects
|
||
- **[Schema Evolution](docs/schema-evolution.md)**: Versioning and compatibility management
|
||
- **[Tempo Guide](docs/tempo-guide.md)**: Beat timing and performance optimization
|
||
|
||
## 🤝 Contributing
|
||
|
||
1. **Fork** this repository
|
||
2. **Create** a feature branch: `git checkout -b feature/amazing-feature`
|
||
3. **Add** tests for your changes
|
||
4. **Run** `make test` to ensure everything passes
|
||
5. **Commit** your changes: `git commit -m 'Add amazing feature'`
|
||
6. **Push** to the branch: `git push origin feature/amazing-feature`
|
||
7. **Open** a Pull Request
|
||
|
||
### Schema Changes
|
||
|
||
- **Minor changes** (new optional fields): Create PR with updated schema
|
||
- **Major changes** (breaking): Discuss in issue first, follow migration process
|
||
- **All changes**: Update examples and tests accordingly
|
||
|
||
## 🔍 Troubleshooting
|
||
|
||
### Common Validation Errors
|
||
|
||
| Error | Cause | Fix |
|
||
|-------|-------|-----|
|
||
| `type field is required` | Missing `type` field | Add correct message type |
|
||
| `hlc must match pattern` | Invalid HLC format | Use `XXXX:XXXX:XXXX` hex format |
|
||
| `window_id must be exactly 32 hex characters` | Wrong window ID | Use 32-character hex string |
|
||
| `phase must be one of: plan, execute, review` | Invalid phase | Use exact phase names |
|
||
| `tempo_bpm must be at least 0.1` | Tempo too low | Use tempo ≥ 0.1 BPM |
|
||
|
||
### Performance Issues
|
||
|
||
- **Beat processing too slow**: Reduce tempo or optimize code
|
||
- **High CPU usage**: Consider lower tempo or horizontal scaling
|
||
- **Network saturation**: Reduce message frequency or size
|
||
- **Memory leaks**: Ensure proper cleanup in beat handlers
|
||
|
||
### Getting Help
|
||
|
||
- **Issues**: [GitHub Issues](https://github.com/chorus-services/backbeat/issues)
|
||
- **Discussions**: [GitHub Discussions](https://github.com/chorus-services/backbeat/discussions)
|
||
- **Documentation**: Check the [docs/](docs/) directory
|
||
- **Examples**: See [tests/examples/](tests/examples/) for message samples
|
||
|
||
## 📜 License
|
||
|
||
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
||
|
||
## 🎵 About BACKBEAT
|
||
|
||
BACKBEAT provides the rhythmic heartbeat that synchronizes distributed systems across CHORUS 2.0.0. Just as musicians use a metronome to stay in time, BACKBEAT keeps your services coordinated and responsive.
|
||
|
||
**Key Benefits:**
|
||
- 🎯 **Predictable Timing**: Know exactly when coordination happens
|
||
- 🔄 **Graceful Coordination**: Services sync without tight coupling
|
||
- 📊 **Health Visibility**: Real-time insight into cluster performance
|
||
- 🛡️ **Fault Tolerance**: Detect and recover from failures quickly
|
||
- ⚡ **Scalable**: Works from development (0.1 BPM) to high-frequency (1000+ BPM)
|
||
|
||
---
|
||
|
||
**Made with ❤️ by the CHORUS 2.0.0 team**
|
||
|
||
*"In rhythm there is coordination, in coordination there is reliability."* |