backbeat: add module sources
This commit is contained in:
anthonyrawlins
366
contracts/README.md
Normal file
366
contracts/README.md
Normal file
@@ -0,0 +1,366 @@
|
||||
# BACKBEAT Contracts Package
|
||||
|
||||
[](https://github.com/chorus-services/backbeat/actions/workflows/contracts.yml)
|
||||
[](schemas/)
|
||||
[](LICENSE)
|
||||
|
||||
The authoritative contract definitions and validation tools for BACKBEAT distributed orchestration across the CHORUS 2.0.0 ecosystem.
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
BACKBEAT provides synchronized distributed execution through three core message interfaces:
|
||||
|
||||
- **INT-A (BeatFrame)**: 🥁 Rhythm coordination from Pulse → All Services
|
||||
- **INT-B (StatusClaim)**: 📊 Agent status reporting from Agents → Reverb
|
||||
- **INT-C (BarReport)**: 📈 Periodic summaries from Reverb → All Services
|
||||
|
||||
This contracts package ensures all CHORUS 2.0.0 projects can reliably integrate with BACKBEAT through:
|
||||
|
||||
✅ **JSON Schema Validation** - Semver-versioned schemas for all interfaces
|
||||
✅ **Conformance Testing** - Comprehensive test suites with valid/invalid examples
|
||||
✅ **CI Integration** - Drop-in validation for any CI pipeline
|
||||
✅ **Documentation** - Complete integration guides and best practices
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### 1. Validate Your Messages
|
||||
|
||||
```bash
|
||||
# Clone the contracts repository
|
||||
git clone https://github.com/chorus-services/backbeat.git
|
||||
cd backbeat/contracts
|
||||
|
||||
# Build the validation tool
|
||||
cd tests/integration && make build
|
||||
|
||||
# Validate your BACKBEAT messages
|
||||
./backbeat-validate --schemas ../../schemas --dir /path/to/your/messages --exit-code
|
||||
```
|
||||
|
||||
### 2. Add to CI Pipeline
|
||||
|
||||
#### GitHub Actions
|
||||
```yaml
|
||||
- name: Validate BACKBEAT Contracts
|
||||
run: |
|
||||
git clone https://github.com/chorus-services/backbeat.git
|
||||
cd backbeat/contracts/tests/integration
|
||||
make build
|
||||
./backbeat-validate --schemas ../../schemas --dir ${{ github.workspace }}/messages --exit-code
|
||||
```
|
||||
|
||||
#### GitLab CI
|
||||
```yaml
|
||||
validate-backbeat:
|
||||
script:
|
||||
- git clone https://github.com/chorus-services/backbeat.git
|
||||
- cd backbeat/contracts/tests/integration && make build
|
||||
- ./backbeat-validate --schemas ../../schemas --dir messages --exit-code
|
||||
```
|
||||
|
||||
### 3. Integrate with Your Project
|
||||
|
||||
Add to your `Makefile`:
|
||||
```makefile
|
||||
validate-backbeat:
|
||||
@git clone https://github.com/chorus-services/backbeat.git .backbeat 2>/dev/null || true
|
||||
@cd .backbeat/contracts/tests/integration && make build
|
||||
@.backbeat/contracts/tests/integration/backbeat-validate --schemas .backbeat/contracts/schemas --dir messages --exit-code
|
||||
```
|
||||
|
||||
## 📁 Package Structure
|
||||
|
||||
```
|
||||
contracts/
|
||||
├── schemas/ # JSON Schema definitions
|
||||
│ ├── beatframe-v1.schema.json # INT-A: Pulse → All Services
|
||||
│ ├── statusclaim-v1.schema.json # INT-B: Agents → Reverb
|
||||
│ └── barreport-v1.schema.json # INT-C: Reverb → All Services
|
||||
├── tests/
|
||||
│ ├── conformance_test.go # Go conformance test suite
|
||||
│ ├── examples/ # Valid/invalid message examples
|
||||
│ │ ├── beatframe-valid.json
|
||||
│ │ ├── beatframe-invalid.json
|
||||
│ │ ├── statusclaim-valid.json
|
||||
│ │ ├── statusclaim-invalid.json
|
||||
│ │ ├── barreport-valid.json
|
||||
│ │ └── barreport-invalid.json
|
||||
│ └── integration/ # CI integration helpers
|
||||
│ ├── validator.go # Message validation library
|
||||
│ ├── ci_helper.go # CI integration utilities
|
||||
│ ├── cmd/backbeat-validate/ # CLI validation tool
|
||||
│ └── Makefile # Build and test automation
|
||||
├── docs/
|
||||
│ ├── integration-guide.md # How to BACKBEAT-enable services
|
||||
│ ├── schema-evolution.md # Versioning and compatibility
|
||||
│ └── tempo-guide.md # Beat timing recommendations
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## 🔧 Core Interfaces
|
||||
|
||||
### INT-A: BeatFrame (Pulse → All Services)
|
||||
|
||||
Synchronization messages broadcast every beat:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "backbeat.beatframe.v1",
|
||||
"cluster_id": "chorus-prod",
|
||||
"beat_index": 1337,
|
||||
"downbeat": false,
|
||||
"phase": "execute",
|
||||
"hlc": "7ffd:0001:abcd",
|
||||
"deadline_at": "2025-09-05T12:30:00Z",
|
||||
"tempo_bpm": 2.0,
|
||||
"window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5"
|
||||
}
|
||||
```
|
||||
|
||||
**Key Fields:**
|
||||
- `beat_index`: Monotonic counter since cluster start
|
||||
- `phase`: `"plan"`, `"execute"`, or `"review"`
|
||||
- `tempo_bpm`: Current beats per minute (default: 2.0 = 30-second beats)
|
||||
- `deadline_at`: When this phase must complete
|
||||
|
||||
### INT-B: StatusClaim (Agents → Reverb)
|
||||
|
||||
Agent status reports during beat execution:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "backbeat.statusclaim.v1",
|
||||
"agent_id": "search-indexer:worker-03",
|
||||
"task_id": "index-batch:20250905-120",
|
||||
"beat_index": 1337,
|
||||
"state": "executing",
|
||||
"beats_left": 3,
|
||||
"progress": 0.65,
|
||||
"notes": "processing batch 120/200",
|
||||
"hlc": "7ffd:0001:beef"
|
||||
}
|
||||
```
|
||||
|
||||
**Key Fields:**
|
||||
- `state`: `"idle"`, `"planning"`, `"executing"`, `"reviewing"`, `"completed"`, `"failed"`, `"blocked"`, `"helping"`
|
||||
- `beats_left`: Estimated beats to completion
|
||||
- `progress`: Completion percentage (0.0 - 1.0)
|
||||
|
||||
### INT-C: BarReport (Reverb → All Services)
|
||||
|
||||
Periodic cluster health summaries:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "backbeat.barreport.v1",
|
||||
"window_id": "7e9b0e6c4c9a4e59b7f2d9a3c1b2e4d5",
|
||||
"from_beat": 240,
|
||||
"to_beat": 359,
|
||||
"agents_reporting": 978,
|
||||
"on_time_reviews": 942,
|
||||
"help_promises_fulfilled": 87,
|
||||
"secret_rotations_ok": true,
|
||||
"tempo_drift_ms": 7.3,
|
||||
"issues": []
|
||||
}
|
||||
```
|
||||
|
||||
**Key Fields:**
|
||||
- `agents_reporting`: Total active agents in window
|
||||
- `on_time_reviews`: Agents completing review phase on time
|
||||
- `tempo_drift_ms`: Timing drift (positive = behind, negative = ahead)
|
||||
|
||||
## 🛠️ Usage Examples
|
||||
|
||||
### Validate Single Message
|
||||
|
||||
```bash
|
||||
# Validate from file
|
||||
./backbeat-validate --schemas ../schemas --file message.json
|
||||
|
||||
# Validate from stdin
|
||||
echo '{"type":"backbeat.beatframe.v1",...}' | ./backbeat-validate --schemas ../schemas --message -
|
||||
|
||||
# Get JSON output for programmatic use
|
||||
./backbeat-validate --schemas ../schemas --file message.json --json
|
||||
```
|
||||
|
||||
### Validate Directory
|
||||
|
||||
```bash
|
||||
# Validate all JSON files in directory
|
||||
./backbeat-validate --schemas ../schemas --dir messages/
|
||||
|
||||
# Quiet mode (only errors)
|
||||
./backbeat-validate --schemas ../schemas --dir messages/ --quiet
|
||||
|
||||
# Exit with error code on validation failures
|
||||
./backbeat-validate --schemas ../schemas --dir messages/ --exit-code
|
||||
```
|
||||
|
||||
### Go Integration
|
||||
|
||||
```go
|
||||
import "github.com/chorus-services/backbeat/contracts/tests/integration"
|
||||
|
||||
// Create validator
|
||||
validator, err := integration.NewMessageValidator("./schemas")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// Validate message
|
||||
result, err := validator.ValidateMessageString(`{"type":"backbeat.beatframe.v1",...}`)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
if !result.Valid {
|
||||
log.Errorf("Validation failed: %v", result.Errors)
|
||||
}
|
||||
```
|
||||
|
||||
## 📊 Tempo Recommendations
|
||||
|
||||
| Use Case | Tempo (BPM) | Beat Duration | Example Services |
|
||||
|----------|-------------|---------------|------------------|
|
||||
| **Development** | 0.1 - 0.5 | 2-10 minutes | Testing, debugging |
|
||||
| **Batch Processing** | 0.5 - 2.0 | 30s - 2 minutes | ETL, data warehouses |
|
||||
| **Standard Services** | 2.0 - 10.0 | 6-30 seconds | APIs, web apps |
|
||||
| **Responsive Apps** | 10.0 - 60.0 | 1-6 seconds | Dashboards, monitoring |
|
||||
| **High-Frequency** | 60+ | <1 second | Trading, IoT processing |
|
||||
|
||||
**Default**: 2.0 BPM (30-second beats) works well for most CHORUS services.
|
||||
|
||||
## 📋 Integration Checklist
|
||||
|
||||
- [ ] **Message Validation**: Add schema validation to your CI pipeline
|
||||
- [ ] **BeatFrame Handler**: Implement INT-A message consumption
|
||||
- [ ] **StatusClaim Publisher**: Implement INT-B message publishing (if you have agents)
|
||||
- [ ] **BarReport Consumer**: Implement INT-C message consumption (optional)
|
||||
- [ ] **Tempo Selection**: Choose appropriate BPM for your workload
|
||||
- [ ] **Error Handling**: Handle validation failures and timing issues
|
||||
- [ ] **Monitoring**: Track beat processing latency and deadline misses
|
||||
- [ ] **Load Testing**: Verify performance at production tempo
|
||||
|
||||
## 🔄 Schema Versioning
|
||||
|
||||
Schemas follow [Semantic Versioning](https://semver.org/):
|
||||
|
||||
- **MAJOR** (1.0.0 → 2.0.0): Breaking changes requiring code updates
|
||||
- **MINOR** (1.0.0 → 1.1.0): Backward-compatible additions
|
||||
- **PATCH** (1.0.0 → 1.0.1): Documentation and example updates
|
||||
|
||||
Current versions:
|
||||
- **BeatFrame**: v1.0.0 (`backbeat.beatframe.v1`)
|
||||
- **StatusClaim**: v1.0.0 (`backbeat.statusclaim.v1`)
|
||||
- **BarReport**: v1.0.0 (`backbeat.barreport.v1`)
|
||||
|
||||
See [schema-evolution.md](docs/schema-evolution.md) for migration strategies.
|
||||
|
||||
## 🧪 Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
make test
|
||||
|
||||
# Test schemas are valid JSON
|
||||
make test-schemas
|
||||
|
||||
# Test example messages
|
||||
make test-examples
|
||||
|
||||
# Run Go integration tests
|
||||
make test-integration
|
||||
|
||||
# Validate built-in examples
|
||||
make validate-examples
|
||||
```
|
||||
|
||||
## 🏗️ Building
|
||||
|
||||
```bash
|
||||
# Build CLI validation tool
|
||||
make build
|
||||
|
||||
# Install Go dependencies
|
||||
make deps
|
||||
|
||||
# Format code
|
||||
make fmt
|
||||
|
||||
# Run linter
|
||||
make lint
|
||||
|
||||
# Generate CI configuration examples
|
||||
make examples
|
||||
```
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
- **[Integration Guide](docs/integration-guide.md)**: Complete guide for CHORUS 2.0.0 projects
|
||||
- **[Schema Evolution](docs/schema-evolution.md)**: Versioning and compatibility management
|
||||
- **[Tempo Guide](docs/tempo-guide.md)**: Beat timing and performance optimization
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
1. **Fork** this repository
|
||||
2. **Create** a feature branch: `git checkout -b feature/amazing-feature`
|
||||
3. **Add** tests for your changes
|
||||
4. **Run** `make test` to ensure everything passes
|
||||
5. **Commit** your changes: `git commit -m 'Add amazing feature'`
|
||||
6. **Push** to the branch: `git push origin feature/amazing-feature`
|
||||
7. **Open** a Pull Request
|
||||
|
||||
### Schema Changes
|
||||
|
||||
- **Minor changes** (new optional fields): Create PR with updated schema
|
||||
- **Major changes** (breaking): Discuss in issue first, follow migration process
|
||||
- **All changes**: Update examples and tests accordingly
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Common Validation Errors
|
||||
|
||||
| Error | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| `type field is required` | Missing `type` field | Add correct message type |
|
||||
| `hlc must match pattern` | Invalid HLC format | Use `XXXX:XXXX:XXXX` hex format |
|
||||
| `window_id must be exactly 32 hex characters` | Wrong window ID | Use 32-character hex string |
|
||||
| `phase must be one of: plan, execute, review` | Invalid phase | Use exact phase names |
|
||||
| `tempo_bpm must be at least 0.1` | Tempo too low | Use tempo ≥ 0.1 BPM |
|
||||
|
||||
### Performance Issues
|
||||
|
||||
- **Beat processing too slow**: Reduce tempo or optimize code
|
||||
- **High CPU usage**: Consider lower tempo or horizontal scaling
|
||||
- **Network saturation**: Reduce message frequency or size
|
||||
- **Memory leaks**: Ensure proper cleanup in beat handlers
|
||||
|
||||
### Getting Help
|
||||
|
||||
- **Issues**: [GitHub Issues](https://github.com/chorus-services/backbeat/issues)
|
||||
- **Discussions**: [GitHub Discussions](https://github.com/chorus-services/backbeat/discussions)
|
||||
- **Documentation**: Check the [docs/](docs/) directory
|
||||
- **Examples**: See [tests/examples/](tests/examples/) for message samples
|
||||
|
||||
## 📜 License
|
||||
|
||||
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
||||
|
||||
## 🎵 About BACKBEAT
|
||||
|
||||
BACKBEAT provides the rhythmic heartbeat that synchronizes distributed systems across CHORUS 2.0.0. Just as musicians use a metronome to stay in time, BACKBEAT keeps your services coordinated and responsive.
|
||||
|
||||
**Key Benefits:**
|
||||
- 🎯 **Predictable Timing**: Know exactly when coordination happens
|
||||
- 🔄 **Graceful Coordination**: Services sync without tight coupling
|
||||
- 📊 **Health Visibility**: Real-time insight into cluster performance
|
||||
- 🛡️ **Fault Tolerance**: Detect and recover from failures quickly
|
||||
- ⚡ **Scalable**: Works from development (0.1 BPM) to high-frequency (1000+ BPM)
|
||||
|
||||
---
|
||||
|
||||
**Made with ❤️ by the CHORUS 2.0.0 team**
|
||||
|
||||
*"In rhythm there is coordination, in coordination there is reliability."*
|
||||
Reference in New Issue
Block a user