🎵 CHORUS Services - Distributed AI Orchestration Platform
CHORUS Services eliminates context loss, reduces hallucinations, and enables scalable multi-agent collaboration through intelligent context management and distributed reasoning.
🚀 Quick Start
# Initialize submodules (first time only)
./chorus.sh init
# Login to Docker registry
./chorus.sh login
# Build and push images to registry
./chorus.sh build
# Start all services
./chorus.sh start
# Check service health
./chorus.sh health
Access Points:
- 🌐 Marketing Website: https://www.chorus.services (production)
- 🎛️ Dashboard: https://dashboard.chorus.services (production)
- 📡 API: https://api.chorus.services (production)
- 📊 Grafana Monitoring: http://localhost:3002 (admin/chorusadmin)
- 🔍 Prometheus Metrics: http://localhost:9092
Local Development:
- Dashboard: http://localhost:3001
- API Docs: http://localhost:8087/docs
🏗️ Architecture Overview
CHORUS Services integrates five core components into a unified platform:
┌─────────────────────────────────────────────────────────────────┐
│ CHORUS Services Platform │
├─────────────────────────────────────────────────────────────────┤
│ 🎛️ WHOOSH Orchestrator │ 📊 Monitoring & Analytics │
│ - Workflow Management │ - Prometheus Metrics │
│ - Agent Coordination │ - Grafana Dashboards │
│ - Task Distribution │ - Real-time Health Monitoring │
├─────────────────────────────────────────────────────────────────┤
│ 🐝 BZZZ P2P Network │ 🧠 SLURP Context Management │
│ - Agent Mesh Networking │ - Hierarchical Context Storage │
│ - Peer Discovery │ - Semantic Search & Indexing │
│ - Distributed Coordination │ - Multi-language SDK Support │
├─────────────────────────────────────────────────────────────────┤
│ 🎯 COOEE Feedback System (RL Context SLURP Integration) │
│ - Performance-based Learning │ - Context Relevance Tuning │
│ - Agent Feedback Collection │ - Role-based Access Control │
└─────────────────────────────────────────────────────────────────┘
🧩 Core Components
🎛️ WHOOSH - Orchestration Engine
- Enterprise workflow management for AI agents
- Visual workflow editor with React Flow
- Real-time performance monitoring and metrics
- Multi-agent task distribution and coordination
🐝 BZZZ - P2P Agent Coordination
- Mesh networking with libp2p for resilient communication
- Automatic peer discovery via mDNS
- Distributed task coordination without single points of failure
- Go-based high-performance networking layer
🧠 SLURP - Context Curator Service
- Context curation from Hypercore logs based on agent roles and triggers
- Role-based context filtering for permissions, deprecation, feature changes
- SQL-based context delivery with intelligent relevance scoring
- Integration with HCFS for transparent filesystem-based context access
🎯 COOEE - Feedback & Learning (RL Context SLURP)
- Reinforcement learning for context relevance tuning
- Agent feedback collection with upvote/downvote systems
- Role-based context filtering and access control
- Continuous improvement through real-world performance data
📊 Monitoring & Analytics
- Prometheus metrics collection across all services
- Grafana dashboards for visualization and alerting
- Health checks and performance monitoring
- Audit trails with complete traceability
🛠️ Management Commands
The ./chorus.sh script provides unified management:
# Service Management
./chorus.sh start # Start all services (registry images)
./chorus.sh stop # Stop all services
./chorus.sh restart # Restart all services
./chorus.sh status # Show service status
./chorus.sh dev # Start in development mode (local builds)
# Docker Registry Operations
./chorus.sh login # Login to Docker registry
./chorus.sh build # Build and push all images to registry
./chorus.sh pull # Pull latest images from registry
# Individual Component Builds
./build-and-push.sh website # Build and push website only
./build-and-push.sh whoosh-backend # Build and push WHOOSH backend only
./build-and-push.sh bzzz # Build and push BZZZ coordinator only
# Production Deployment
./chorus.sh deploy # Deploy to Docker Swarm (production)
./chorus.sh undeploy # Remove from Docker Swarm
# Development & Maintenance
./chorus.sh update # Update submodules to latest
./chorus.sh logs [service] # View logs
./chorus.sh health # Check service health
./chorus.sh clean # Clean up resources
# First-time Setup
./chorus.sh init # Initialize git submodules
🌐 Service Endpoints
| Service | Port | Purpose | Health Check |
|---|---|---|---|
| WHOOSH Dashboard | 3001 | Web UI for orchestration | http://localhost:3001 |
| WHOOSH API | 8087 | REST API + WebSocket | http://localhost:8087/health |
| BZZZ Coordinator | 8080 | P2P coordination API | http://localhost:8080/health |
| SLURP API | 8088 | Context management API | http://localhost:8088/health |
| COOEE RL Tuner | 8089 | Feedback & learning API | http://localhost:8089/health |
| Grafana | 3002 | Monitoring dashboards | http://localhost:3002 |
| Prometheus | 9092 | Metrics collection | http://localhost:9092 |
| PostgreSQL | 5433 | Database | - |
| Redis | 6380 | Cache & message queue | - |
📁 Project Structure
chorus.services/
├── 📋 README.md # This file
├── 🎛️ chorus.sh # Management script
├── 🐳 docker-compose.yml # Service orchestration
├── 🗄️ init-db.sql # Database initialization
├── 📊 monitoring/ # Monitoring configuration
│ ├── prometheus.yml
│ └── grafana/
├── 📚 docs/ # Project documentation
│ ├── PROJECT_PLAN.md
│ ├── DESIGN.md
│ └── Copywriting.md
└── 🧩 modules/ # Git submodules
├── whoosh/ # Orchestration platform
├── bzzz/ # P2P coordination
├── slurp/ # Context management (HCFS)
└── website/ # Marketing website (Next.js)
⚙️ Configuration
Environment Variables
Key configuration options in docker-compose.yml:
- Database:
DATABASE_URL,POSTGRES_* - Redis:
REDIS_URL - CORS:
CORS_ORIGINS - Logging:
LOG_LEVEL - Environment:
ENVIRONMENT(development/production)
Adding New Agents
Edit agent configurations in modules/whoosh/config/whoosh.yaml:
whoosh:
agents:
my_agent:
name: "My Agent"
endpoint: "http://192.168.1.100:11434"
model: "llama3.1"
specialization: "coding"
capabilities: ["python", "javascript"]
🔧 Development Setup
Prerequisites
- Docker and Docker Compose
- Git
- 8GB+ RAM recommended
- Access to Ollama agents on network
First-Time Setup
# Clone with submodules
git clone --recursive <repository-url>
cd chorus.services
# Or initialize submodules if already cloned
./chorus.sh init
# Login to Docker registry
./chorus.sh login
# Build and push all images to registry
./chorus.sh build
# Start services
./chorus.sh start
Development Workflow
# Development mode (local builds, live reloading)
./chorus.sh dev
# Update submodules to latest
./chorus.sh update
# Rebuild and push after changes
./chorus.sh build
# Pull latest images from registry
./chorus.sh pull
# View logs during development
./chorus.sh logs
# Check service health
./chorus.sh health
# Restart after changes
./chorus.sh restart
Production Deployment
# Deploy to Docker Swarm (production)
./chorus.sh deploy
# Access at https://*.home.deepblack.cloud endpoints
# Remove from swarm
./chorus.sh undeploy
🚀 Git Submodules Guide
CHORUS Services uses git submodules to integrate independent components:
Basic Submodule Commands
# Initialize submodules (first time)
git submodule init
git submodule update
# Update to latest commits
git submodule update --remote
# Check submodule status
git submodule status
# Enter a submodule to work on it
cd modules/whoosh
git checkout main
# Make changes, commit, push
# Return to main project and commit submodule updates
cd ../..
git add modules/whoosh
git commit -m "Update whoosh submodule"
Working with Submodules
- Each submodule is an independent git repository
- Changes within submodules must be committed in the submodule first
- Parent project tracks specific commits of each submodule
- Use
./chorus.sh updateto pull latest changes from all submodules
📊 Monitoring & Metrics
Key Metrics Tracked
- Agent Performance: Response time, throughput, availability
- Context Management: Search performance, storage efficiency
- P2P Network: Peer connectivity, message latency
- System Health: CPU, memory, GPU utilization
- Workflow Execution: Success rate, execution time
Grafana Dashboards
- CHORUS Overview: Platform-wide health and metrics
- Agent Performance: Individual agent monitoring
- Context Analytics: SLURP usage and performance
- Network Health: BZZZ P2P network monitoring
🛡️ Security Features
- Authentication: JWT tokens and API key support
- Role-based Access: Context filtering by agent roles
- Audit Trails: Complete logging of all operations
- Network Security: Internal container networking
- Data Privacy: Encrypted context storage
🤝 Contributing
Component Development
Each component can be developed independently:
# Work on WHOOSH orchestrator
cd modules/whoosh
# Follow component-specific development guide
# Work on BZZZ P2P system
cd modules/bzzz
# Follow Go development practices
# Work on SLURP context system
cd modules/slurp
# Follow Python development practices
Integration Testing
# Test full platform integration
./chorus.sh start
./chorus.sh health
# Run component-specific tests
cd modules/[component]
# Follow component test procedures
📈 Performance Metrics
Production-Ready Performance:
- API Response Times: <5ms cached, <50ms uncached
- Context Search: <100ms semantic search across 1000+ contexts
- P2P Network: <10ms peer communication latency
- Workflow Execution: Support for complex multi-agent workflows
- Concurrent Agents: Scales to 10+ simultaneous agents
🎯 Use Cases
Enterprise AI Development
- Multi-agent software development with specialized AI agents
- Context-aware code generation with organizational knowledge
- Distributed task execution across development infrastructure
Research & Collaboration
- AI agent coordination research with real-world deployment
- Context management studies with hierarchical storage
- Distributed systems research with P2P networking
Production AI Systems
- Enterprise AI orchestration with monitoring and compliance
- Context-aware AI applications with persistent memory
- Scalable multi-agent deployments with automatic coordination
📞 Support & Documentation
- 🛠️ Management:
./chorus.shfor all operations - 📋 Component Docs: See individual
modules/*/README.md - 🔧 API Documentation: http://localhost:8087/docs (when running)
- 📊 Monitoring: http://localhost:3002 (Grafana dashboards)
🎉 Welcome to CHORUS Services!
CHORUS Services represents the future of distributed AI orchestration, providing enterprise-ready tools for context management, agent coordination, and intelligent workflow execution.
🎵 "Individual components make music, but CHORUS Services creates symphony."
Ready to orchestrate your AI agents?
./chorus.sh start