tony/WHOOSH
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

Add WHOOSH search service with BACKBEAT integration

Browse Source 
Complete implementation:
- Go-based search service with PostgreSQL and Redis backend
- BACKBEAT SDK integration for beat-aware search operations
- Docker containerization with multi-stage builds
- Comprehensive API endpoints for project analysis and search
- Database migrations and schema management
- GITEA integration for repository management
- Team composition analysis and recommendations

Key features:
- Beat-synchronized search operations with timing coordination
- Phase-based operation tracking (started → querying → ranking → completed)
- Docker Swarm deployment configuration
- Health checks and monitoring
- Secure configuration with environment variables

Architecture:
- Microservice design with clean API boundaries
- Background processing for long-running analysis
- Modular internal structure with proper separation of concerns
- Integration with CHORUS ecosystem via BACKBEAT timing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude Code
2025-09-06 11:16:39 +10:00
parent 595b05335d
commit 33676bae6d
29 changed files with 4262 additions and 185 deletions
Download Patch File Download Diff File Expand all files Collapse all files

304
README.md
View File

@@ -1,179 +1,195 @@
# WHOOSH - Autonomous AI Development Teams
**Orchestration platform for self-organizing AI development teams with democratic consensus and P2P collaboration.**
WHOOSH is the orchestration platform for autonomous AI development teams in the CHORUS ecosystem. It transforms from a simple project template tool into a sophisticated system that enables AI agents to form optimal teams, collaborate democratically, and deliver high-quality solutions through consensus-driven development processes.
## 🎯 Overview
## 🎯 MVP Goals
WHOOSH has evolved from a simple project template tool into a sophisticated **Autonomous AI Development Teams Architecture** that enables AI agents to form optimal development teams, collaborate through P2P channels, and deliver high-quality solutions through democratic consensus processes.
The current MVP focuses on:
1. **Single-Agent Execution**: Process `bzzz-task` labeled issues with single-agent teams
2. **GITEA Integration**: Webhook handling for task discovery and PR management
3. **Basic Team Management**: Minimal team state tracking and assignment
4. **SLURP Integration**: Artifact submission and retrieval proxy
5. **Docker Swarm Deployment**: Production-ready containerization
## 🏗️ Architecture
### Core Components
- **🧠 Team Composer**: LLM-powered task analysis and optimal team formation
- **🤖 Agent Self-Organization**: CHORUS agents autonomously discover and apply to teams
- **🔗 P2P Collaboration**: UCXL addressing with structured reasoning (HMMM)
- **🗳️ Democratic Consensus**: Voting systems with quality gates and institutional compliance
- **📦 Knowledge Preservation**: Complete context capture for SLURP with provenance tracking
- **Go Backend**: HTTP server with chi/echo framework, structured logging with zerolog
- **PostgreSQL Database**: Team, agent, and task state management with migrations
- **GITEA Integration**: Webhook processing and API client for issue management
- **Docker Swarm**: Production deployment with secrets management
- **Redis**: Optional caching and session management
### Integration Ecosystem
### MVP Workflow
```
WHOOSH Team Composer → GITEA Team Issues → CHORUS Agent Discovery → P2P Team Channels → SLURP Artifact Submission
```
## 📋 Development Status
**Current Phase**: Foundation & Planning
- ✅ Comprehensive architecture specifications
- ✅ Database schema design
- ✅ API specification
- ✅ Team Composer design
- ✅ CHORUS integration specification
- 🚧 Implementation in progress
1. GITEA webhook receives issue with `bzzz-task` label
2. WHOOSH parses task information and creates team assignment
3. Single-agent executor processes task (stubbed Team Composer)
4. Results submitted via SLURP proxy for artifact preservation
5. PR creation and status updates back to GITEA
## 🚀 Quick Start
### Prerequisites
- Python 3.11+
- PostgreSQL 15+
- Redis 7+
- Docker & Docker Compose
- Access to Ollama models or cloud LLM APIs
### Development Setup
### Local Development
```bash
# Clone repository
git clone https://gitea.chorus.services/tony/WHOOSH.git
cd WHOOSH
# Setup Python environment
uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
# Copy environment configuration
cp .env.example .env
# Edit .env with your configuration
# Setup database
docker-compose up -d postgres redis
python scripts/setup_database.py
# Start with Docker Compose
docker-compose up -d
# Run development server
python -m whoosh.main
# Or run locally
go run ./cmd/whoosh
```
### Production Deployment
```bash
# Setup Docker Swarm secrets
./scripts/setup-secrets.sh
# Deploy to swarm
./scripts/deploy-swarm.sh v0.1.0-mvp
```
## 📋 API Endpoints
### Health & Status
- `GET /health` - Service health check
- `GET /health/ready` - Readiness check with database connection
### Teams (MVP Minimal)
- `GET /api/v1/teams` - List teams
- `POST /api/v1/teams` - Create team (stub)
- `GET /api/v1/teams/{teamID}` - Get team details (stub)
- `PUT /api/v1/teams/{teamID}/status` - Update team status (stub)
### Task Management
- `POST /api/v1/tasks/ingest` - Task ingestion (stub)
- `GET /api/v1/tasks/{taskID}` - Get task details (stub)
### SLURP Integration
- `POST /api/v1/slurp/submit` - Submit artifacts (stub)
- `GET /api/v1/slurp/artifacts/{ucxlAddr}` - Retrieve artifacts (stub)
### CHORUS Integration
- `GET /api/v1/projects/{projectID}/tasks` - List project tasks
- `GET /api/v1/projects/{projectID}/tasks/available` - List available tasks
- `GET /api/v1/projects/{projectID}/repository` - Get project repository info
- `GET /api/v1/projects/{projectID}/tasks/{taskNumber}` - Get specific task
- `POST /api/v1/projects/{projectID}/tasks/{taskNumber}/claim` - Claim task for agent
- `PUT /api/v1/projects/{projectID}/tasks/{taskNumber}/status` - Update task status
- `POST /api/v1/projects/{projectID}/tasks/{taskNumber}/complete` - Complete task
- `POST /api/v1/agents/register` - Register CHORUS agent
- `PUT /api/v1/agents/{agentID}/status` - Update agent status
### Webhooks
- `POST /webhooks/gitea` - GITEA webhook endpoint (implemented)
## 🗄️ Database Schema
### Core Tables (MVP)
- **teams**: Team management and status tracking
- **team_roles**: Available roles (executor, coordinator, reviewer)
- **team_assignments**: Agent-to-team assignments
- **agents**: Minimal agent registry
- **slurp_submissions**: Artifact tracking
## 🔐 Security Features
- **Docker Swarm Secrets**: Sensitive data management
- **SHHH Integration**: Data redaction and encryption
- **JWT Authentication**: Service and user token validation
- **Webhook Signature Validation**: GITEA webhook authenticity
- **Rate Limiting**: API endpoint protection
## 🛠️ Development Commands
```bash
# Build binary
go build ./cmd/whoosh
# Run tests
go test ./...
# Format code
go fmt ./...
# Static analysis
go vet ./...
# Database migrations
migrate -path migrations -database "postgres://..." up
```
## 📊 Monitoring
### Docker Swarm Services
```bash
# Service status
docker service ls --filter label=com.docker.stack.namespace=whoosh
# Service logs
docker service logs -f whoosh_whoosh
# Scale services
docker service scale whoosh_whoosh=3
```
### Health Endpoints
- Health: `https://whoosh.chorus.services/health`
- Ready: `https://whoosh.chorus.services/health/ready`
## 🔄 Future Roadmap
### Post-MVP Features
1. **Team Composer**: LLM-powered task analysis and team formation
2. **P2P Communication**: UCXL addressing and HMMM integration
3. **Agent Self-Organization**: Automatic team application and consensus
4. **Advanced Analytics**: Performance metrics and team effectiveness
5. **Multi-Repository Support**: Cross-project team coordination
### Integration Points
- **CHORUS Agents**: P2P task coordination and execution
- **BZZZ System**: Distributed task management integration
- **SHHH Encryption**: Secure data handling and transmission
- **UCXL Addressing**: Decentralized resource identification
- **SLURP Storage**: Comprehensive artifact preservation
## 📚 Documentation
### Architecture & Design
- [📋 Development Plan](docs/DEVELOPMENT_PLAN.md) - Complete 24-week roadmap
- [🗄️ Database Schema](docs/DATABASE_SCHEMA.md) - Comprehensive data architecture
- [🌐 API Specification](docs/API_SPECIFICATION.md) - Complete REST & WebSocket APIs
### Core Systems
- [🧠 Team Composer](docs/TEAM_COMPOSER_SPEC.md) - LLM-powered team formation engine
- [🤖 CHORUS Integration](docs/CHORUS_INTEGRATION_SPEC.md) - Agent self-organization & P2P collaboration
- [📖 Original Vision](docs/Modules/WHOOSH.md) - Autonomous AI development teams concept
## 🔧 Key Features
### Team Formation
- **Intelligent Analysis**: LLM-powered task complexity and skill requirement analysis
- **Optimal Composition**: Dynamic team sizing with role-based agent matching
- **Risk Assessment**: Comprehensive project risk evaluation and mitigation
- **Timeline Planning**: Automated formation scheduling with contingencies
### Agent Coordination
- **Self-Assessment**: Agents evaluate their own capabilities and availability
- **Opportunity Discovery**: Automated scanning of team formation opportunities
- **Autonomous Applications**: Intelligent team application with value propositions
- **Performance Tracking**: Continuous learning from team outcomes
### Collaboration Systems
- **P2P Channels**: UCXL-addressed team communication channels
- **HMMM Reasoning**: Structured thought processes with evidence and consensus
- **Democratic Voting**: Multiple consensus mechanisms (majority, supermajority, unanimous)
- **Quality Gates**: Institutional compliance with provenance and security validation
### Knowledge Management
- **Context Preservation**: Complete capture of team processes and decisions
- **SLURP Integration**: Automated artifact bundling and submission
- **Decision Rationale**: Comprehensive reasoning chains and consensus records
- **Learning Loop**: Continuous improvement from team performance feedback
## 🛠️ Technology Stack
### Backend
- **Language**: Python 3.11+ with FastAPI
- **Database**: PostgreSQL 15+ with async support
- **Cache**: Redis 7+ for sessions and real-time data
- **LLM Integration**: Ollama + Cloud APIs (OpenAI, Anthropic)
- **P2P**: libp2p for peer-to-peer networking
### Frontend
- **Framework**: React 18 with TypeScript
- **State**: Zustand for complex state management
- **UI**: Tailwind CSS with Headless UI components
- **Real-time**: WebSocket with auto-reconnect
- **Charts**: D3.js for advanced visualizations
### Infrastructure
- **Containers**: Docker with multi-stage builds
- **Orchestration**: Docker Swarm (cluster deployment)
- **Proxy**: Traefik with SSL termination
- **Monitoring**: Prometheus + Grafana
- **CI/CD**: GITEA Actions with automated testing
## 🎯 Roadmap
### Phase 1: Foundation (Weeks 1-4)
- Core infrastructure and Team Composer service
- Database schema implementation
- Basic API endpoints and WebSocket infrastructure
### Phase 2: CHORUS Integration (Weeks 5-8)
- Agent self-organization capabilities
- GITEA team issue integration
- P2P communication infrastructure
### Phase 3: Collaboration Systems (Weeks 9-12)
- Democratic consensus mechanisms
- HMMM reasoning integration
- Team lifecycle management
### Phase 4: SLURP Integration (Weeks 13-16)
- Artifact packaging and submission
- Knowledge preservation systems
- Quality validation pipelines
### Phase 5: Frontend & UX (Weeks 17-20)
- Complete user interface
- Real-time dashboards
- Administrative controls
### Phase 6: Advanced Features (Weeks 21-24)
- Machine learning optimization
- Cloud LLM integration
- Advanced analytics and reporting
- [Development Plan](docs/DEVELOPMENT_PLAN.md) - Comprehensive transformation roadmap
- [Database Schema](docs/DATABASE_SCHEMA.md) - Complete schema documentation
- [API Specification](docs/API_SPECIFICATION.md) - Full API reference
- [Team Composer Spec](docs/TEAM_COMPOSER_SPEC.md) - LLM integration details
## 🤝 Contributing
1. Fork the repository on GITEA
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
WHOOSH follows the CHORUS ecosystem development patterns:
1. Branch from `main` for features
2. Implement with comprehensive tests
3. Update version tags for container builds
4. Deploy to staging for validation
5. Create PR with detailed description
## 📄 License
This project is part of the CHORUS ecosystem and follows the same licensing terms.
## 🔗 Related Projects
- **[CHORUS](https://gitea.chorus.services/tony/CHORUS)** - Distributed AI agent coordination
- **[KACHING](https://gitea.chorus.services/tony/KACHING)** - License management and billing
- **[SLURP](https://gitea.chorus.services/tony/SLURP)** - Knowledge artifact management
- **[BZZZ](https://gitea.chorus.services/tony/BZZZ)** - Original task coordination (legacy)
This project is part of the CHORUS ecosystem. All rights reserved.
---
**WHOOSH** - *Where AI agents become autonomous development teams* 🚀
**WHOOSH** - *Where autonomous AI development teams come together* 🎭