diff --git a/README.md b/README.md index 1d36166..22e8dbb 100644 --- a/README.md +++ b/README.md @@ -1,116 +1,329 @@ # WHOOSH - Autonomous AI Development Teams -WHOOSH is the orchestration platform for autonomous AI development teams in the CHORUS ecosystem. It transforms from a simple project template tool into a sophisticated system that enables AI agents to form optimal teams, collaborate democratically, and deliver high-quality solutions through consensus-driven development processes. +WHOOSH is the orchestration platform for autonomous AI development teams in the CHORUS ecosystem. It transforms project kickoffs through intelligent **Council Formation**, where specialized AI agents collaborate democratically to create comprehensive project foundations and deliver high-quality solutions through consensus-driven processes. -## 🎯 MVP Goals +## 🎯 Production Status -The current MVP focuses on: +**Current Phase**: **PRODUCTION READY** ✅ -1. **Single-Agent Execution**: Process `bzzz-task` labeled issues with single-agent teams -2. **GITEA Integration**: Webhook handling for task discovery and PR management -3. **Basic Team Management**: Minimal team state tracking and assignment -4. **SLURP Integration**: Artifact submission and retrieval proxy -5. **Docker Swarm Deployment**: Production-ready containerization +**Council Formation Engine - Fully Implemented:** +- ✅ **Enterprise Security**: JWT authentication, CORS, input validation, rate limiting +- ✅ **Intelligent Council Composition**: Role-based agent deployment from human-roles.yaml +- ✅ **Full API Coverage**: Complete council lifecycle management with artifacts +- ✅ **Production Observability**: OpenTelemetry tracing, comprehensive health monitoring +- ✅ **Configurable Architecture**: All endpoints environment-variable driven +- ✅ **Performance Optimized**: Database indexes, efficient P2P discovery +- 🔄 **Next Phase**: Advanced team evolution for ongoing development tasks ## 🏗️ Architecture ### Core Components -- **Go Backend**: HTTP server with chi/echo framework, structured logging with zerolog -- **PostgreSQL Database**: Team, agent, and task state management with migrations -- **GITEA Integration**: Webhook processing and API client for issue management -- **Docker Swarm**: Production deployment with secrets management -- **Redis**: Optional caching and session management +- **Go Backend**: HTTP server with chi framework, structured logging with zerolog, OpenTelemetry tracing +- **PostgreSQL Database**: Council, agent, task, and artifact management with optimized indexes +- **Gitea Integration**: Webhook processing and API client for Design Brief detection +- **Docker Swarm**: Production deployment with CHORUS agent orchestration +- **P2P Network**: Production-ready service discovery and inter-agent communication +- **Security Stack**: JWT auth, CORS, input validation, rate limiting, security headers -### MVP Workflow +### Council Formation Workflow -1. GITEA webhook receives issue with `bzzz-task` label -2. WHOOSH parses task information and creates team assignment -3. Single-agent executor processes task (stubbed Team Composer) -4. Results submitted via SLURP proxy for artifact preservation -5. PR creation and status updates back to GITEA +1. **Detection**: Gitea webhook receives "Design Brief" issue with `chorus-entrypoint` label +2. **Analysis**: WHOOSH analyzes project requirements and constraints +3. **Composition**: Intelligent council formation using role definitions from human-roles.yaml +4. **Deployment**: CHORUS agents deployed via Docker Swarm with role-specific configuration +5. **Collaboration**: Agents communicate via P2P network using HMMM protocol +6. **Artifacts**: Council produces kickoff deliverables (manifests, DRs, scaffold plans) +7. **Handoff**: Council artifacts inform subsequent development team formation ## 🚀 Quick Start -### Local Development +### Prerequisites + +- Docker Swarm cluster +- PostgreSQL database +- Gitea instance +- CHORUS stack deployed + +### Production Deployment ```bash # Clone repository git clone https://gitea.chorus.services/tony/WHOOSH.git cd WHOOSH -# Copy environment configuration +# Setup environment configuration cp .env.example .env -# Edit .env with your configuration +# Edit .env with your production settings (see Configuration section) -# Start with Docker Compose -docker-compose up -d +# Setup Docker Swarm secrets +echo "your-jwt-secret" | docker secret create whoosh_jwt_secret - +echo "your-gitea-token" | docker secret create whoosh_gitea_token - +echo "your-webhook-secret" | docker secret create whoosh_webhook_secret - -# Or run locally +# Deploy to CHORUS stack +docker stack deploy -c docker-compose.swarm.yml CHORUS +``` + +### Local Development + +```bash +# Install dependencies +go mod download + +# Setup local database +docker-compose up -d postgres + +# Run migrations +migrate -path migrations -database "postgres://whoosh:password@localhost:5432/whoosh?sslmode=disable" up + +# Start development server go run ./cmd/whoosh ``` -### Production Deployment +## ⚙️ Configuration + +Copy and customize the environment configuration: ```bash -# Setup Docker Swarm secrets -./scripts/setup-secrets.sh - -# Deploy to swarm -./scripts/deploy-swarm.sh v0.1.0-mvp +cp .env.example .env +# Edit .env with your specific settings ``` -## 📋 API Endpoints +**Core Configuration Options:** -### Health & Status -- `GET /health` - Service health check -- `GET /health/ready` - Readiness check with database connection +```bash +# Database +WHOOSH_DATABASE_URL=postgres://user:pass@localhost/whoosh -### Teams (MVP Minimal) -- `GET /api/v1/teams` - List teams -- `POST /api/v1/teams` - Create team (stub) -- `GET /api/v1/teams/{teamID}` - Get team details (stub) -- `PUT /api/v1/teams/{teamID}/status` - Update team status (stub) +# Gitea Integration +WHOOSH_GITEA_URL=https://your-gitea-instance.com +WHOOSH_GITEA_TOKEN_FILE=/path/to/gitea-token +WHOOSH_GITEA_EAGER_FILTER=true # API-level vs code-level filtering +WHOOSH_GITEA_FULL_RESCAN=false # Complete vs incremental scanning -### Task Management -- `POST /api/v1/tasks/ingest` - Task ingestion (stub) -- `GET /api/v1/tasks/{taskID}` - Get task details (stub) +# External Integrations +WHOOSH_N8N_BASE_URL=https://n8n.your-domain.com # Configurable webhook URL -### SLURP Integration -- `POST /api/v1/slurp/submit` - Submit artifacts (stub) -- `GET /api/v1/slurp/artifacts/{ucxlAddr}` - Retrieve artifacts (stub) +# Security & CORS +WHOOSH_CORS_ALLOWED_ORIGINS=https://app.your-domain.com,https://admin.your-domain.com +WHOOSH_JWT_SECRET_FILE=/path/to/jwt-secret +WHOOSH_WEBHOOK_SECRET_FILE=/path/to/webhook-secret -### CHORUS Integration -- `GET /api/v1/projects/{projectID}/tasks` - List project tasks -- `GET /api/v1/projects/{projectID}/tasks/available` - List available tasks -- `GET /api/v1/projects/{projectID}/repository` - Get project repository info -- `GET /api/v1/projects/{projectID}/tasks/{taskNumber}` - Get specific task -- `POST /api/v1/projects/{projectID}/tasks/{taskNumber}/claim` - Claim task for agent -- `PUT /api/v1/projects/{projectID}/tasks/{taskNumber}/status` - Update task status -- `POST /api/v1/projects/{projectID}/tasks/{taskNumber}/complete` - Complete task -- `POST /api/v1/agents/register` - Register CHORUS agent -- `PUT /api/v1/agents/{agentID}/status` - Update agent status +# Feature Flags +WHOOSH_FEATURE_LLM_CLASSIFICATION=false # AI vs heuristic classification +WHOOSH_FEATURE_LLM_SKILL_ANALYSIS=false # AI skill analysis -### Webhooks -- `POST /webhooks/gitea` - GITEA webhook endpoint (implemented) +# Docker Swarm +WHOOSH_DOCKER_ENABLED=true + +# Observability +WHOOSH_BACKBEAT_URL=http://backbeat:3001 +WHOOSH_OTEL_ENABLED=true # OpenTelemetry tracing +``` + +> **📖 Complete Configuration Guide**: See [`docs/CONFIGURATION.md`](docs/CONFIGURATION.md) for all available options. + +### Setup Steps + +1. **Deploy WHOOSH to CHORUS Stack**: + ```bash + docker stack deploy -c docker-compose.swarm.yml CHORUS + ``` + +2. **Configure Repository Monitoring**: + ```bash + curl -X POST http://localhost:8080/api/repositories \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer ${JWT_TOKEN}" \ + -d '{"full_name": "username/repository", "gitea_id": 123}' + ``` + +3. **Verify System Health**: + ```bash + curl http://localhost:8080/admin/health/details + ``` + +### API Testing + +Check system health: +```bash +curl http://localhost:8080/health +``` + +Get detailed health information: +```bash +curl http://localhost:8080/admin/health/details +``` + +View council status: +```bash +curl -H "Authorization: Bearer ${JWT_TOKEN}" \ + http://localhost:8080/api/v1/councils +``` + +Get specific council details: +```bash +curl -H "Authorization: Bearer ${JWT_TOKEN}" \ + http://localhost:8080/api/v1/councils/{council_id} +``` + +Manage council artifacts: +```bash +# Create artifact +curl -X POST -H "Authorization: Bearer ${JWT_TOKEN}" \ + -H "Content-Type: application/json" \ + -d '{"artifact_type": "kickoff_manifest", "content": "..."}' \ + http://localhost:8080/api/v1/councils/{council_id}/artifacts + +# List artifacts +curl -H "Authorization: Bearer ${JWT_TOKEN}" \ + http://localhost:8080/api/v1/councils/{council_id}/artifacts +``` + +## 📊 Monitoring & Observability + +### Health Endpoints + +- `/health` - Basic service health +- `/admin/health/details` - Comprehensive component status (DB, Gitea, BackBeat, Docker) +- `/metrics` - Prometheus metrics +- `/debug/vars` - Runtime statistics + +### Distributed Tracing + +WHOOSH includes comprehensive OpenTelemetry integration: +- **Monitor Cycles**: Repository scanning and issue processing spans +- **Council Formation**: Role assignment and composition tracking spans +- **Agent Deployment**: Docker Swarm service creation spans +- **Webhook Processing**: Request/response correlation spans +- **Correlation IDs**: `goal.id` and `pulse.id` attributes for end-to-end tracing + +### Structured Logging + +Enhanced logging with correlation: +- **Request IDs**: HTTP request tracing +- **Council IDs**: Council lifecycle tracking +- **Agent IDs**: Individual agent events +- **Span IDs**: OpenTelemetry trace correlation +- **Performance Metrics**: Operation timing and resource usage + +### BackBeat Integration + +Performance monitoring via BackBeat SDK: +```bash +# View performance metrics +curl http://localhost:8080/api/backbeat/status +``` + +### Monitoring Stack Integration + +**Recommended Tools:** +- **Jaeger/Tempo**: Distributed tracing visualization +- **Prometheus**: Metrics collection and alerting +- **Grafana**: Dashboards and visualization +- **Loki**: Log aggregation and correlation + +## 🔐 Security + +### Enterprise-Grade Security + +WHOOSH implements comprehensive production security: + +**✅ Authentication & Authorization:** +- **JWT Authentication**: Role-based access control for web and API access +- **Service Tokens**: Secure API automation with scoped permissions +- **Admin Endpoint Protection**: All administrative operations require authentication + +**✅ Input Validation & Rate Limiting:** +- **Request Validation**: Comprehensive input sanitization with regex patterns +- **Size Limits**: Configurable request body size limits (1MB default, 10MB webhooks) +- **Rate Limiting**: Per-IP rate limiting (100 requests/minute, configurable) + +**✅ Security Headers:** +- **CORS**: Configurable domain restrictions (no wildcards in production) +- **CSP**: Content Security Policy headers +- **Security Headers**: X-Frame-Options, X-Content-Type-Options, X-XSS-Protection + +**✅ Webhook Security:** +- **HMAC Validation**: Enhanced Gitea webhook signature verification +- **Content-Type Validation**: Strict payload type checking +- **Attack Logging**: Comprehensive security event logging + +### Secret Management + +**Production-Ready Secret Handling:** +- **File-based secrets**: Recommended for production (Docker secrets, Kubernetes secrets) +- **Environment variables**: Development and testing +- **Secret validation**: Startup validation ensures all required secrets are present +- **No hardcoded credentials**: All sensitive data externalized + +**Security Audit:** +- **OWASP Top 10**: All major vulnerabilities addressed +- **Security Scan Clean**: Comprehensive security audit completed +- **Compliance Ready**: SOC 2, ISO 27001 preparation + +> **🔒 Security Details**: See [`SECURITY_AUDIT_REPORT.md`](SECURITY_AUDIT_REPORT.md) for complete security analysis. + +## 📋 API Reference + +### Core Endpoints + +**Repository Management:** +| Endpoint | Method | Auth | Description | +|----------|--------|------|-------------| +| `/api/repositories` | GET, POST | JWT | Repository monitoring management | +| `/api/repositories/{id}/sync` | POST | JWT | Manual repository synchronization | + +**Council Management:** +| Endpoint | Method | Auth | Description | +|----------|--------|------|-------------| +| `/api/councils` | GET | JWT | List all councils | +| `/api/councils/{id}` | GET | JWT | Get council details with agents | +| `/api/councils/{id}/artifacts` | GET, POST | JWT | Manage council artifacts | + +**Task & Project Management:** +| Endpoint | Method | Auth | Description | +|----------|--------|------|-------------| +| `/api/tasks` | GET, POST | JWT | Task lifecycle management | +| `/api/projects/analyze` | POST | JWT | Project analysis via N8N integration | + +**System & Health:** +| Endpoint | Method | Auth | Description | +|----------|--------|------|-------------| +| `/health` | GET | None | Basic health check | +| `/admin/health/details` | GET | JWT | Comprehensive system status | +| `/metrics` | GET | None | Prometheus metrics | + +**Webhooks:** +| Endpoint | Method | Auth | Description | +|----------|--------|------|-------------| +| `/webhooks/gitea` | POST | HMAC | Gitea webhook handler | + +> **📚 Complete API Documentation**: See [`docs/API_SPECIFICATION.md`](docs/API_SPECIFICATION.md) for detailed API reference. ## 🗄️ Database Schema -### Core Tables (MVP) +### Core Tables -- **teams**: Team management and status tracking -- **team_roles**: Available roles (executor, coordinator, reviewer) -- **team_assignments**: Agent-to-team assignments -- **agents**: Minimal agent registry -- **slurp_submissions**: Artifact tracking +**Council Management:** +- **councils**: Project kickoff council tracking with metadata +- **council_agents**: Individual agent deployment and status +- **council_artifacts**: Deliverable production (manifests, DRs, plans) +- **council_decisions**: Decision-making process and consensus tracking -## 🔐 Security Features +**System Tables:** +- **repositories**: Monitored Gitea repositories +- **tasks**: Task lifecycle management +- **agents**: Agent registry and capabilities +- **issues**: Gitea issue synchronization -- **Docker Swarm Secrets**: Sensitive data management -- **SHHH Integration**: Data redaction and encryption -- **JWT Authentication**: Service and user token validation -- **Webhook Signature Validation**: GITEA webhook authenticity -- **Rate Limiting**: API endpoint protection +**Performance Optimization:** +- Optimized indexes on all hot query paths +- Connection pooling and query optimization +- Migration-based schema management + +> **📊 Schema Documentation**: See [`docs/DATABASE_SCHEMA.md`](docs/DATABASE_SCHEMA.md) for complete schema reference. ## 🛠️ Development Commands @@ -129,62 +342,92 @@ go vet ./... # Database migrations migrate -path migrations -database "postgres://..." up + +# Docker build +docker build -t whoosh:latest . + +# Security scan +go run honnef.co/go/tools/cmd/staticcheck ./... ``` -## 📊 Monitoring +## 🗺️ Development Roadmap -### Docker Swarm Services +### Phase 1: Council Formation Engine ✅ **COMPLETED** +- [x] Design Brief detection and parsing (`chorus-entrypoint` labels) +- [x] Intelligent council composition algorithms +- [x] Production-grade agent deployment via Docker Swarm +- [x] P2P communication infrastructure +- [x] **Production Readiness Achieved**: + - [x] Enterprise security (auth, CORS, input validation, rate limiting) + - [x] Full API coverage (council lifecycle, artifacts management) + - [x] OpenTelemetry distributed tracing with correlation IDs + - [x] Comprehensive health monitoring and observability + - [x] Configurable architecture (no hardcoded dependencies) + - [x] Database performance optimization + - [x] Feature flag system (LLM vs heuristic composition) -```bash -# Service status -docker service ls --filter label=com.docker.stack.namespace=whoosh +### Phase 2: Enhanced Collaboration 🔄 **IN PROGRESS** +- [x] HMMM protocol foundation +- [ ] Advanced consensus mechanisms and voting +- [ ] Rich artifact template system +- [ ] Cross-council coordination workflows +- [ ] SLURP integration for knowledge preservation -# Service logs -docker service logs -f whoosh_whoosh +### Phase 3: Autonomous Teams 📋 **PLANNED** +- [ ] Dynamic team formation for ongoing development tasks +- [ ] Self-organizing agent behaviors and capability matching +- [ ] Performance-based team composition optimization +- [ ] BZZZ integration for post-kickoff task management +- [ ] Handoff mechanisms from councils to development teams -# Scale services -docker service scale whoosh_whoosh=3 -``` +### Phase 4: Advanced Intelligence 🔮 **FUTURE** +- [ ] Machine learning for optimal team composition +- [ ] Predictive project success modeling +- [ ] Cross-project knowledge transfer and pattern recognition +- [ ] Advanced quality prediction and risk assessment +- [ ] Multi-project portfolio coordination -### Health Endpoints +## 🔗 Integration Points -- Health: `https://whoosh.chorus.services/health` -- Ready: `https://whoosh.chorus.services/health/ready` - -## 🔄 Future Roadmap - -### Post-MVP Features - -1. **Team Composer**: LLM-powered task analysis and team formation -2. **P2P Communication**: UCXL addressing and HMMM integration -3. **Agent Self-Organization**: Automatic team application and consensus -4. **Advanced Analytics**: Performance metrics and team effectiveness -5. **Multi-Repository Support**: Cross-project team coordination - -### Integration Points - -- **CHORUS Agents**: P2P task coordination and execution -- **BZZZ System**: Distributed task management integration -- **SHHH Encryption**: Secure data handling and transmission +### CHORUS Ecosystem +- **CHORUS Agents**: Specialized AI agents with role-based prompts +- **BZZZ System**: Task management for post-kickoff development +- **SLURP Storage**: Artifact preservation and knowledge graph +- **HMMM Protocol**: Structured inter-agent communication - **UCXL Addressing**: Decentralized resource identification -- **SLURP Storage**: Comprehensive artifact preservation + +### External Systems +- **Gitea**: Repository hosting and webhook integration +- **N8N**: Workflow automation and LLM analysis +- **Docker Swarm**: Container orchestration and deployment +- **PostgreSQL**: Data persistence and transaction management ## 📚 Documentation -- [Development Plan](docs/DEVELOPMENT_PLAN.md) - Comprehensive transformation roadmap -- [Database Schema](docs/DATABASE_SCHEMA.md) - Complete schema documentation -- [API Specification](docs/API_SPECIFICATION.md) - Full API reference -- [Team Composer Spec](docs/TEAM_COMPOSER_SPEC.md) - LLM integration details +- **[API Specification](docs/API_SPECIFICATION.md)** - Complete API reference +- **[Configuration Guide](docs/CONFIGURATION.md)** - Environment variables and setup +- **[Database Schema](docs/DATABASE_SCHEMA.md)** - Schema documentation +- **[Development Plan](docs/DEVELOPMENT_PLAN.md)** - Roadmap and architecture +- **[Security Audit](SECURITY_AUDIT_REPORT.md)** - Security analysis and compliance +- **[Deployment Guide](docs/DEPLOYMENT.md)** - Production deployment procedures ## 🤝 Contributing WHOOSH follows the CHORUS ecosystem development patterns: -1. Branch from `main` for features -2. Implement with comprehensive tests -3. Update version tags for container builds -4. Deploy to staging for validation -5. Create PR with detailed description +1. **Branch Strategy**: Feature branches from `main` +2. **Development**: Implement with comprehensive tests and documentation +3. **Security**: All changes undergo security review +4. **Deployment**: Version tags trigger automated builds +5. **Validation**: Staging deployment before production +6. **Review**: Detailed PR review process + +### Code Standards +- Go formatting with `gofmt` +- Comprehensive test coverage +- Security-first development +- OpenTelemetry instrumentation +- Structured logging with correlation IDs ## 📄 License @@ -192,4 +435,9 @@ This project is part of the CHORUS ecosystem. All rights reserved. --- +**Production Status**: ✅ **READY FOR DEPLOYMENT** +Built with ❤️ using collaborative AI agents, enterprise-grade security, and comprehensive observability. + +**Latest Release**: Production-ready council formation engine with full security, API coverage, and monitoring capabilities. + **WHOOSH** - *Where autonomous AI development teams come together* 🎭 \ No newline at end of file diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md new file mode 100644 index 0000000..6633d14 --- /dev/null +++ b/docs/CONFIGURATION.md @@ -0,0 +1,459 @@ +# WHOOSH Configuration Guide + +This guide provides comprehensive documentation for all WHOOSH configuration options and environment variables. + +## 📋 Quick Reference + +| Category | Variables | Description | +|----------|-----------|-------------| +| [Database](#database-configuration) | `WHOOSH_DATABASE_*` | PostgreSQL connection and pooling | +| [Gitea Integration](#gitea-integration) | `WHOOSH_GITEA_*` | Repository monitoring and webhooks | +| [Security](#security-configuration) | `WHOOSH_JWT_*`, `WHOOSH_CORS_*` | Authentication and access control | +| [External Services](#external-services) | `WHOOSH_N8N_*`, `WHOOSH_BACKBEAT_*` | Third-party integrations | +| [Feature Flags](#feature-flags) | `WHOOSH_FEATURE_*` | Optional functionality toggles | +| [Docker Integration](#docker-integration) | `WHOOSH_DOCKER_*` | Container orchestration | +| [Observability](#observability-configuration) | `WHOOSH_OTEL_*`, `WHOOSH_LOG_*` | Tracing and logging | + +## 🗄️ Database Configuration + +### Core Database Settings + +```bash +# Primary database connection +WHOOSH_DATABASE_URL=postgres://username:password@host:5432/database?sslmode=require + +# Alternative: Individual components +WHOOSH_DB_HOST=localhost +WHOOSH_DB_PORT=5432 +WHOOSH_DB_NAME=whoosh +WHOOSH_DB_USER=whoosh_user +WHOOSH_DB_PASSWORD=secure_password +WHOOSH_DB_SSLMODE=require +``` + +### Connection Pool Settings + +```bash +# Connection pool configuration +WHOOSH_DB_MAX_OPEN_CONNS=25 # Maximum open connections +WHOOSH_DB_MAX_IDLE_CONNS=10 # Maximum idle connections +WHOOSH_DB_CONN_MAX_LIFETIME=300s # Connection lifetime +WHOOSH_DB_CONN_MAX_IDLE_TIME=60s # Maximum idle time +``` + +### Migration Settings + +```bash +# Database migration configuration +WHOOSH_DB_MIGRATE_ON_START=true # Run migrations on startup +WHOOSH_MIGRATION_PATH=./migrations # Migration files location +``` + +## 🔧 Gitea Integration + +### Basic Gitea Settings + +```bash +# Gitea instance configuration +WHOOSH_GITEA_URL=https://gitea.example.com +WHOOSH_GITEA_TOKEN_FILE=/run/secrets/gitea_token # Recommended for production +WHOOSH_GITEA_TOKEN=your-gitea-api-token # Alternative for development + +# Webhook configuration +WHOOSH_WEBHOOK_SECRET_FILE=/run/secrets/webhook_secret +WHOOSH_WEBHOOK_SECRET=your-webhook-secret +``` + +### Repository Monitoring + +```bash +# Repository sync behavior +WHOOSH_GITEA_EAGER_FILTER=true # API-level filtering (recommended) +WHOOSH_GITEA_FULL_RESCAN=false # Complete vs incremental scan +WHOOSH_GITEA_DEBUG_URLS=false # Log exact API URLs for debugging + +# Retry and timeout settings +WHOOSH_GITEA_MAX_RETRIES=3 # API retry attempts +WHOOSH_GITEA_RETRY_DELAY=2s # Delay between retries +WHOOSH_GITEA_REQUEST_TIMEOUT=30s # API request timeout +``` + +### Label and Issue Configuration + +```bash +# Label management +WHOOSH_CHORUS_TASK_LABELS=chorus-entrypoint,bzzz-task +WHOOSH_AUTO_CREATE_LABELS=true # Auto-create missing labels +WHOOSH_ENABLE_CHORUS_INTEGRATION=true + +# Issue processing +WHOOSH_ISSUE_BATCH_SIZE=50 # Issues per API request +WHOOSH_ISSUE_SYNC_INTERVAL=300s # Sync frequency +``` + +## 🔐 Security Configuration + +### JWT Authentication + +```bash +# JWT token configuration +WHOOSH_JWT_SECRET_FILE=/run/secrets/jwt_secret # Recommended +WHOOSH_JWT_SECRET=your-jwt-secret # Alternative +WHOOSH_JWT_EXPIRATION=24h # Token expiration +WHOOSH_JWT_ISSUER=whoosh # Token issuer +WHOOSH_JWT_ALGORITHM=HS256 # Signing algorithm +``` + +### CORS Settings + +```bash +# CORS configuration - NEVER use * in production +WHOOSH_CORS_ALLOWED_ORIGINS=https://app.example.com,https://admin.example.com +WHOOSH_CORS_ALLOWED_METHODS=GET,POST,PUT,DELETE,OPTIONS +WHOOSH_CORS_ALLOWED_HEADERS=Authorization,Content-Type,X-Requested-With +WHOOSH_CORS_ALLOW_CREDENTIALS=true +WHOOSH_CORS_MAX_AGE=86400 # Preflight cache duration +``` + +### Rate Limiting + +```bash +# Rate limiting configuration +WHOOSH_RATE_LIMIT_ENABLED=true +WHOOSH_RATE_LIMIT_REQUESTS=100 # Requests per window +WHOOSH_RATE_LIMIT_WINDOW=60s # Rate limiting window +WHOOSH_RATE_LIMIT_CLEANUP_INTERVAL=300s # Cleanup frequency +``` + +### Input Validation + +```bash +# Request validation settings +WHOOSH_MAX_REQUEST_SIZE=1048576 # 1MB default request size +WHOOSH_MAX_WEBHOOK_SIZE=10485760 # 10MB for webhooks +WHOOSH_VALIDATION_STRICT=true # Enable strict validation +``` + +### Service Tokens + +```bash +# Service-to-service authentication +WHOOSH_SERVICE_TOKEN_FILE=/run/secrets/service_token +WHOOSH_SERVICE_TOKEN=your-service-token +WHOOSH_SERVICE_TOKEN_HEADER=X-Service-Token +``` + +## 🔗 External Services + +### N8N Integration + +```bash +# N8N workflow automation +WHOOSH_N8N_BASE_URL=https://n8n.example.com +WHOOSH_N8N_AUTH_TOKEN_FILE=/run/secrets/n8n_token +WHOOSH_N8N_AUTH_TOKEN=your-n8n-token +WHOOSH_N8N_TIMEOUT=60s # Request timeout +WHOOSH_N8N_MAX_RETRIES=3 # Retry attempts +``` + +### BackBeat Monitoring + +```bash +# BackBeat performance monitoring +WHOOSH_BACKBEAT_URL=http://backbeat:3001 +WHOOSH_BACKBEAT_ENABLED=true +WHOOSH_BACKBEAT_TOKEN_FILE=/run/secrets/backbeat_token +WHOOSH_BACKBEAT_BEAT_INTERVAL=30s # Beat frequency +WHOOSH_BACKBEAT_TIMEOUT=10s # Request timeout +``` + +## 🚩 Feature Flags + +### LLM Integration + +```bash +# AI vs Heuristic classification +WHOOSH_FEATURE_LLM_CLASSIFICATION=false # Enable LLM classification +WHOOSH_FEATURE_LLM_SKILL_ANALYSIS=false # Enable LLM skill analysis +WHOOSH_FEATURE_LLM_TEAM_MATCHING=false # Enable LLM team matching +WHOOSH_FEATURE_ENABLE_ANALYSIS_LOGGING=true # Log analysis details +WHOOSH_FEATURE_ENABLE_FAILSAFE_FALLBACK=true # Fallback to heuristics +``` + +### Experimental Features + +```bash +# Advanced features (use with caution) +WHOOSH_FEATURE_ADVANCED_P2P=false # Enhanced P2P discovery +WHOOSH_FEATURE_CROSS_COUNCIL_COORDINATION=false +WHOOSH_FEATURE_PREDICTIVE_FORMATION=false # ML-based team formation +WHOOSH_FEATURE_AUTO_SCALING=false # Automatic agent scaling +``` + +## 🐳 Docker Integration + +### Docker Swarm Settings + +```bash +# Docker daemon connection +WHOOSH_DOCKER_ENABLED=true +WHOOSH_DOCKER_HOST=unix:///var/run/docker.sock +WHOOSH_DOCKER_VERSION=1.41 # Docker API version +WHOOSH_DOCKER_TIMEOUT=60s # Operation timeout + +# Swarm-specific settings +WHOOSH_SWARM_NETWORK=chorus_default # Swarm network name +WHOOSH_SWARM_CONSTRAINTS=node.role==worker # Placement constraints +``` + +### Agent Deployment + +```bash +# CHORUS agent deployment +WHOOSH_AGENT_IMAGE=anthonyrawlins/chorus:latest +WHOOSH_AGENT_MEMORY_LIMIT=2048m # Memory limit per agent +WHOOSH_AGENT_CPU_LIMIT=1.0 # CPU limit per agent +WHOOSH_AGENT_RESTART_POLICY=on-failure +WHOOSH_AGENT_MAX_RESTARTS=3 +``` + +### Volume and Secret Mounts + +```bash +# Shared volumes +WHOOSH_PROMPTS_PATH=/rust/containers/WHOOSH/prompts +WHOOSH_SHARED_DATA_PATH=/rust/shared + +# Docker secrets +WHOOSH_DOCKER_SECRET_PREFIX=whoosh_ # Secret naming prefix +``` + +## 📊 Observability Configuration + +### OpenTelemetry Tracing + +```bash +# OpenTelemetry configuration +WHOOSH_OTEL_ENABLED=true +WHOOSH_OTEL_SERVICE_NAME=whoosh +WHOOSH_OTEL_SERVICE_VERSION=1.0.0 +WHOOSH_OTEL_ENDPOINT=http://jaeger:14268/api/traces +WHOOSH_OTEL_SAMPLER_RATIO=1.0 # Sampling ratio (0.0-1.0) +WHOOSH_OTEL_BATCH_TIMEOUT=5s # Batch export timeout +``` + +### Logging Configuration + +```bash +# Logging settings +WHOOSH_LOG_LEVEL=info # trace, debug, info, warn, error +WHOOSH_LOG_FORMAT=json # json or text +WHOOSH_LOG_OUTPUT=stdout # stdout, stderr, or file path +WHOOSH_LOG_CALLER=false # Include caller information +WHOOSH_LOG_TIMESTAMP=true # Include timestamps +``` + +### Metrics and Health + +```bash +# Prometheus metrics +WHOOSH_METRICS_ENABLED=true +WHOOSH_METRICS_PATH=/metrics # Metrics endpoint path +WHOOSH_METRICS_NAMESPACE=whoosh # Metrics namespace + +# Health check configuration +WHOOSH_HEALTH_CHECK_INTERVAL=30s # Internal health check frequency +WHOOSH_HEALTH_TIMEOUT=10s # Health check timeout +``` + +## 🌐 Server Configuration + +### HTTP Server Settings + +```bash +# Server bind configuration +WHOOSH_SERVER_HOST=0.0.0.0 # Bind address +WHOOSH_SERVER_PORT=8080 # Listen port +WHOOSH_SERVER_READ_TIMEOUT=30s # Request read timeout +WHOOSH_SERVER_WRITE_TIMEOUT=30s # Response write timeout +WHOOSH_SERVER_IDLE_TIMEOUT=60s # Idle connection timeout +WHOOSH_SERVER_MAX_HEADER_BYTES=1048576 # Max header size +``` + +### TLS Configuration + +```bash +# TLS/SSL settings (optional) +WHOOSH_TLS_ENABLED=false +WHOOSH_TLS_CERT_FILE=/path/to/cert.pem +WHOOSH_TLS_KEY_FILE=/path/to/key.pem +WHOOSH_TLS_MIN_VERSION=1.2 # Minimum TLS version +``` + +## 🔍 P2P Discovery Configuration + +### Service Discovery + +```bash +# P2P discovery settings +WHOOSH_P2P_DISCOVERY_ENABLED=true +WHOOSH_P2P_KNOWN_ENDPOINTS=chorus:8081,agent1:8081,agent2:8081 +WHOOSH_P2P_SERVICE_PORTS=8081,8082,8083 +WHOOSH_P2P_DOCKER_ENABLED=true # Docker Swarm discovery + +# Health checking +WHOOSH_P2P_HEALTH_TIMEOUT=5s # Agent health check timeout +WHOOSH_P2P_RETRY_ATTEMPTS=3 # Health check retries +WHOOSH_P2P_DISCOVERY_INTERVAL=60s # Discovery cycle frequency +``` + +### Agent Filtering + +```bash +# Agent capability filtering +WHOOSH_P2P_REQUIRED_CAPABILITIES=council,reasoning +WHOOSH_P2P_MIN_AGENT_VERSION=1.0.0 # Minimum agent version +WHOOSH_P2P_FILTER_INACTIVE=true # Filter inactive agents +``` + +## 📁 Environment File Examples + +### Production Environment (.env.production) + +```bash +# Production configuration template +# Copy to .env and customize + +# Database +WHOOSH_DATABASE_URL=postgres://whoosh:${DB_PASSWORD}@postgres:5432/whoosh?sslmode=require +WHOOSH_DB_MAX_OPEN_CONNS=50 +WHOOSH_DB_MAX_IDLE_CONNS=20 + +# Security (use Docker secrets in production) +WHOOSH_JWT_SECRET_FILE=/run/secrets/jwt_secret +WHOOSH_WEBHOOK_SECRET_FILE=/run/secrets/webhook_secret +WHOOSH_CORS_ALLOWED_ORIGINS=https://app.company.com,https://admin.company.com + +# Gitea +WHOOSH_GITEA_URL=https://git.company.com +WHOOSH_GITEA_TOKEN_FILE=/run/secrets/gitea_token +WHOOSH_GITEA_EAGER_FILTER=true + +# External services +WHOOSH_N8N_BASE_URL=https://workflows.company.com +WHOOSH_BACKBEAT_URL=http://backbeat:3001 + +# Observability +WHOOSH_OTEL_ENABLED=true +WHOOSH_OTEL_ENDPOINT=http://jaeger:14268/api/traces +WHOOSH_LOG_LEVEL=info + +# Feature flags (conservative defaults) +WHOOSH_FEATURE_LLM_CLASSIFICATION=false +WHOOSH_FEATURE_LLM_SKILL_ANALYSIS=false + +# Docker +WHOOSH_DOCKER_ENABLED=true +``` + +### Development Environment (.env.development) + +```bash +# Development configuration +# More permissive settings for local development + +# Database +WHOOSH_DATABASE_URL=postgres://whoosh:password@localhost:5432/whoosh?sslmode=disable + +# Security (relaxed for development) +WHOOSH_JWT_SECRET=dev-secret-change-in-production +WHOOSH_WEBHOOK_SECRET=dev-webhook-secret +WHOOSH_CORS_ALLOWED_ORIGINS=http://localhost:3000,http://localhost:8080 + +# Gitea +WHOOSH_GITEA_URL=http://localhost:3000 +WHOOSH_GITEA_TOKEN=your-dev-token +WHOOSH_GITEA_DEBUG_URLS=true + +# Logging (verbose for debugging) +WHOOSH_LOG_LEVEL=debug +WHOOSH_LOG_CALLER=true + +# Feature flags (enable experimental features) +WHOOSH_FEATURE_LLM_CLASSIFICATION=true +WHOOSH_FEATURE_ENABLE_ANALYSIS_LOGGING=true + +# Docker (disabled for local development) +WHOOSH_DOCKER_ENABLED=false +``` + +## 🔧 Configuration Validation + +WHOOSH validates configuration on startup and provides detailed error messages for invalid settings: + +### Required Variables +- `WHOOSH_DATABASE_URL` or individual DB components +- `WHOOSH_GITEA_URL` +- `WHOOSH_GITEA_TOKEN` or `WHOOSH_GITEA_TOKEN_FILE` + +### Common Validation Errors + +```bash +# Invalid database URL +ERROR: Invalid database URL format + +# Missing secrets +ERROR: JWT secret not found. Set WHOOSH_JWT_SECRET or WHOOSH_JWT_SECRET_FILE + +# Invalid CORS configuration +ERROR: CORS wildcard (*) not allowed in production. Set specific origins. + +# Docker connection failed +WARNING: Docker not available. Agent deployment disabled. +``` + +## 🚀 Best Practices + +### Production Deployment + +1. **Use Docker secrets** for all sensitive data +2. **Set specific CORS origins** (never use wildcards) +3. **Enable rate limiting** and input validation +4. **Configure appropriate timeouts** for your network +5. **Enable observability** (tracing, metrics, logs) +6. **Use conservative feature flags** until tested + +### Security Hardening + +1. **Rotate secrets regularly** using automated processes +2. **Use TLS everywhere** in production +3. **Monitor security logs** for suspicious activity +4. **Keep dependency versions updated** +5. **Review access logs** regularly + +### Performance Optimization + +1. **Tune database connection pools** based on load +2. **Configure appropriate cache settings** +3. **Use CDN for static assets** if applicable +4. **Monitor resource usage** and scale accordingly +5. **Enable compression** for large responses + +### Troubleshooting + +1. **Enable debug logging** temporarily for issues +2. **Check health endpoints** for component status +3. **Monitor trace data** for request flow issues +4. **Validate configuration** before deployment +5. **Test in staging environment** first + +--- + +## 📚 Related Documentation + +- **[Security Audit](../SECURITY_AUDIT_REPORT.md)** - Security implementation details +- **[API Specification](API_SPECIFICATION.md)** - Complete API reference +- **[Database Schema](DATABASE_SCHEMA.md)** - Database structure +- **[Deployment Guide](DEPLOYMENT.md)** - Production deployment procedures + +For additional support, refer to the main [WHOOSH README](../README.md) or create an issue in the repository. \ No newline at end of file