Commit Graph

15 Commits

Author SHA1 Message Date
anthonyrawlins
f7130b327c feat: Implement council brief processing loop for task execution
Add processBriefs() polling loop that checks for assigned council briefs
and executes them using the ExecutionEngine infrastructure.

Changes:
- Add GetCurrentAssignment() public method to council.Manager
- Make HTTPServer.CouncilManager public for brief access
- Add processBriefs() 15-second polling loop in agent_support.go
- Add executeBrief() to initialize and run ExecutionEngine
- Add buildExecutionRequest() to convert briefs to execution requests
- Add uploadResults() to send completed work to WHOOSH
- Wire processBriefs() into StartAgentMode() as background goroutine

This addresses the root cause of task execution not happening: briefs
were being stored but never polled or executed. The execution
infrastructure (ExecutionEngine, AI providers, prompt system) was
complete but not connected to the council workflow.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-11 12:15:49 +11:00
anthonyrawlins
7381137db5 feat(chorus): run chorus-agent (replace deprecated wrapper); deterministic council role-claim shuffle; compose: WHOOSH UI env + Traefik label fixes + rotated JWT secret 2025-10-08 23:52:06 +11:00
anthonyrawlins
63dab5c4d4 Add LightRAG MCP integration for RAG-enhanced AI reasoning
This commit integrates LightRAG (Retrieval-Augmented Generation) MCP server
support into CHORUS, enabling graph-based knowledge retrieval to enrich AI
reasoning and context resolution.

## New Components

1. **LightRAG Client** (pkg/mcp/lightrag_client.go)
   - HTTP client for LightRAG MCP server
   - Supports 4 query modes: naive, local, global, hybrid
   - Health checking, document insertion, context retrieval
   - 277 lines with comprehensive error handling

2. **Integration Tests** (pkg/mcp/lightrag_client_test.go)
   - Unit and integration tests
   - Tests all query modes and operations
   - 239 lines with detailed test cases

3. **SLURP Context Enricher** (pkg/slurp/context/lightrag.go)
   - Enriches SLURP context nodes with RAG data
   - Batch processing support
   - Knowledge base building over time
   - 203 lines

4. **Documentation** (docs/LIGHTRAG_INTEGRATION.md)
   - Complete integration guide
   - Configuration examples
   - Usage patterns and troubleshooting
   - 350+ lines

## Modified Components

1. **Configuration** (pkg/config/config.go)
   - Added LightRAGConfig struct
   - Environment variable support (5 variables)
   - Default configuration with hybrid mode

2. **Reasoning Engine** (reasoning/reasoning.go)
   - GenerateResponseWithRAG() - RAG-enriched generation
   - GenerateResponseSmartWithRAG() - Smart model + RAG
   - SetLightRAGClient() - Client configuration
   - Non-fatal error handling (graceful degradation)

3. **Runtime Initialization** (internal/runtime/shared.go)
   - Automatic LightRAG client setup
   - Health check on startup
   - Integration with reasoning engine

## Configuration

Environment variables:
- CHORUS_LIGHTRAG_ENABLED (default: false)
- CHORUS_LIGHTRAG_BASE_URL (default: http://127.0.0.1:9621)
- CHORUS_LIGHTRAG_TIMEOUT (default: 30s)
- CHORUS_LIGHTRAG_API_KEY (optional)
- CHORUS_LIGHTRAG_DEFAULT_MODE (default: hybrid)

## Features

-  Optional and non-blocking (graceful degradation)
-  Four query modes for different use cases
-  Context enrichment for SLURP system
-  Knowledge base building over time
-  Health monitoring and error handling
-  Comprehensive tests and documentation

## Testing

LightRAG server tested at http://127.0.0.1:9621
- Health check:  Passed
- Query operations:  Tested
- Integration points:  Verified

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-30 23:56:09 +10:00
anthonyrawlins
17673c38a6 fix: P2P connectivity regression + dynamic versioning system
## P2P Connectivity Fixes
- **Root Cause**: mDNS discovery was conditionally disabled in Task Execution Engine implementation
- **Solution**: Restored always-enabled mDNS discovery from working baseline (eb2e05f)
- **Result**: 9/9 Docker Swarm replicas with working P2P mesh, democratic elections, and leader consensus

## Dynamic Version System
- **Problem**: Hardcoded version "0.1.0-dev" in 1000+ builds made debugging impossible
- **Solution**: Implemented build-time version injection via ldflags
- **Features**: Shows commit hash, build date, and semantic version
- **Example**: `CHORUS-agent 0.5.5 (build: 9dbd361, 2025-09-26_05:55:55)`

## Container Compatibility
- **Issue**: Binary execution failed in Alpine due to glibc/musl incompatibility
- **Solution**: Added Ubuntu-based Dockerfile for proper glibc support
- **Benefit**: Reliable container execution across Docker Swarm nodes

## Key Changes
- `internal/runtime/shared.go`: Always enable mDNS discovery, dynamic version vars
- `cmd/agent/main.go`: Build-time version injection and display
- `p2p/node.go`: Restored working "🐝 Bzzz Node Status" logging format
- `Makefile`: Updated version to 0.5.5, proper ldflags configuration
- `Dockerfile.ubuntu`: New glibc-compatible container base
- `docker-compose.yml`: Updated to latest image tag for Watchtower auto-updates

## Verification
 P2P mesh connectivity: Peers exchanging availability broadcasts
 Democratic elections: Candidacy announcements and leader selection
 BACKBEAT integration: Beat synchronization and degraded mode handling
 Dynamic versioning: All containers show v0.5.5 with build metadata
 Task Execution Engine: All Phase 4 functionality preserved and working

Fixes P2P connectivity regression while preserving complete Task Execution Engine implementation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 16:05:25 +10:00
anthonyrawlins
e523c4b543 feat: Implement CHORUS scaling improvements for robust autoscaling
Address WHOOSH issue #7 with comprehensive scaling optimizations to prevent
license server, bootstrap peer, and control plane collapse during fast scale-out.

HIGH-RISK FIXES (Must-Do):
 License gate already implemented with cache + circuit breaker + grace window
 mDNS disabled in container environments (CHORUS_MDNS_ENABLED=false)
 Connection rate limiting (5 dials/sec, 16 concurrent DHT queries)
 Connection manager with watermarks (32 low, 128 high)
 AutoNAT enabled for container networking

MEDIUM-RISK FIXES (Next Priority):
 Assignment merge layer with HTTP/file config + SIGHUP reload
 Runtime configuration system with WHOOSH assignment API support
 Election stability windows to prevent churn:
  - CHORUS_ELECTION_MIN_TERM=30s (minimum time between elections)
  - CHORUS_LEADER_MIN_TERM=45s (minimum time before challenging healthy leader)
 Bootstrap pool JSON support with priority sorting and join stagger

NEW FEATURES:
- Runtime config system with assignment overrides from WHOOSH
- SIGHUP reload handler for live configuration updates
- JSON bootstrap configuration with peer metadata (region, roles, priority)
- Configurable election stability windows with environment variables
- Multi-format bootstrap support: Assignment → JSON → CSV

FILES MODIFIED:
- pkg/config/assignment.go (NEW): Runtime assignment merge system
- docker/bootstrap.json (NEW): Example JSON bootstrap configuration
- pkg/election/election.go: Added stability windows and churn prevention
- internal/runtime/shared.go: Integrated assignment loading and conditional mDNS
- p2p/node.go: Added connection management and rate limiting
- pkg/config/hybrid_config.go: Added rate limiting configuration fields
- docker/docker-compose.yml: Updated environment variables and configs
- README.md: Updated status table with scaling milestone

This implementation enables wave-based autoscaling without system collapse,
addressing all scaling concerns from WHOOSH issue #7.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-23 17:50:40 +10:00
anthonyrawlins
eb2e05ff84 feat: Preserve comprehensive CHORUS enhancements and P2P improvements
This commit preserves substantial development work including:

## Core Infrastructure:
- **Bootstrap Pool Manager** (pkg/bootstrap/pool_manager.go): Advanced peer
  discovery and connection management for distributed CHORUS clusters
- **Runtime Configuration System** (pkg/config/runtime_config.go): Dynamic
  configuration updates and assignment-based role management
- **Cryptographic Key Derivation** (pkg/crypto/key_derivation.go): Secure
  key management for P2P networking and DHT operations

## Enhanced Monitoring & Operations:
- **Comprehensive Monitoring Stack**: Added Prometheus and Grafana services
  with full metrics collection, alerting, and dashboard visualization
- **License Gate System** (internal/licensing/license_gate.go): Advanced
  license validation with circuit breaker patterns
- **Enhanced P2P Configuration**: Improved networking configuration for
  better peer discovery and connection reliability

## Health & Reliability:
- **DHT Health Check Fix**: Temporarily disabled problematic DHT health
  checks to prevent container shutdown issues
- **Enhanced License Validation**: Improved error handling and retry logic
  for license server communication

## Docker & Deployment:
- **Optimized Container Configuration**: Updated Dockerfile and compose
  configurations for better resource management and networking
- **Static Binary Support**: Proper compilation flags for Alpine containers

This work addresses the P2P networking issues that were preventing proper
leader election in CHORUS clusters and establishes the foundation for
reliable distributed operation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-23 00:02:37 +10:00
anthonyrawlins
95784822ce fix(logging): resolve duplicate type case compilation error in hypercore.go
@goal: CHORUS-REQ-001 - Fix critical compilation error blocking development

- Remove duplicate type cases for interface{}/any and []interface{}/[]any
- Go 1.18+ treats interface{} and any as identical types
- Standardize on 'any' type for consistency with modern Go practices
- Add proper type conversion for cloneLogMap compatibility
- Include requirement traceability comments

Fixes: CHORUS issue #1
Test: go build ./internal/logging/... passes without errors

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-21 17:16:38 +10:00
anthonyrawlins
1bb736c09a Harden CHORUS security and messaging stack 2025-09-20 23:21:35 +10:00
anthonyrawlins
e820770409 Complete Phase 3: Enhanced Human Workflows for CHORUS HAP
PHASE 3 IMPLEMENTATION COMPLETE:

 Collaborative Editing Interfaces:
- Full session management (start, join, list, status, leave)
- DHT-based persistent collaborative sessions
- Real-time collaborative editor with conflict resolution
- Multi-participant support with automatic sync
- Chat integration for collaborative coordination
- HMMM network integration for all collaborative events

 Decision Tracking and Approval Workflows:
- Complete decision lifecycle (create, view, vote, track)
- DHT storage system for persistent decisions
- Rich voting system (approve, reject, defer, abstain)
- Real-time vote tracking with approval percentages
- HMMM announcements for proposals and votes
- Multiple decision types (technical, operational, policy, emergency)

 Web Bridge for Browser-Based HAP Interface:
- Complete HTTP server on port 8090
- Modern responsive web UI with card-based layout
- Functional decision management with JavaScript voting
- Real-time status monitoring and network information
- REST API endpoints for all major HAP functions
- WebSocket infrastructure for real-time updates

TECHNICAL HIGHLIGHTS:
- Added CollaborativeSession and Decision data structures
- Enhanced TerminalInterface with web server support
- Full P2P integration (DHT storage, HMMM messaging)
- Professional web interface with intuitive navigation
- API-driven architecture ready for multi-user scenarios

FEATURES DELIVERED:
- Multi-modal access (terminal + web interfaces)
- Real-time P2P coordination across all workflows
- Network-wide event distribution and collaboration
- Production-ready error handling and validation
- Scalable architecture supporting mixed human/agent teams

Phase 3 objectives fully achieved. CHORUS HAP now provides comprehensive
human agent participation in P2P task coordination with both power-user
terminal access and user-friendly web interfaces.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-07 10:04:14 +10:00
anthonyrawlins
aea4d45fd8 Implement Phase 2 & 3: Complete HAP Terminal Interface with Patch Management
🎭 Phase 2: HAP Terminal Interface Implementation
 **Core Terminal Interface**: Interactive command-driven HAP terminal with help system
 **HMMM Message Composition System**:
   - New reasoning messages, thread replies, network queries, decision proposals
   - Complete message metadata handling (topics, threads, timestamps)
 **UCXL Context Browsing System**:
   - Address parsing, content retrieval from DHT encrypted storage
   - Search functionality, content creation, history navigation
 **Decision Participation System**:
   - Active decision listing, decision details with voting status
   - Vote casting with reasoning, decision proposals, HMMM integration

🔧 Phase 3: Enhanced Human Workflows
 **Patch Creation and Submission Workflows**:
   - Complete patch lifecycle management (create, review, submit, track)
   - Multiple patch types (context, code, config, docs)
   - UCXL integration with DHT storage, HMMM coordination
 **Time-Travel Diff Support**:
   - Temporal navigation operators (~~<n>, ^^<n>, @<time>)
   - Decision-hop analysis, visual diff display, version comparison

🏗️ **Architecture Highlights**:
- **Multi-binary structure**: Separate chorus-agent and chorus-hap binaries
- **Shared P2P runtime**: Both binaries use identical libp2p, DHT, HMMM, UCXL systems
- **Interactive sub-shells**: Dedicated command environments for HMMM, UCXL, patches, decisions
- **Network integration**: All features connect to distributed P2P agent network
- **Human-agent parity**: Humans participate as first-class network citizens

📦 **New Files**:
- internal/hapui/terminal.go: Complete HAP terminal interface (2400+ lines)
- prompts/human-roles.yaml: Role-based prompt configuration
- docs/decisions/*: HAP conversion decision record

🔗 **Integration Points**:
- HMMM: Collaborative reasoning and patch/decision announcements
- UCXL: Context addressing and version management
- DHT: Distributed storage of patches and content
- Decision System: Formal approval and consensus workflows

The HAP terminal interface now provides comprehensive human portal into the CHORUS
autonomous agent network, enabling collaborative reasoning, context sharing, patch
management, and distributed decision-making between humans and AI agents.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-07 09:38:14 +10:00
anthonyrawlins
0dbb6bb588 Implement Phase 1: CHORUS Human Agent Portal (HAP) Multi-Binary Architecture
This commit completes Phase 1 of the HAP implementation by restructuring CHORUS
from a single binary to a dual-binary architecture that supports both autonomous
agents and human agent portals using shared P2P infrastructure.

## Key Changes

### Multi-Binary Architecture
- **cmd/agent/main.go**: Autonomous agent binary (preserves all original functionality)
- **cmd/hap/main.go**: Human Agent Portal binary (Phase 2 stub implementation)
- **cmd/chorus/main.go**: Backward compatibility wrapper with deprecation notices

### Shared Runtime Infrastructure
- **internal/runtime/shared.go**: Extracted all P2P infrastructure initialization
- **internal/runtime/agent_support.go**: Agent-specific behaviors and health monitoring
- Preserves 100% of existing CHORUS functionality in shared components

### Enhanced Build System
- **Makefile**: Complete multi-binary build system
  - `make build` - Builds all binaries (agent, hap, compatibility wrapper)
  - `make build-agent` - Agent only
  - `make build-hap` - HAP only
  - `make test-compile` - Compilation verification

## Architecture Achievement

 **Shared P2P Infrastructure**: Both binaries use identical libp2p, DHT, HMMM, UCXL systems
 **Protocol Compatibility**: Human agents appear as valid peers to autonomous agents
 **Container-First Design**: Maintains CHORUS's container deployment model
 **Zero Functionality Loss**: Existing users see no disruption

## Phase 1 Success Metrics - ALL ACHIEVED

 `make build` produces `chorus-agent`, `chorus-hap`, and `chorus` binaries
 Existing autonomous agent functionality unchanged
 Both new binaries can join same P2P mesh
 Clean deprecation path for existing users

## Next Steps

Phase 2 will implement the interactive terminal interface for chorus-hap, enabling:
- HMMM message composition helpers
- UCXL context browsing
- Human-friendly command interface
- Collaborative decision participation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-06 20:49:05 +10:00
anthonyrawlins
1806a4fe09 feat(prompts): load system prompts and defaults from Docker volume; set runtime system prompt; add BACKBEAT standards 2025-09-06 15:42:41 +10:00
anthonyrawlins
9bdcbe0447 Integrate BACKBEAT SDK and resolve KACHING license validation
Major integrations and fixes:
- Added BACKBEAT SDK integration for P2P operation timing
- Implemented beat-aware status tracking for distributed operations
- Added Docker secrets support for secure license management
- Resolved KACHING license validation via HTTPS/TLS
- Updated docker-compose configuration for clean stack deployment
- Disabled rollback policies to prevent deployment failures
- Added license credential storage (CHORUS-DEV-MULTI-001)

Technical improvements:
- BACKBEAT P2P operation tracking with phase management
- Enhanced configuration system with file-based secrets
- Improved error handling for license validation
- Clean separation of KACHING and CHORUS deployment stacks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-06 07:56:26 +10:00
anthonyrawlins
543ab216f9 Complete BZZZ functionality port to CHORUS
🎭 CHORUS now contains full BZZZ functionality adapted for containers

Core systems ported:
- P2P networking (libp2p with DHT and PubSub)
- Task coordination (COOEE protocol)
- HMMM collaborative reasoning
- SHHH encryption and security
- SLURP admin election system
- UCXL content addressing
- UCXI server integration
- Hypercore logging system
- Health monitoring and graceful shutdown
- License validation with KACHING

Container adaptations:
- Environment variable configuration (no YAML files)
- Container-optimized logging to stdout/stderr
- Auto-generated agent IDs for container deployments
- Docker-first architecture

All proven BZZZ P2P protocols, AI integration, and collaboration
features are now available in containerized form.

Next: Build and test container deployment.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 20:02:37 +10:00
anthonyrawlins
7c6cbd562a Initial CHORUS project setup
🎭 CHORUS - Container-First P2P Task Coordination System

- Docker-first architecture designed from ground up
- Environment variable-based configuration (no config files)
- Structured logging to stdout/stderr for container runtimes
- License validation required for operation
- Clean separation from BZZZ legacy systemd approach

Core features implemented:
- Container-optimized logging system
- Environment-based configuration management
- License validation with KACHING integration
- Basic HTTP API and health endpoints
- Docker build and deployment configuration

Ready for P2P protocol development and AI integration.

🤖 Generated with Claude Code
2025-09-02 19:53:33 +10:00