# CHORUS Configuration Package **Package Path**: `/home/tony/chorus/project-queues/active/CHORUS/pkg/config/` The configuration package provides a comprehensive, environment-based configuration system for CHORUS agents. It supports static configuration, runtime assignment overrides from WHOOSH, role-based configuration, and dynamic reloading via SIGHUP signals. ## Table of Contents 1. [Overview](#overview) 2. [Configuration Loading](#configuration-loading) 3. [Configuration Structures](#configuration-structures) 4. [Runtime Configuration](#runtime-configuration) 5. [Dynamic Assignments](#dynamic-assignments) 6. [Role Definitions](#role-definitions) 7. [Hybrid Configuration](#hybrid-configuration) 8. [Security Configuration](#security-configuration) 9. [Environment Variables Reference](#environment-variables-reference) 10. [Assignment Schema](#assignment-schema) 11. [Configuration Validation](#configuration-validation) 12. [Usage Examples](#usage-examples) ## Overview The CHORUS configuration system is designed for containerized deployments where configuration is managed through environment variables. It provides: - **Static Configuration**: Base configuration loaded from environment variables - **Runtime Configuration**: Dynamic configuration with assignment overrides - **WHOOSH Integration**: Fetch task-specific assignments from WHOOSH API - **SIGHUP Reload**: Reload configuration without restarting the agent - **Role-Based Access**: Predefined roles with authority levels and encryption keys - **Hybrid Mode Support**: Phase 2 feature flags for DHT, UCXL, and discovery - **Docker Secrets**: Support for reading sensitive values from files ## Configuration Loading ### Basic Configuration Loading ```go import "github.com/chorus/pkg/config" // Load static configuration from environment variables cfg, err := config.LoadFromEnvironment() if err != nil { log.Fatalf("Failed to load configuration: %v", err) } // Validate configuration if err := cfg.Validate(); err != nil { log.Fatalf("Configuration validation failed: %v", err) } ``` ### Runtime Configuration with Assignment Support ```go // Load runtime configuration (includes base + assignment support) runtimeCfg, err := config.LoadRuntimeConfig() if err != nil { log.Fatalf("Failed to load runtime configuration: %v", err) } // Fetch assignment from WHOOSH ctx := context.Background() assignURL := "https://whoosh.chorus.services/api/v1/assignments" if err := runtimeCfg.LoadAssignment(ctx, assignURL); err != nil { log.Printf("No assignment available: %v", err) } // Get merged configuration (base + overrides) effectiveCfg := runtimeCfg.GetConfig() // Start SIGHUP reload handler runtimeCfg.StartReloadHandler(ctx, assignURL) ``` ## Configuration Structures ### Main Configuration Structure The `Config` struct represents the complete CHORUS configuration: ```go type Config struct { Agent AgentConfig // Agent-specific settings Network NetworkConfig // Network and API settings License LicenseConfig // Licensing configuration AI AIConfig // AI service configuration Logging LoggingConfig // Logging settings V2 V2Config // DHT and P2P settings UCXL UCXLConfig // UCXL protocol settings Slurp SlurpConfig // SLURP integration Security SecurityConfig // Security settings WHOOSHAPI WHOOSHAPIConfig // WHOOSH API integration } ``` ### Agent Configuration Defines agent identity, capabilities, and role: ```go type AgentConfig struct { ID string // Unique agent identifier Specialization string // Agent specialization type MaxTasks int // Maximum concurrent tasks Capabilities []string // Agent capabilities Models []string // Available AI models Role string // Agent role (see Role Definitions) Project string // Project assignment Expertise []string // Domain expertise areas ReportsTo string // Reporting hierarchy Deliverables []string // Expected deliverables ModelSelectionWebhook string // Webhook for model selection DefaultReasoningModel string // Default AI model } ``` **Default Values**: - `Specialization`: "general_developer" - `MaxTasks`: 3 - `Capabilities`: ["general_development", "task_coordination"] - `Models`: ["meta/llama-3.1-8b-instruct"] - `DefaultReasoningModel`: "meta/llama-3.1-8b-instruct" ### Network Configuration Defines network ports and binding: ```go type NetworkConfig struct { P2PPort int // P2P network port (default: 9000) APIPort int // API server port (default: 8080) HealthPort int // Health check port (default: 8081) BindAddr string // Bind address (default: "0.0.0.0") } ``` ### License Configuration Licensing and cluster identification: ```go type LicenseConfig struct { LicenseID string // Required: License identifier ClusterID string // Cluster identifier (default: "default-cluster") OrganizationName string // Organization name KachingURL string // License validation URL IsActive bool // License active status LastValidated time.Time // Last validation timestamp GracePeriodHours int // Grace period (default: 72 hours) LicenseType string // License type ExpiresAt time.Time // Expiration timestamp MaxNodes int // Maximum nodes allowed } ``` **Required**: `LicenseID` must be provided via `CHORUS_LICENSE_ID` or `CHORUS_LICENSE_ID_FILE`. ### AI Configuration AI service provider settings: ```go type AIConfig struct { Provider string // AI provider ("ollama" or "resetdata") Ollama OllamaConfig // Ollama configuration ResetData ResetDataConfig // ResetData configuration } type OllamaConfig struct { Endpoint string // Ollama endpoint (default: "http://localhost:11434") Timeout time.Duration // Request timeout (default: 30s) } type ResetDataConfig struct { BaseURL string // ResetData API URL APIKey string // API key (from env or file) Model string // Model name (default: "meta/llama-3.1-8b-instruct") Timeout time.Duration // Request timeout (default: 30s) } ``` **Default Provider**: "resetdata" ### Logging Configuration ```go type LoggingConfig struct { Level string // Log level: "debug", "info", "warn", "error" (default: "info") Format string // Log format: "structured" or "text" (default: "structured") } ``` ### DHT Configuration (V2) ```go type V2Config struct { DHT DHTConfig } type DHTConfig struct { Enabled bool // Enable DHT (default: true) BootstrapPeers []string // Bootstrap peer multiaddrs MDNSEnabled bool // Enable mDNS discovery (default: true) } ``` ### UCXL Configuration UCXL (Universal Content Exchange Layer) protocol settings: ```go type UCXLConfig struct { Enabled bool // Enable UCXL (default: true) Server ServerConfig // Server settings Storage StorageConfig // Storage settings Resolution ResolutionConfig // Resolution settings } type ServerConfig struct { Enabled bool // Enable UCXL server (default: true) Port int // Server port (default: 8082) BasePath string // Base path for server } type StorageConfig struct { Directory string // Storage directory (default: "/tmp/chorus-ucxi-storage") } type ResolutionConfig struct { CacheTTL time.Duration // Cache TTL (default: 1 hour) } ``` ### SLURP Configuration SLURP (Decision tracking system) integration: ```go type SlurpConfig struct { Enabled bool // Enable SLURP (default: false) BaseURL string // SLURP API URL APIKey string // API key (from env or file) Timeout time.Duration // Request timeout (default: 15s) RetryCount int // Retry count (default: 3) RetryDelay time.Duration // Retry delay (default: 2s) TemporalAnalysis SlurpTemporalAnalysisConfig // Temporal analysis settings Performance SlurpPerformanceConfig // Performance settings } type SlurpTemporalAnalysisConfig struct { MaxDecisionHops int // Max decision hops (default: 5) StalenessCheckInterval time.Duration // Staleness check interval (default: 5m) StalenessThreshold float64 // Staleness threshold (default: 0.2) } type SlurpPerformanceConfig struct { MaxConcurrentResolutions int // Max concurrent resolutions (default: 4) MetricsCollectionInterval time.Duration // Metrics interval (default: 1m) } ``` ### WHOOSH API Configuration ```go type WHOOSHAPIConfig struct { URL string // WHOOSH API URL (default: "http://localhost:3000") BaseURL string // Base URL (default: "http://localhost:3000") Token string // API token (required if enabled) Enabled bool // Enable WHOOSH integration (default: false) } ``` ## Runtime Configuration The `RuntimeConfig` type manages dynamic configuration with assignment overrides from WHOOSH. ### RuntimeConfig Structure ```go type RuntimeConfig struct { Base *Config // Base configuration from environment Override *AssignmentConfig // Assignment overrides from WHOOSH mu sync.RWMutex // Concurrent access protection reloadCh chan struct{} // Reload trigger channel } ``` ### Creating Runtime Configuration ```go // Load base configuration baseConfig, err := config.LoadFromEnvironment() if err != nil { return err } // Create runtime configuration manager runtimeConfig := config.NewRuntimeConfig(baseConfig) ``` ### Getting Effective Configuration The `GetConfig()` method returns a merged configuration with overrides applied: ```go // Get merged configuration (base + overrides) effectiveConfig := runtimeConfig.GetConfig() // Use effective configuration agentID := effectiveConfig.Agent.ID role := effectiveConfig.Agent.Role ``` ### Merge Behavior Runtime configuration merges base and override values: 1. **Override takes precedence**: If a field is set in the assignment override, it replaces the base value 2. **Base as fallback**: If a field is not set in the override, the base value is used 3. **Non-zero values**: Only non-zero/non-empty override values are applied **Example Merge**: ``` Base Config: Agent.ID: "chorus-default" Agent.Role: "backend_developer" Agent.MaxTasks: 3 Assignment Override: Agent.ID: "chorus-task-12345" Agent.Role: "frontend_developer" (MaxTasks not specified) Effective Config: Agent.ID: "chorus-task-12345" (from override) Agent.Role: "frontend_developer" (from override) Agent.MaxTasks: 3 (from base) ``` ## Dynamic Assignments CHORUS agents can fetch task-specific configuration from the WHOOSH API, enabling dynamic role assignment and configuration updates. ### Assignment Configuration Structure ```go type AssignmentConfig struct { // Assignment metadata AssignmentID string // Unique assignment identifier TaskSlot string // Docker Swarm task slot TaskID string // WHOOSH task ID ClusterID string // Cluster identifier AssignedAt time.Time // Assignment timestamp ExpiresAt time.Time // Optional expiration // Agent configuration overrides Agent *AgentConfig // Agent config overrides Network *NetworkConfig // Network config overrides AI *AIConfig // AI config overrides Logging *LoggingConfig // Logging config overrides // Bootstrap configuration for scaling BootstrapPeers []string // Bootstrap peer list JoinStagger int // Join stagger delay (ms) // Runtime capabilities RuntimeCapabilities []string // Additional capabilities // Key derivation for encryption RoleKey string // Role encryption key ClusterSecret string // Cluster secret // Custom fields Custom map[string]interface{} // Custom configuration } ``` ### Loading Assignments from WHOOSH ```go ctx := context.Background() assignURL := "https://whoosh.chorus.services/api/v1/assignments" // Fetch and apply assignment if err := runtimeCfg.LoadAssignment(ctx, assignURL); err != nil { log.Printf("Failed to load assignment: %v", err) } else { log.Println("Assignment loaded successfully") } ``` ### Assignment Request CHORUS sends the following information when requesting an assignment: ```go type AssignmentRequest struct { ClusterID string // Cluster identifier TaskSlot string // Docker task slot (from TASK_SLOT env var) TaskID string // Task ID (from TASK_ID env var) AgentID string // Agent identifier NodeID string // Node identifier (from NODE_ID env var) Timestamp time.Time // Request timestamp } ``` **HTTP Request**: ``` GET /api/v1/assignments?cluster_id=prod-cluster&agent_id=chorus-agent-1&node_id=node-001&task_slot=1&task_id=task-12345 Accept: application/json User-Agent: CHORUS-Agent/0.1.0 ``` ### Assignment Response WHOOSH should return a JSON response matching the `AssignmentConfig` structure: ```json { "assignment_id": "assign-67890", "task_slot": "1", "task_id": "task-12345", "cluster_id": "prod-cluster", "assigned_at": "2025-09-30T10:00:00Z", "expires_at": "2025-09-30T18:00:00Z", "agent": { "id": "chorus-frontend-specialist", "role": "frontend_developer", "specialization": "react_specialist", "max_tasks": 5, "capabilities": ["frontend", "ui", "react", "typescript"], "models": ["claude-sonnet-3.5"], "project": "web-redesign", "expertise": ["react", "tailwind", "accessibility"], "reports_to": "project_manager", "deliverables": ["component_library", "responsive_layouts"] }, "bootstrap_peers": [ "/ip4/10.0.1.10/tcp/9000/p2p/12D3KooWPeer1", "/ip4/10.0.1.11/tcp/9000/p2p/12D3KooWPeer2" ], "join_stagger": 2000, "runtime_capabilities": ["hot_reload", "fast_refresh"], "role_key": "age1xxxxxx...", "cluster_secret": "cluster-secret-xyz", "custom": { "build_target": "production", "enable_profiling": true } } ``` ### SIGHUP Configuration Reload CHORUS agents can reload configuration dynamically by sending a SIGHUP signal: ```bash # Find the CHORUS agent process docker ps | grep chorus-agent # Send SIGHUP to reload configuration docker exec kill -SIGHUP 1 # Or use Docker kill command docker kill --signal=SIGHUP ``` The reload handler automatically: 1. Fetches the latest assignment from WHOOSH 2. Merges the new assignment with base configuration 3. Applies the updated configuration without restart **Reload Handler Setup**: ```go // Start SIGHUP reload handler ctx := context.Background() assignURL := "https://whoosh.chorus.services/api/v1/assignments" runtimeCfg.StartReloadHandler(ctx, assignURL) // Agent continues running with updated configuration // Log output on SIGHUP: // 📡 Received SIGHUP, reloading assignment configuration... // ✅ Assignment configuration reloaded successfully ``` ### Manual Reload Trigger ```go // Trigger manual reload (without SIGHUP signal) runtimeCfg.Reload() ``` ### Bootstrap Peers with Assignment Override Bootstrap peers can be configured from multiple sources with priority: ```go // Get bootstrap peers with override support peers := runtimeCfg.GetBootstrapPeers() // Priority order: // 1. Assignment override from WHOOSH (highest priority) // 2. JSON bootstrap configuration (BOOTSTRAP_JSON env var) // 3. Environment variable CSV (CHORUS_BOOTSTRAP_PEERS) ``` **Bootstrap JSON Format** (`BOOTSTRAP_JSON` env var): ```json { "peers": [ { "address": "/ip4/10.0.1.10/tcp/9000/p2p/12D3KooWPeer1", "priority": 100, "region": "us-east", "roles": ["coordinator"], "enabled": true }, { "address": "/ip4/10.0.1.11/tcp/9000/p2p/12D3KooWPeer2", "priority": 90, "region": "us-west", "roles": ["worker"], "enabled": true } ], "metadata": { "generated_at": "2025-09-30T10:00:00Z", "cluster_id": "prod-cluster", "version": "1.0", "notes": "Production bootstrap peers" } } ``` ### Join Stagger with Assignment Override Join stagger prevents thundering herd problems during scaling: ```go // Get join stagger delay with override support stagger := runtimeCfg.GetJoinStagger() if stagger > 0 { time.Sleep(stagger) } // Priority order: // 1. Assignment override (join_stagger field) // 2. Environment variable (CHORUS_JOIN_STAGGER_MS) ``` ## Role Definitions CHORUS uses predefined roles for access control and capability management. ### Authority Levels ```go const ( AuthorityMaster AuthorityLevel = "master" // Highest authority AuthorityAdmin AuthorityLevel = "admin" // Administrative AuthorityDecision AuthorityLevel = "decision" // Can make decisions AuthorityCoordination AuthorityLevel = "coordination" // Can coordinate AuthorityFull AuthorityLevel = "full" // Full execution AuthoritySuggestion AuthorityLevel = "suggestion" // Suggestion only AuthorityReadOnly AuthorityLevel = "readonly" // Read-only access ) ``` ### Predefined Roles | Role | Description | Authority Level | Access Level | Capabilities | |------|-------------|-----------------|--------------|--------------| | `project_manager` | Project coordination and management | Master | High | coordination, planning, oversight | | `backend_developer` | Backend development and API work | Decision | Medium | backend, api, database | | `frontend_developer` | Frontend UI development | Coordination | Medium | frontend, ui, components | | `devops_engineer` | Infrastructure and deployment | Decision | High | infrastructure, deployment, monitoring | | `security_engineer` | Security oversight and hardening | Master | High | security, audit, compliance | | `security_expert` | Advanced security analysis | Master | High | security, policy, response | | `senior_software_architect` | Architecture governance | Decision | High | architecture, design, coordination | | `qa_engineer` | Quality assurance and testing | Coordination | Medium | testing, validation | | `readonly_user` | Read-only observer | ReadOnly | Low | observation | | `suggestion_only_role` | Propose suggestions only | Suggestion | Low | recommendation | ### Role Definition Structure ```go type RoleDefinition struct { Name string // Role name Description string // Role description Capabilities []string // Role capabilities AccessLevel string // Access level: "low", "medium", "high" AuthorityLevel AuthorityLevel // Authority level Keys *AgeKeyPair // Encryption keys CanDecrypt []string // Roles this role can decrypt } ``` ### Using Role Definitions ```go // Get all predefined roles roles := config.GetPredefinedRoles() // Get specific role pmRole := roles["project_manager"] // Get role authority level from configuration authority, err := cfg.GetRoleAuthority("backend_developer") if err != nil { log.Fatalf("Unknown role: %v", err) } // Check if agent can decrypt content for target role canDecrypt, err := cfg.CanDecryptRole("frontend_developer") if err != nil || !canDecrypt { log.Println("Cannot decrypt content for frontend_developer") } ``` ### Encryption Key Management Roles support Age encryption key pairs for secure communication: ```go type AgeKeyPair struct { PublicKey string // Age public key PrivateKey string // Age private key } ``` ## Hybrid Configuration The `HybridConfig` manages feature flags for Phase 2 hybrid mode, supporting gradual migration from mock to real implementations. ### Hybrid Configuration Structure ```go type HybridConfig struct { DHT HybridDHTConfig // DHT configuration UCXL HybridUCXLConfig // UCXL configuration Discovery DiscoveryConfig // Discovery configuration Monitoring MonitoringConfig // Monitoring configuration } ``` ### Hybrid DHT Configuration ```go type HybridDHTConfig struct { Backend string // "mock", "real", or "hybrid" BootstrapNodes []string // Bootstrap nodes for real DHT FallbackOnError bool // Fallback to mock on error (default: true) HealthCheckInterval time.Duration // Health check interval (default: 30s) MaxRetries int // Max retries (default: 3) RetryBackoff time.Duration // Retry backoff (default: 1s) OperationTimeout time.Duration // Operation timeout (default: 10s) } ``` **DHT Backend Modes**: - `"mock"`: Use mock DHT only (default) - `"real"`: Use real libp2p DHT only - `"hybrid"`: Try real DHT, fallback to mock on error ### Hybrid UCXL Configuration ```go type HybridUCXLConfig struct { CacheEnabled bool // Enable caching (default: true) CacheTTL time.Duration // Cache TTL (default: 5m) UseDistributed bool // Use distributed UCXL (default: false) MaxCacheSize int // Max cache entries (default: 10000) } ``` ### Discovery Configuration ```go type DiscoveryConfig struct { MDNSEnabled bool // Enable mDNS (default: true) DHTDiscovery bool // Enable DHT discovery (default: false) AnnounceInterval time.Duration // Announce interval (default: 30s) ServiceName string // Service name (default: "CHORUS") // Rate limiting for scaling DialsPerSecond int // Dials per second (default: 5) MaxConcurrentDHT int // Max concurrent DHT ops (default: 16) MaxConcurrentDials int // Max concurrent dials (default: 10) JoinStaggerMS int // Join stagger (default: 0) } ``` ### Monitoring Configuration ```go type MonitoringConfig struct { Enabled bool // Enable monitoring (default: true) MetricsInterval time.Duration // Metrics interval (default: 15s) HealthEndpoint string // Health endpoint (default: "/health") MetricsEndpoint string // Metrics endpoint (default: "/metrics") } ``` ### Loading Hybrid Configuration ```go // Load hybrid configuration from environment hybridCfg, err := config.LoadHybridConfig() if err != nil { log.Fatalf("Failed to load hybrid configuration: %v", err) } // Check DHT mode if hybridCfg.IsRealDHTEnabled() { log.Println("Real DHT is enabled") } if hybridCfg.IsMockDHTEnabled() { log.Println("Mock DHT is enabled") } if hybridCfg.IsFallbackEnabled() { log.Println("Fallback to mock DHT is enabled") } // Get bootstrap nodes bootstrapNodes := hybridCfg.GetDHTBootstrapNodes() ``` ## Security Configuration Security settings for key rotation, audit logging, and election behavior. ### Security Configuration Structure ```go type SecurityConfig struct { KeyRotationDays int // Key rotation interval (default: 30) AuditLogging bool // Enable audit logging (default: true) AuditPath string // Audit log path (default: "/tmp/chorus-audit.log") ElectionConfig ElectionConfig // Election configuration } ``` ### Election Configuration ```go type ElectionConfig struct { DiscoveryTimeout time.Duration // Discovery timeout (default: 15s) HeartbeatTimeout time.Duration // Heartbeat timeout (default: 30s) ElectionTimeout time.Duration // Election timeout (default: 60s) DiscoveryBackoff time.Duration // Discovery backoff (default: 5s) LeadershipScoring *LeadershipScoring // Leadership scoring weights } type LeadershipScoring struct { UptimeWeight float64 // Uptime weight (default: 0.4) CapabilityWeight float64 // Capability weight (default: 0.3) ExperienceWeight float64 // Experience weight (default: 0.2) LoadWeight float64 // Load weight (default: 0.1) } ``` ## Environment Variables Reference ### Required Variables | Variable | Type | Description | |----------|------|-------------| | `CHORUS_LICENSE_ID` or `CHORUS_LICENSE_ID_FILE` | string | License identifier (required) | ### Agent Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_AGENT_ID` | string | auto-generated | Unique agent identifier | | `CHORUS_SPECIALIZATION` | string | "general_developer" | Agent specialization | | `CHORUS_MAX_TASKS` | int | 3 | Maximum concurrent tasks | | `CHORUS_CAPABILITIES` | []string | ["general_development", "task_coordination"] | Agent capabilities (CSV) | | `CHORUS_MODELS` | []string | ["meta/llama-3.1-8b-instruct"] | Available AI models (CSV) | | `CHORUS_ROLE` | string | "" | Agent role (see Role Definitions) | | `CHORUS_PROJECT` | string | "chorus" | Project assignment | | `CHORUS_EXPERTISE` | []string | [] | Domain expertise (CSV) | | `CHORUS_REPORTS_TO` | string | "" | Reporting hierarchy | | `CHORUS_DELIVERABLES` | []string | [] | Expected deliverables (CSV) | | `CHORUS_MODEL_SELECTION_WEBHOOK` | string | "" | Model selection webhook URL | | `CHORUS_DEFAULT_REASONING_MODEL` | string | "meta/llama-3.1-8b-instruct" | Default AI model | ### Network Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_P2P_PORT` | int | 9000 | P2P network port | | `CHORUS_API_PORT` | int | 8080 | API server port | | `CHORUS_HEALTH_PORT` | int | 8081 | Health check port | | `CHORUS_BIND_ADDRESS` | string | "0.0.0.0" | Bind address | ### License Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_LICENSE_ID` | string | - | **Required**: License identifier | | `CHORUS_LICENSE_ID_FILE` | path | - | Path to file containing license ID | | `CHORUS_CLUSTER_ID` | string | "default-cluster" | Cluster identifier | | `CHORUS_ORGANIZATION_NAME` | string | "" | Organization name | | `CHORUS_KACHING_URL` | string | "https://kaching.chorus.services" | License validation URL | | `CHORUS_GRACE_PERIOD_HOURS` | int | 72 | Grace period in hours | ### AI Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_AI_PROVIDER` | string | "resetdata" | AI provider ("ollama" or "resetdata") | | `OLLAMA_ENDPOINT` | string | "http://localhost:11434" | Ollama endpoint | | `OLLAMA_TIMEOUT` | duration | 30s | Ollama request timeout | | `RESETDATA_BASE_URL` | string | "https://models.au-syd.resetdata.ai/v1" | ResetData API URL | | `RESETDATA_API_KEY` | string | - | ResetData API key | | `RESETDATA_API_KEY_FILE` | path | - | Path to file containing API key | | `RESETDATA_MODEL` | string | "meta/llama-3.1-8b-instruct" | ResetData model name | | `RESETDATA_TIMEOUT` | duration | 30s | ResetData request timeout | ### Logging Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `LOG_LEVEL` | string | "info" | Log level: debug, info, warn, error | | `LOG_FORMAT` | string | "structured" | Log format: structured or text | ### DHT Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_DHT_ENABLED` | bool | true | Enable DHT | | `CHORUS_BOOTSTRAP_PEERS` | []string | [] | Bootstrap peer multiaddrs (CSV) | | `CHORUS_MDNS_ENABLED` | bool | true | Enable mDNS discovery | | `BOOTSTRAP_JSON` | path | - | Path to bootstrap JSON config | | `CHORUS_JOIN_STAGGER_MS` | int | 0 | Join stagger delay in milliseconds | ### UCXL Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_UCXL_ENABLED` | bool | true | Enable UCXL | | `CHORUS_UCXL_SERVER_ENABLED` | bool | true | Enable UCXL server | | `CHORUS_UCXL_SERVER_PORT` | int | 8082 | UCXL server port | | `CHORUS_UCXL_SERVER_BASE_PATH` | string | "" | UCXL server base path | | `CHORUS_UCXL_STORAGE_DIRECTORY` | string | "/tmp/chorus-ucxi-storage" | UCXL storage directory | | `CHORUS_UCXL_CACHE_TTL` | duration | 1h | UCXL cache TTL | ### SLURP Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_SLURP_ENABLED` | bool | false | Enable SLURP integration | | `CHORUS_SLURP_API_BASE_URL` | string | "http://localhost:9090" | SLURP API URL | | `CHORUS_SLURP_API_KEY` | string | - | SLURP API key | | `CHORUS_SLURP_API_KEY_FILE` | path | - | Path to file containing API key | | `CHORUS_SLURP_API_TIMEOUT` | duration | 15s | SLURP request timeout | | `CHORUS_SLURP_API_RETRY_COUNT` | int | 3 | SLURP retry count | | `CHORUS_SLURP_API_RETRY_DELAY` | duration | 2s | SLURP retry delay | | `CHORUS_SLURP_MAX_DECISION_HOPS` | int | 5 | Max decision hops | | `CHORUS_SLURP_STALENESS_CHECK_INTERVAL` | duration | 5m | Staleness check interval | | `CHORUS_SLURP_MAX_CONCURRENT_RESOLUTIONS` | int | 4 | Max concurrent resolutions | | `CHORUS_SLURP_METRICS_COLLECTION_INTERVAL` | duration | 1m | Metrics collection interval | ### WHOOSH API Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `WHOOSH_API_URL` | string | "http://localhost:3000" | WHOOSH API URL | | `WHOOSH_API_BASE_URL` | string | "http://localhost:3000" | WHOOSH base URL | | `WHOOSH_API_TOKEN` | string | - | WHOOSH API token | | `WHOOSH_API_ENABLED` | bool | false | Enable WHOOSH integration | ### Security Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_KEY_ROTATION_DAYS` | int | 30 | Key rotation interval in days | | `CHORUS_AUDIT_LOGGING` | bool | true | Enable audit logging | | `CHORUS_AUDIT_PATH` | string | "/tmp/chorus-audit.log" | Audit log file path | | `CHORUS_DISCOVERY_TIMEOUT` | duration | 15s | Discovery timeout | | `CHORUS_HEARTBEAT_TIMEOUT` | duration | 30s | Heartbeat timeout | | `CHORUS_ELECTION_TIMEOUT` | duration | 60s | Election timeout | | `CHORUS_DISCOVERY_BACKOFF` | duration | 5s | Discovery backoff | ### Hybrid Configuration | Variable | Type | Default | Description | |----------|------|---------|-------------| | `CHORUS_DHT_BACKEND` | string | "mock" | DHT backend: "mock", "real", or "hybrid" | | `CHORUS_DHT_BOOTSTRAP_NODES` | []string | [] | DHT bootstrap nodes (CSV) | | `CHORUS_FALLBACK_ON_ERROR` | bool | true | Fallback to mock on error | | `CHORUS_HEALTH_CHECK_INTERVAL` | duration | 30s | Health check interval | | `CHORUS_DHT_MAX_RETRIES` | int | 3 | DHT max retries | | `CHORUS_DHT_RETRY_BACKOFF` | duration | 1s | DHT retry backoff | | `CHORUS_DHT_OPERATION_TIMEOUT` | duration | 10s | DHT operation timeout | | `CHORUS_UCXL_CACHE_ENABLED` | bool | true | Enable UCXL caching | | `CHORUS_UCXL_CACHE_TTL` | duration | 5m | UCXL cache TTL | | `CHORUS_UCXL_USE_DISTRIBUTED` | bool | false | Use distributed UCXL | | `CHORUS_UCXL_MAX_CACHE_SIZE` | int | 10000 | Max UCXL cache size | | `CHORUS_MDNS_ENABLED` | bool | true | Enable mDNS discovery | | `CHORUS_DHT_DISCOVERY` | bool | false | Enable DHT discovery | | `CHORUS_ANNOUNCE_INTERVAL` | duration | 30s | Announce interval | | `CHORUS_SERVICE_NAME` | string | "CHORUS" | Service name | | `CHORUS_DIALS_PER_SEC` | int | 5 | Dials per second rate limit | | `CHORUS_MAX_CONCURRENT_DHT` | int | 16 | Max concurrent DHT operations | | `CHORUS_MAX_CONCURRENT_DIALS` | int | 10 | Max concurrent dials | | `CHORUS_JOIN_STAGGER_MS` | int | 0 | Join stagger in milliseconds | | `CHORUS_MONITORING_ENABLED` | bool | true | Enable monitoring | | `CHORUS_METRICS_INTERVAL` | duration | 15s | Metrics collection interval | | `CHORUS_HEALTH_ENDPOINT` | string | "/health" | Health endpoint path | | `CHORUS_METRICS_ENDPOINT` | string | "/metrics" | Metrics endpoint path | ### Docker-specific Variables | Variable | Description | |----------|-------------| | `HOSTNAME` | Container hostname (set by Docker) | | `TASK_SLOT` | Docker Swarm task slot number | | `TASK_ID` | Docker Swarm task ID | | `NODE_ID` | Docker Swarm node ID | ## Assignment Schema ### Full Assignment JSON Schema ```json { "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "assignment_id": { "type": "string", "description": "Unique assignment identifier" }, "task_slot": { "type": "string", "description": "Docker Swarm task slot number" }, "task_id": { "type": "string", "description": "WHOOSH task identifier" }, "cluster_id": { "type": "string", "description": "Cluster identifier" }, "assigned_at": { "type": "string", "format": "date-time", "description": "Assignment timestamp" }, "expires_at": { "type": "string", "format": "date-time", "description": "Optional expiration timestamp" }, "agent": { "type": "object", "properties": { "id": {"type": "string"}, "specialization": {"type": "string"}, "max_tasks": {"type": "integer"}, "capabilities": {"type": "array", "items": {"type": "string"}}, "models": {"type": "array", "items": {"type": "string"}}, "role": {"type": "string"}, "project": {"type": "string"}, "expertise": {"type": "array", "items": {"type": "string"}}, "reports_to": {"type": "string"}, "deliverables": {"type": "array", "items": {"type": "string"}}, "model_selection_webhook": {"type": "string"}, "default_reasoning_model": {"type": "string"} } }, "network": { "type": "object", "properties": { "p2p_port": {"type": "integer"}, "api_port": {"type": "integer"}, "health_port": {"type": "integer"}, "bind_address": {"type": "string"} } }, "ai": { "type": "object", "properties": { "provider": {"type": "string"}, "ollama": { "type": "object", "properties": { "endpoint": {"type": "string"}, "timeout": {"type": "string"} } }, "resetdata": { "type": "object", "properties": { "base_url": {"type": "string"}, "model": {"type": "string"}, "timeout": {"type": "string"} } } } }, "logging": { "type": "object", "properties": { "level": {"type": "string"}, "format": {"type": "string"} } }, "bootstrap_peers": { "type": "array", "items": {"type": "string"}, "description": "Bootstrap peer multiaddrs" }, "join_stagger": { "type": "integer", "description": "Join stagger delay in milliseconds" }, "runtime_capabilities": { "type": "array", "items": {"type": "string"}, "description": "Additional runtime capabilities" }, "role_key": { "type": "string", "description": "Role encryption key" }, "cluster_secret": { "type": "string", "description": "Cluster secret" }, "custom": { "type": "object", "description": "Custom configuration fields", "additionalProperties": true } }, "required": ["assignment_id", "cluster_id", "assigned_at"] } ``` ### Example Assignment Scenarios #### Scenario 1: Frontend Developer Assignment ```json { "assignment_id": "frontend-task-001", "task_slot": "1", "task_id": "web-redesign-homepage", "cluster_id": "prod-cluster", "assigned_at": "2025-09-30T10:00:00Z", "expires_at": "2025-09-30T18:00:00Z", "agent": { "id": "chorus-frontend-1", "role": "frontend_developer", "specialization": "react_specialist", "max_tasks": 3, "capabilities": ["frontend", "ui", "react", "typescript"], "models": ["claude-sonnet-3.5"], "project": "web-redesign", "expertise": ["react", "tailwind", "accessibility"], "reports_to": "project_manager", "deliverables": ["responsive_homepage", "mobile_navigation"] }, "logging": { "level": "debug" } } ``` #### Scenario 2: DevOps Engineer with Bootstrap Peers ```json { "assignment_id": "devops-scaling-001", "task_slot": "5", "task_id": "scale-infrastructure", "cluster_id": "prod-cluster", "assigned_at": "2025-09-30T11:00:00Z", "agent": { "id": "chorus-devops-5", "role": "devops_engineer", "specialization": "kubernetes_specialist", "max_tasks": 5, "capabilities": ["infrastructure", "deployment", "monitoring", "kubernetes"], "models": ["claude-sonnet-3.5", "gpt-4"], "project": "infrastructure-modernization" }, "bootstrap_peers": [ "/ip4/10.0.1.10/tcp/9000/p2p/12D3KooWCoordinator1", "/ip4/10.0.1.11/tcp/9000/p2p/12D3KooWCoordinator2" ], "join_stagger": 3000, "custom": { "deployment_region": "us-east-1", "enable_autoscaling": true, "max_replicas": 10 } } ``` #### Scenario 3: Minimal Assignment ```json { "assignment_id": "minimal-001", "cluster_id": "test-cluster", "assigned_at": "2025-09-30T12:00:00Z", "agent": { "role": "backend_developer" } } ``` ## Configuration Validation ### Validation Rules The `Validate()` method checks configuration for: 1. **Required Fields**: - `License.LicenseID` must be provided 2. **Agent ID Auto-generation**: - If `Agent.ID` is empty, it's auto-generated from hostname or container ID - Format: `chorus-` or `chorus-` 3. **Hybrid Configuration Validation**: - DHT backend must be one of: "mock", "real", "hybrid" - Health check interval must be >= 1 second - Operation timeout must be >= 100 milliseconds - Cache size must be non-negative ### Validation Examples ```go // Validate configuration cfg, err := config.LoadFromEnvironment() if err != nil { log.Fatalf("Failed to load config: %v", err) } if err := cfg.Validate(); err != nil { log.Fatalf("Invalid configuration: %v", err) } // Validate hybrid configuration hybridCfg, err := config.LoadHybridConfig() if err != nil { log.Fatalf("Failed to load hybrid config: %v", err) } if err := hybridCfg.Validate(); err != nil { log.Fatalf("Invalid hybrid configuration: %v", err) } ``` ### Common Validation Errors | Error | Cause | Solution | |-------|-------|----------| | "CHORUS_LICENSE_ID is required" | Missing license ID | Set `CHORUS_LICENSE_ID` or `CHORUS_LICENSE_ID_FILE` | | "invalid DHT backend 'xyz'" | Invalid DHT backend | Use "mock", "real", or "hybrid" | | "health check interval too short" | Health check < 1s | Increase `CHORUS_HEALTH_CHECK_INTERVAL` | | "operation timeout too short" | Operation timeout < 100ms | Increase `CHORUS_DHT_OPERATION_TIMEOUT` | | "max cache size must be non-negative" | Negative cache size | Set `CHORUS_UCXL_MAX_CACHE_SIZE` >= 0 | ## Usage Examples ### Example 1: Basic Configuration Loading ```go package main import ( "log" "github.com/chorus/pkg/config" ) func main() { // Load configuration from environment cfg, err := config.LoadFromEnvironment() if err != nil { log.Fatalf("Failed to load configuration: %v", err) } log.Printf("Agent ID: %s", cfg.Agent.ID) log.Printf("Agent Role: %s", cfg.Agent.Role) log.Printf("API Port: %d", cfg.Network.APIPort) } ``` ### Example 2: Runtime Configuration with WHOOSH ```go package main import ( "context" "log" "os" "github.com/chorus/pkg/config" ) func main() { ctx := context.Background() // Load runtime configuration runtimeCfg, err := config.LoadRuntimeConfig() if err != nil { log.Fatalf("Failed to load runtime config: %v", err) } // Fetch assignment from WHOOSH (if enabled) whooshURL := os.Getenv("WHOOSH_API_URL") if whooshURL != "" { assignURL := whooshURL + "/api/v1/assignments" if err := runtimeCfg.LoadAssignment(ctx, assignURL); err != nil { log.Printf("No assignment available: %v", err) } else { log.Println("Assignment loaded successfully") } } // Start SIGHUP reload handler go runtimeCfg.StartReloadHandler(ctx, assignURL) // Get effective configuration cfg := runtimeCfg.GetConfig() log.Printf("Effective Agent ID: %s", cfg.Agent.ID) log.Printf("Effective Role: %s", cfg.Agent.Role) // Check assignment info if assignment := runtimeCfg.GetAssignmentInfo(); assignment != nil { log.Printf("Assignment ID: %s", assignment.AssignmentID) log.Printf("Task ID: %s", assignment.TaskID) log.Printf("Assigned At: %s", assignment.AssignedAt) } } ``` ### Example 3: Bootstrap Peers with Priority ```go package main import ( "log" "os" "github.com/chorus/pkg/config" ) func main() { // Set bootstrap JSON path os.Setenv("BOOTSTRAP_JSON", "/etc/chorus/bootstrap.json") // Load runtime configuration runtimeCfg, err := config.LoadRuntimeConfig() if err != nil { log.Fatalf("Failed to load config: %v", err) } // Get bootstrap peers (assignment > JSON > env var) peers := runtimeCfg.GetBootstrapPeers() log.Printf("Bootstrap peers: %v", peers) // Get join stagger stagger := runtimeCfg.GetJoinStagger() log.Printf("Join stagger: %s", stagger) } ``` ### Example 4: Role-Based Access Control ```go package main import ( "log" "github.com/chorus/pkg/config" ) func main() { // Load configuration cfg, err := config.LoadFromEnvironment() if err != nil { log.Fatalf("Failed to load config: %v", err) } // Get role authority level authority, err := cfg.GetRoleAuthority(cfg.Agent.Role) if err != nil { log.Fatalf("Invalid role: %v", err) } log.Printf("Authority level: %s", authority) // Check decryption capability canDecrypt, err := cfg.CanDecryptRole("frontend_developer") if err != nil { log.Fatalf("Error checking decrypt capability: %v", err) } if canDecrypt { log.Println("Can decrypt frontend_developer content") } else { log.Println("Cannot decrypt frontend_developer content") } // Get all predefined roles roles := config.GetPredefinedRoles() for name, role := range roles { log.Printf("Role: %s, Authority: %s, Access: %s", name, role.AuthorityLevel, role.AccessLevel) } } ``` ### Example 5: Hybrid Configuration Mode ```go package main import ( "log" "os" "github.com/chorus/pkg/config" ) func main() { // Set hybrid mode environment variables os.Setenv("CHORUS_DHT_BACKEND", "hybrid") os.Setenv("CHORUS_FALLBACK_ON_ERROR", "true") os.Setenv("CHORUS_UCXL_USE_DISTRIBUTED", "true") // Load hybrid configuration hybridCfg, err := config.LoadHybridConfig() if err != nil { log.Fatalf("Failed to load hybrid config: %v", err) } // Check DHT mode log.Printf("Real DHT enabled: %v", hybridCfg.IsRealDHTEnabled()) log.Printf("Mock DHT enabled: %v", hybridCfg.IsMockDHTEnabled()) log.Printf("Fallback enabled: %v", hybridCfg.IsFallbackEnabled()) // Get DHT bootstrap nodes nodes := hybridCfg.GetDHTBootstrapNodes() log.Printf("Bootstrap nodes: %v", nodes) } ``` ### Example 6: Docker Secrets Support ```go package main import ( "log" "os" "github.com/chorus/pkg/config" ) func main() { // Use Docker secrets for sensitive values os.Setenv("CHORUS_LICENSE_ID_FILE", "/run/secrets/chorus_license") os.Setenv("RESETDATA_API_KEY_FILE", "/run/secrets/resetdata_api_key") os.Setenv("CHORUS_SLURP_API_KEY_FILE", "/run/secrets/slurp_api_key") // Load configuration (reads from files automatically) cfg, err := config.LoadFromEnvironment() if err != nil { log.Fatalf("Failed to load config: %v", err) } // License ID read from file log.Printf("License ID loaded from file") // API keys read from files if cfg.AI.ResetData.APIKey != "" { log.Println("ResetData API key loaded") } if cfg.Slurp.APIKey != "" { log.Println("SLURP API key loaded") } } ``` ### Example 7: Configuration Watcher ```go package main import ( "log" "time" "github.com/chorus/pkg/config" ) func main() { // Load hybrid configuration hybridCfg, err := config.LoadHybridConfig() if err != nil { log.Fatalf("Failed to load config: %v", err) } // Create configuration watcher watcher := config.NewConfigWatcher(hybridCfg) defer watcher.Close() // Start watching for configuration changes go func() { for event := range watcher.Events() { log.Printf("Configuration changed: %s (%v -> %v)", event.Component, event.Old, event.New) } }() // Simulate configuration change time.Sleep(2 * time.Second) if err := watcher.UpdateDHTBackend("real"); err != nil { log.Printf("Failed to update DHT backend: %v", err) } // Keep running select {} } ``` ## Related Documentation - **BZZZ Integration**: `/home/tony/chorus/project-queues/active/BZZZ/docs/api.md` - **WHOOSH API**: `/home/tony/chorus/project-queues/active/WHOOSH/docs/api.md` - **SLURP Integration**: `/home/tony/chorus/project-queues/active/CHORUS/docs/slurp-integration.md` - **UCXL Protocol**: `/home/tony/chorus/project-queues/active/RUSTLE/docs/ucxl-spec.md` - **Docker Deployment**: `/home/tony/chorus/project-queues/active/CHORUS/docs/deployment.md` ## Summary The CHORUS configuration package provides: 1. **Environment-based Configuration**: All configuration via environment variables for containerized deployments 2. **Runtime Assignment Support**: Dynamic configuration from WHOOSH with SIGHUP reload 3. **Role-Based Access Control**: Predefined roles with authority levels and encryption keys 4. **Hybrid Mode Support**: Feature flags for gradual migration from mock to real implementations 5. **Docker Secrets Support**: Read sensitive values from files for Docker secrets integration 6. **Comprehensive Validation**: Required field checks and constraint validation 7. **Bootstrap Configuration**: Multiple sources with priority (assignment > JSON > env var) 8. **Configuration Merging**: Clean merge semantics for base + override configuration For questions or issues, refer to the test file at `/home/tony/chorus/project-queues/active/CHORUS/pkg/config/config_test.go` for additional usage examples.