# Package: pkg/coordination **Location**: `/home/tony/chorus/project-queues/active/CHORUS/pkg/coordination/` ## Overview The `pkg/coordination` package provides **advanced cross-repository coordination primitives** for managing complex task dependencies and multi-agent collaboration in CHORUS. It includes AI-powered dependency detection, meta-coordination sessions, and automated escalation handling to enable sophisticated distributed development workflows. ## Architecture ### Coordination Layers ``` ┌─────────────────────────────────────────────────┐ │ MetaCoordinator │ │ - Session management │ │ - AI-powered coordination planning │ │ - Escalation handling │ │ - SLURP integration │ └─────────────────┬───────────────────────────────┘ │ ┌─────────────────▼───────────────────────────────┐ │ DependencyDetector │ │ - Cross-repo dependency detection │ │ - Rule-based pattern matching │ │ - Relationship analysis │ └─────────────────┬───────────────────────────────┘ │ ┌─────────────────▼───────────────────────────────┐ │ PubSub (HMMM Meta-Discussion) │ │ - Coordination messages │ │ - Session broadcasts │ │ - Escalation notifications │ └─────────────────────────────────────────────────┘ ``` ## Core Components ### MetaCoordinator Manages advanced cross-repository coordination and multi-agent collaboration sessions. ```go type MetaCoordinator struct { pubsub *pubsub.PubSub ctx context.Context dependencyDetector *DependencyDetector slurpIntegrator *integration.SlurpEventIntegrator // Active coordination sessions activeSessions map[string]*CoordinationSession sessionLock sync.RWMutex // Configuration maxSessionDuration time.Duration // Default: 30 minutes maxParticipants int // Default: 5 escalationThreshold int // Default: 10 messages } ``` **Key Responsibilities:** - Create and manage coordination sessions - Generate AI-powered coordination plans - Monitor session progress and health - Escalate to humans when needed - Generate SLURP events from coordination outcomes - Integrate with HMMM for meta-discussion ### DependencyDetector Analyzes tasks across repositories to detect relationships and dependencies. ```go type DependencyDetector struct { pubsub *pubsub.PubSub ctx context.Context knownTasks map[string]*TaskContext dependencyRules []DependencyRule coordinationHops int // Default: 3 } ``` **Key Responsibilities:** - Track tasks across multiple repositories - Apply pattern-based dependency detection rules - Identify task relationships (API contracts, schema changes, etc.) - Broadcast dependency alerts - Trigger coordination sessions ### CoordinationSession Represents an active multi-agent coordination session. ```go type CoordinationSession struct { SessionID string Type string // dependency, conflict, planning Participants map[string]*Participant TasksInvolved []*TaskContext Messages []CoordinationMessage Status string // active, resolved, escalated CreatedAt time.Time LastActivity time.Time Resolution string EscalationReason string } ``` **Session Types:** - **dependency**: Coordinating dependent tasks across repos - **conflict**: Resolving conflicts or competing changes - **planning**: Joint planning for complex multi-repo features **Session States:** - **active**: Session in progress - **resolved**: Consensus reached, coordination complete - **escalated**: Requires human intervention ## Data Structures ### TaskContext Represents a task with its repository and project context for dependency analysis. ```go type TaskContext struct { TaskID int ProjectID int Repository string Title string Description string Keywords []string AgentID string ClaimedAt time.Time } ``` ### Participant Represents an agent participating in a coordination session. ```go type Participant struct { AgentID string PeerID string Repository string Capabilities []string LastSeen time.Time Active bool } ``` ### CoordinationMessage A message within a coordination session. ```go type CoordinationMessage struct { MessageID string FromAgentID string FromPeerID string Content string MessageType string // proposal, question, agreement, concern Timestamp time.Time Metadata map[string]interface{} } ``` **Message Types:** - **proposal**: Proposed solution or approach - **question**: Request for clarification - **agreement**: Agreement with proposal - **concern**: Concern or objection ### TaskDependency Represents a detected relationship between tasks. ```go type TaskDependency struct { Task1 *TaskContext Task2 *TaskContext Relationship string // Rule name (e.g., "API_Contract") Confidence float64 // 0.0 - 1.0 Reason string // Human-readable explanation DetectedAt time.Time } ``` ### DependencyRule Defines how to detect task relationships. ```go type DependencyRule struct { Name string Description string Keywords []string Validator func(task1, task2 *TaskContext) (bool, string) } ``` ## Dependency Detection ### Built-in Detection Rules #### 1. API Contract Rule Detects dependencies between API definitions and implementations. ```go { Name: "API_Contract", Description: "Tasks involving API contracts and implementations", Keywords: []string{"api", "endpoint", "contract", "interface", "schema"}, Validator: func(task1, task2 *TaskContext) (bool, string) { text1 := strings.ToLower(task1.Title + " " + task1.Description) text2 := strings.ToLower(task2.Title + " " + task2.Description) if (strings.Contains(text1, "api") && strings.Contains(text2, "implement")) || (strings.Contains(text2, "api") && strings.Contains(text1, "implement")) { return true, "API definition and implementation dependency" } return false, "" }, } ``` **Example Detection:** - Task 1: "Define user authentication API" - Task 2: "Implement authentication endpoint" - **Detected**: API_Contract dependency #### 2. Database Schema Rule Detects schema changes affecting multiple services. ```go { Name: "Database_Schema", Description: "Database schema changes affecting multiple services", Keywords: []string{"database", "schema", "migration", "table", "model"}, Validator: func(task1, task2 *TaskContext) (bool, string) { // Checks for database-related keywords in both tasks // Returns true if both tasks involve database work }, } ``` **Example Detection:** - Task 1: "Add user preferences table" - Task 2: "Update user service for preferences" - **Detected**: Database_Schema dependency #### 3. Configuration Dependency Rule Detects configuration changes affecting multiple components. ```go { Name: "Configuration_Dependency", Description: "Configuration changes affecting multiple components", Keywords: []string{"config", "environment", "settings", "parameters"}, } ``` **Example Detection:** - Task 1: "Add feature flag for new UI" - Task 2: "Implement feature flag checks in backend" - **Detected**: Configuration_Dependency #### 4. Security Compliance Rule Detects security changes requiring coordinated implementation. ```go { Name: "Security_Compliance", Description: "Security changes requiring coordinated implementation", Keywords: []string{"security", "auth", "permission", "token", "encrypt"}, } ``` **Example Detection:** - Task 1: "Implement JWT token refresh" - Task 2: "Update authentication middleware" - **Detected**: Security_Compliance dependency ### Custom Rules Add project-specific dependency detection: ```go customRule := DependencyRule{ Name: "GraphQL_Schema", Description: "GraphQL schema and resolver dependencies", Keywords: []string{"graphql", "schema", "resolver", "query", "mutation"}, Validator: func(task1, task2 *TaskContext) (bool, string) { text1 := strings.ToLower(task1.Title + " " + task1.Description) text2 := strings.ToLower(task2.Title + " " + task2.Description) hasSchema := strings.Contains(text1, "schema") || strings.Contains(text2, "schema") hasResolver := strings.Contains(text1, "resolver") || strings.Contains(text2, "resolver") if hasSchema && hasResolver { return true, "GraphQL schema and resolver must be coordinated" } return false, "" }, } dependencyDetector.AddCustomRule(customRule) ``` ## Coordination Flow ### 1. Task Registration and Detection ``` Task Claimed by Agent A → RegisterTask() → DependencyDetector ↓ detectDependencies() ↓ Apply all dependency rules to known tasks ↓ Dependency detected? → Yes → announceDependency() ↓ ↓ No MetaCoordinator ``` ### 2. Dependency Announcement ```go // Dependency detector announces to HMMM meta-discussion coordMsg := map[string]interface{}{ "message_type": "dependency_detected", "dependency": dep, "coordination_request": "Cross-repository dependency detected...", "agents_involved": [agentA, agentB], "repositories": [repoA, repoB], "hop_count": 0, "max_hops": 3, } pubsub.PublishHmmmMessage(MetaDiscussion, coordMsg) ``` ### 3. Session Creation ``` MetaCoordinator receives dependency_detected message ↓ handleDependencyDetection() ↓ Create CoordinationSession ↓ Add participating agents ↓ Generate AI coordination plan ↓ Broadcast plan to participants ``` ### 4. AI-Powered Coordination Planning ```go prompt := ` You are an expert AI project coordinator managing a distributed development team. SITUATION: - A dependency has been detected between two tasks in different repositories - Task 1: repo1/title #42 (Agent: agent-001) - Task 2: repo2/title #43 (Agent: agent-002) - Relationship: API_Contract - Reason: API definition and implementation dependency COORDINATION REQUIRED: Generate a concise coordination plan that addresses: 1. What specific coordination is needed between the agents 2. What order should tasks be completed in (if any) 3. What information/artifacts need to be shared 4. What potential conflicts to watch for 5. Success criteria for coordinated completion ` plan := reasoning.GenerateResponse(ctx, "phi3", prompt) ``` **Plan Output Example:** ``` COORDINATION PLAN: 1. SEQUENCE: - Task 1 (API definition) must be completed first - Task 2 (implementation) depends on finalized API contract 2. INFORMATION SHARING: - Agent-001 must share: API specification document, endpoint definitions - Agent-002 must share: Implementation plan, integration tests 3. COORDINATION POINTS: - Review API spec before implementation begins - Daily sync on implementation progress - Joint testing before completion 4. POTENTIAL CONFLICTS: - API spec changes during implementation - Performance requirements not captured in spec - Authentication/authorization approach 5. SUCCESS CRITERIA: - API spec reviewed and approved - Implementation matches spec - Integration tests pass - Documentation complete ``` ### 5. Session Progress Monitoring ``` Agents respond to coordination plan ↓ handleCoordinationResponse() ↓ Add message to session ↓ Update participant activity ↓ evaluateSessionProgress() ↓ ┌──────────────────────┐ │ Check conditions: │ │ - Message count │ │ - Session duration │ │ - Agreement keywords │ └──────┬───────────────┘ │ ┌──────▼──────┬──────────────┐ │ │ │ Consensus? Too long? Too many msgs? │ │ │ Resolved Escalate Escalate ``` ### 6. Session Resolution **Consensus Reached:** ```go // Detect agreement in recent messages agreementKeywords := []string{ "agree", "sounds good", "approved", "looks good", "confirmed" } if agreementCount >= len(participants)-1 { resolveSession(session, "Consensus reached among participants") } ``` **Session Resolved:** 1. Update session status to "resolved" 2. Record resolution reason 3. Generate SLURP event (if integrator available) 4. Broadcast resolution to participants 5. Clean up after timeout ### 7. Session Escalation **Escalation Triggers:** - Message count exceeds threshold (default: 10) - Session duration exceeds limit (default: 30 minutes) - Explicit escalation request from agent **Escalation Process:** ```go escalateSession(session, reason) ↓ Update status to "escalated" ↓ Generate SLURP event for human review ↓ Broadcast escalation notification ↓ Human intervention required ``` ## SLURP Integration ### Event Generation from Sessions When sessions are resolved or escalated, the MetaCoordinator generates SLURP events: ```go discussionContext := integration.HmmmDiscussionContext{ DiscussionID: session.SessionID, SessionID: session.SessionID, Participants: [agentIDs], StartTime: session.CreatedAt, EndTime: session.LastActivity, Messages: hmmmMessages, ConsensusReached: (outcome == "resolved"), ConsensusStrength: 0.9, // 0.3 for escalated, 0.5 for other OutcomeType: outcome, // "resolved" or "escalated" ProjectPath: projectPath, RelatedTasks: [taskIDs], Metadata: { "session_type": session.Type, "session_status": session.Status, "resolution": session.Resolution, "escalation_reason": session.EscalationReason, "message_count": len(session.Messages), "participant_count": len(session.Participants), }, } slurpIntegrator.ProcessHmmmDiscussion(ctx, discussionContext) ``` **SLURP Event Outcomes:** - **Resolved sessions**: High consensus (0.9), successful coordination - **Escalated sessions**: Low consensus (0.3), human intervention needed - **Other outcomes**: Medium consensus (0.5) ### Policy Learning SLURP uses coordination session data to learn: - Effective coordination patterns - Common dependency types - Escalation triggers - Agent collaboration efficiency - Task complexity indicators ## PubSub Message Types ### 1. dependency_detected Announces a detected dependency between tasks. ```json { "message_type": "dependency_detected", "dependency": { "task1": { "task_id": 42, "project_id": 1, "repository": "backend-api", "title": "Define user authentication API", "agent_id": "agent-001" }, "task2": { "task_id": 43, "project_id": 2, "repository": "frontend-app", "title": "Implement login page", "agent_id": "agent-002" }, "relationship": "API_Contract", "confidence": 0.8, "reason": "API definition and implementation dependency", "detected_at": "2025-09-30T10:00:00Z" }, "coordination_request": "Cross-repository dependency detected...", "agents_involved": ["agent-001", "agent-002"], "repositories": ["backend-api", "frontend-app"], "hop_count": 0, "max_hops": 3 } ``` ### 2. coordination_plan Broadcasts AI-generated coordination plan to participants. ```json { "message_type": "coordination_plan", "session_id": "dep_1_42_1727692800", "plan": "COORDINATION PLAN:\n1. SEQUENCE:\n...", "tasks_involved": [taskContext1, taskContext2], "participants": { "agent-001": { "agent_id": "agent-001", "repository": "backend-api" }, "agent-002": { "agent_id": "agent-002", "repository": "frontend-app" } }, "message": "Coordination plan generated for dependency: API_Contract" } ``` ### 3. coordination_response Agent response to coordination plan or session message. ```json { "message_type": "coordination_response", "session_id": "dep_1_42_1727692800", "agent_id": "agent-001", "response": "I agree with the proposed sequence. API spec will be ready by EOD.", "timestamp": "2025-09-30T10:05:00Z" } ``` ### 4. session_message General message within a coordination session. ```json { "message_type": "session_message", "session_id": "dep_1_42_1727692800", "from_agent": "agent-002", "content": "Can we schedule a quick sync to review the API spec?", "timestamp": "2025-09-30T10:10:00Z" } ``` ### 5. escalation Session escalated to human intervention. ```json { "message_type": "escalation", "session_id": "dep_1_42_1727692800", "escalation_reason": "Message limit exceeded - human intervention needed", "session_summary": "Session dep_1_42_1727692800 (dependency): 2 participants, 12 messages, duration 35m", "participants": { /* participant info */ }, "tasks_involved": [ /* task contexts */ ], "requires_human": true } ``` ### 6. resolution Session successfully resolved. ```json { "message_type": "resolution", "session_id": "dep_1_42_1727692800", "resolution": "Consensus reached among participants", "summary": "Session dep_1_42_1727692800 (dependency): 2 participants, 8 messages, duration 15m" } ``` ## Usage Examples ### Basic Setup ```go import ( "context" "chorus/pkg/coordination" "chorus/pubsub" ) // Create MetaCoordinator mc := coordination.NewMetaCoordinator(ctx, pubsubInstance) // Optionally attach SLURP integrator mc.SetSlurpIntegrator(slurpIntegrator) // MetaCoordinator automatically: // - Initializes DependencyDetector // - Sets up HMMM message handlers // - Starts session cleanup loop ``` ### Register Tasks for Dependency Detection ```go // Agent claims a task taskContext := &coordination.TaskContext{ TaskID: 42, ProjectID: 1, Repository: "backend-api", Title: "Define user authentication API", Description: "Create OpenAPI spec for user auth endpoints", Keywords: []string{"api", "authentication", "openapi"}, AgentID: "agent-001", ClaimedAt: time.Now(), } mc.dependencyDetector.RegisterTask(taskContext) ``` ### Add Custom Dependency Rule ```go // Add project-specific rule microserviceRule := coordination.DependencyRule{ Name: "Microservice_Interface", Description: "Microservice interface and consumer dependencies", Keywords: []string{"microservice", "interface", "consumer", "producer"}, Validator: func(task1, task2 *coordination.TaskContext) (bool, string) { t1 := strings.ToLower(task1.Title + " " + task1.Description) t2 := strings.ToLower(task2.Title + " " + task2.Description) hasProducer := strings.Contains(t1, "producer") || strings.Contains(t2, "producer") hasConsumer := strings.Contains(t1, "consumer") || strings.Contains(t2, "consumer") if hasProducer && hasConsumer { return true, "Microservice producer and consumer must coordinate" } return false, "" }, } mc.dependencyDetector.AddCustomRule(microserviceRule) ``` ### Query Active Sessions ```go // Get all active coordination sessions sessions := mc.GetActiveSessions() for sessionID, session := range sessions { fmt.Printf("Session %s:\n", sessionID) fmt.Printf(" Type: %s\n", session.Type) fmt.Printf(" Status: %s\n", session.Status) fmt.Printf(" Participants: %d\n", len(session.Participants)) fmt.Printf(" Messages: %d\n", len(session.Messages)) fmt.Printf(" Duration: %v\n", time.Since(session.CreatedAt)) } ``` ### Monitor Coordination Events ```go // Set custom HMMM message handler pubsub.SetHmmmMessageHandler(func(msg pubsub.Message, from peer.ID) { switch msg.Data["message_type"] { case "dependency_detected": fmt.Printf("🔗 Dependency detected: %v\n", msg.Data) case "coordination_plan": fmt.Printf("📋 Coordination plan: %v\n", msg.Data) case "escalation": fmt.Printf("🚨 Escalation: %v\n", msg.Data) case "resolution": fmt.Printf("✅ Resolution: %v\n", msg.Data) } }) ``` ## Configuration ### MetaCoordinator Configuration ```go mc := coordination.NewMetaCoordinator(ctx, ps) // Adjust session parameters mc.maxSessionDuration = 45 * time.Minute // Extend session timeout mc.maxParticipants = 10 // Support larger teams mc.escalationThreshold = 15 // More messages before escalation ``` ### DependencyDetector Configuration ```go dd := mc.dependencyDetector // Adjust coordination hop limit dd.coordinationHops = 5 // Allow deeper meta-discussion chains ``` ## Session Lifecycle Management ### Automatic Cleanup Sessions are automatically cleaned up by the session cleanup loop: ```go // Runs every 10 minutes func (mc *MetaCoordinator) cleanupInactiveSessions() { for sessionID, session := range mc.activeSessions { // Remove sessions older than 2 hours OR already resolved/escalated if time.Since(session.LastActivity) > 2*time.Hour || session.Status == "resolved" || session.Status == "escalated" { delete(mc.activeSessions, sessionID) } } } ``` **Cleanup Criteria:** - Session inactive for 2+ hours - Session status is "resolved" - Session status is "escalated" ### Manual Session Management ```go // Not exposed in current API, but could be added: // Force resolve session mc.resolveSession(session, "Manual resolution by admin") // Force escalate session mc.escalateSession(session, "Manual escalation requested") // Cancel/close session mc.closeSession(sessionID) ``` ## Performance Considerations ### Memory Usage - **TaskContext Storage**: ~500 bytes per task - **Active Sessions**: ~5KB per session (varies with message count) - **Dependency Rules**: ~1KB per rule **Typical Usage**: 100 tasks + 10 sessions = ~100KB ### CPU Usage - **Dependency Detection**: O(N²) where N = number of tasks per repository - **Rule Evaluation**: O(R) where R = number of rules - **Session Monitoring**: Periodic evaluation (every message received) **Optimization**: Dependency detection skips same-repository comparisons. ### Network Usage - **Dependency Announcements**: ~2KB per dependency - **Coordination Plans**: ~5KB per plan (includes full context) - **Session Messages**: ~1KB per message - **SLURP Events**: ~10KB per event (includes full session history) ## Best Practices ### 1. Rule Design **Good Rule:** ```go // Specific, actionable, clear success criteria { Name: "Database_Migration", Keywords: []string{"migration", "schema", "database"}, Validator: func(t1, t2 *TaskContext) (bool, string) { // Clear matching logic // Specific reason returned }, } ``` **Bad Rule:** ```go // Too broad, unclear coordination needed { Name: "Backend_Tasks", Keywords: []string{"backend"}, Validator: func(t1, t2 *TaskContext) (bool, string) { return strings.Contains(t1.Title, "backend") && strings.Contains(t2.Title, "backend"), "Both backend tasks" }, } ``` ### 2. Session Participation - **Respond promptly**: Keep sessions moving - **Be explicit**: Use clear agreement/disagreement language - **Stay focused**: Don't derail session with unrelated topics - **Escalate when stuck**: Don't let sessions drag on indefinitely ### 3. AI Plan Quality AI plans are most effective when: - Task descriptions are detailed - Dependencies are clear - Agent capabilities are well-defined - Historical context is available ### 4. SLURP Integration For best SLURP learning: - Enable SLURP integrator at startup - Ensure all sessions generate events (resolved or escalated) - Provide rich task metadata - Include project context in task descriptions ## Troubleshooting ### Dependencies Not Detected **Symptoms**: Related tasks not triggering coordination. **Checks:** 1. Verify tasks registered with detector: `dd.GetKnownTasks()` 2. Check rule keywords match task content 3. Test validator logic with task pairs 4. Verify tasks are from different repositories 5. Check PubSub connection for announcements ### Sessions Not Escalating **Symptoms**: Long-running sessions without escalation. **Checks:** 1. Verify escalation threshold: `mc.escalationThreshold` 2. Check session duration limit: `mc.maxSessionDuration` 3. Verify message count in session 4. Check for agreement keywords in messages 5. Test escalation logic manually ### AI Plans Not Generated **Symptoms**: Sessions created but no coordination plan. **Checks:** 1. Verify reasoning engine available: `reasoning.GenerateResponse()` 2. Check AI model configuration 3. Verify network connectivity to AI provider 4. Check reasoning engine error logs 5. Test with simpler dependency ### SLURP Events Not Generated **Symptoms**: Sessions complete but no SLURP events. **Checks:** 1. Verify SLURP integrator attached: `mc.SetSlurpIntegrator()` 2. Check SLURP integrator initialization 3. Verify session outcome triggers event generation 4. Check SLURP integrator error logs 5. Test event generation manually ## Future Enhancements ### Planned Features 1. **Machine Learning Rules**: Learn dependency patterns from historical data 2. **Automated Testing**: Generate integration tests for coordinated tasks 3. **Visualization**: Web UI for monitoring active sessions 4. **Advanced Metrics**: Track coordination efficiency and success rates 5. **Multi-Repo CI/CD**: Coordinate deployments across dependent services 6. **Conflict Resolution**: AI-powered conflict resolution suggestions 7. **Predictive Coordination**: Predict dependencies before tasks are claimed ## See Also - [coordinator/](coordinator.md) - Task coordinator integration - [pubsub/](../pubsub.md) - PubSub messaging for coordination - [pkg/integration/](integration.md) - SLURP integration - [pkg/hmmm/](hmmm.md) - HMMM meta-discussion system - [reasoning/](../reasoning.md) - AI reasoning engine for planning - [internal/logging/](../internal/logging.md) - Hypercore logging