HAP Analysis: Archive docs and create implementation action plan
- Archive all existing markdown documentation files - Create comprehensive HAP_ACTION_PLAN.md with: * Analysis of current BZZZ implementation vs HAP vision * 4-phase implementation strategy * Structural reorganization approach (multi-binary) * HAP interface implementation roadmap - Preserve existing functionality while adding human agent portal - Focus on incremental migration over rewrite 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
anthonyrawlins
523
archive/SLURP_GO_ARCHITECTURE_DESIGN.md
Normal file
523
archive/SLURP_GO_ARCHITECTURE_DESIGN.md
Normal file
@@ -0,0 +1,523 @@
|
||||
# SLURP Contextual Intelligence System - Go Architecture Design
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides the complete architectural design for implementing the SLURP (Storage, Logic, Understanding, Retrieval, Processing) contextual intelligence system in Go, integrated with the existing BZZZ infrastructure.
|
||||
|
||||
## Current BZZZ Architecture Analysis
|
||||
|
||||
### Existing Package Structure
|
||||
```
|
||||
pkg/
|
||||
├── config/ # Configuration management
|
||||
├── crypto/ # Encryption, Shamir's Secret Sharing
|
||||
├── dht/ # Distributed Hash Table (mock + real)
|
||||
├── election/ # Leader election algorithms
|
||||
├── types/ # Common types and interfaces
|
||||
├── ucxl/ # UCXL address parsing and handling
|
||||
└── ...
|
||||
```
|
||||
|
||||
### Key Integration Points
|
||||
- **DHT Integration**: `pkg/dht/` for context distribution
|
||||
- **Crypto Integration**: `pkg/crypto/` for role-based encryption
|
||||
- **Election Integration**: `pkg/election/` for Leader duties
|
||||
- **UCXL Integration**: `pkg/ucxl/` for address parsing
|
||||
- **Config Integration**: `pkg/config/` for system configuration
|
||||
|
||||
## Go Package Design
|
||||
|
||||
### Package Structure
|
||||
```
|
||||
pkg/slurp/
|
||||
├── context/ # Core context types and interfaces
|
||||
├── intelligence/ # Context analysis and generation
|
||||
├── storage/ # Context persistence and retrieval
|
||||
├── distribution/ # Context network distribution
|
||||
├── temporal/ # Decision-hop temporal analysis
|
||||
├── alignment/ # Project goal alignment
|
||||
├── roles/ # Role-based access control
|
||||
└── leader/ # Leader-specific context duties
|
||||
```
|
||||
|
||||
## Core Types and Interfaces
|
||||
|
||||
### 1. Context Types (`pkg/slurp/context/types.go`)
|
||||
|
||||
```go
|
||||
package context
|
||||
|
||||
import (
|
||||
"time"
|
||||
"github.com/your-org/bzzz/pkg/ucxl"
|
||||
"github.com/your-org/bzzz/pkg/types"
|
||||
)
|
||||
|
||||
// ContextNode represents a hierarchical context node
|
||||
type ContextNode struct {
|
||||
Path string `json:"path"`
|
||||
UCXLAddress ucxl.Address `json:"ucxl_address"`
|
||||
Summary string `json:"summary"`
|
||||
Purpose string `json:"purpose"`
|
||||
Technologies []string `json:"technologies"`
|
||||
Tags []string `json:"tags"`
|
||||
Insights []string `json:"insights"`
|
||||
|
||||
// Hierarchy control
|
||||
OverridesParent bool `json:"overrides_parent"`
|
||||
ContextSpecificity int `json:"context_specificity"`
|
||||
AppliesToChildren bool `json:"applies_to_children"`
|
||||
|
||||
// Metadata
|
||||
GeneratedAt time.Time `json:"generated_at"`
|
||||
RAGConfidence float64 `json:"rag_confidence"`
|
||||
}
|
||||
|
||||
// RoleAccessLevel defines encryption levels for different roles
|
||||
type RoleAccessLevel int
|
||||
|
||||
const (
|
||||
AccessPublic RoleAccessLevel = iota
|
||||
AccessLow
|
||||
AccessMedium
|
||||
AccessHigh
|
||||
AccessCritical
|
||||
)
|
||||
|
||||
// EncryptedContext represents role-encrypted context data
|
||||
type EncryptedContext struct {
|
||||
UCXLAddress ucxl.Address `json:"ucxl_address"`
|
||||
Role string `json:"role"`
|
||||
AccessLevel RoleAccessLevel `json:"access_level"`
|
||||
EncryptedData []byte `json:"encrypted_data"`
|
||||
KeyFingerprint string `json:"key_fingerprint"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// ResolvedContext is the final resolved context for consumption
|
||||
type ResolvedContext struct {
|
||||
UCXLAddress ucxl.Address `json:"ucxl_address"`
|
||||
Summary string `json:"summary"`
|
||||
Purpose string `json:"purpose"`
|
||||
Technologies []string `json:"technologies"`
|
||||
Tags []string `json:"tags"`
|
||||
Insights []string `json:"insights"`
|
||||
|
||||
// Resolution metadata
|
||||
ContextSourcePath string `json:"context_source_path"`
|
||||
InheritanceChain []string `json:"inheritance_chain"`
|
||||
ResolutionConfidence float64 `json:"resolution_confidence"`
|
||||
BoundedDepth int `json:"bounded_depth"`
|
||||
GlobalContextsApplied bool `json:"global_contexts_applied"`
|
||||
ResolvedAt time.Time `json:"resolved_at"`
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Context Resolver Interface (`pkg/slurp/context/resolver.go`)
|
||||
|
||||
```go
|
||||
package context
|
||||
|
||||
// ContextResolver defines the interface for hierarchical context resolution
|
||||
type ContextResolver interface {
|
||||
// Resolve context for a UCXL address with bounded hierarchy traversal
|
||||
Resolve(address ucxl.Address, role string, maxDepth int) (*ResolvedContext, error)
|
||||
|
||||
// Add global context that applies to all addresses
|
||||
AddGlobalContext(ctx *ContextNode) error
|
||||
|
||||
// Set maximum hierarchy depth for bounded traversal
|
||||
SetHierarchyDepthLimit(maxDepth int)
|
||||
|
||||
// Get resolution statistics
|
||||
GetStatistics() *ResolutionStatistics
|
||||
}
|
||||
|
||||
type ResolutionStatistics struct {
|
||||
ContextNodes int `json:"context_nodes"`
|
||||
GlobalContexts int `json:"global_contexts"`
|
||||
MaxHierarchyDepth int `json:"max_hierarchy_depth"`
|
||||
CachedResolutions int `json:"cached_resolutions"`
|
||||
TotalResolutions int `json:"total_resolutions"`
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Temporal Decision Analysis (`pkg/slurp/temporal/types.go`)
|
||||
|
||||
```go
|
||||
package temporal
|
||||
|
||||
import (
|
||||
"time"
|
||||
"github.com/your-org/bzzz/pkg/ucxl"
|
||||
)
|
||||
|
||||
// ChangeReason represents why a context changed
|
||||
type ChangeReason string
|
||||
|
||||
const (
|
||||
InitialCreation ChangeReason = "initial_creation"
|
||||
CodeChange ChangeReason = "code_change"
|
||||
DesignDecision ChangeReason = "design_decision"
|
||||
Refactoring ChangeReason = "refactoring"
|
||||
ArchitectureChange ChangeReason = "architecture_change"
|
||||
RequirementsChange ChangeReason = "requirements_change"
|
||||
LearningEvolution ChangeReason = "learning_evolution"
|
||||
RAGEnhancement ChangeReason = "rag_enhancement"
|
||||
TeamInput ChangeReason = "team_input"
|
||||
)
|
||||
|
||||
// DecisionMetadata captures information about a decision
|
||||
type DecisionMetadata struct {
|
||||
DecisionMaker string `json:"decision_maker"`
|
||||
DecisionID string `json:"decision_id"` // Git commit, ticket ID, etc.
|
||||
DecisionRationale string `json:"decision_rationale"`
|
||||
ImpactScope string `json:"impact_scope"` // local, module, project, system
|
||||
ConfidenceLevel float64 `json:"confidence_level"`
|
||||
ExternalReferences []string `json:"external_references"`
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
}
|
||||
|
||||
// TemporalContextNode represents context at a specific decision point
|
||||
type TemporalContextNode struct {
|
||||
UCXLAddress ucxl.Address `json:"ucxl_address"`
|
||||
Version int `json:"version"`
|
||||
|
||||
// Core context (embedded)
|
||||
Context *ContextNode `json:"context"`
|
||||
|
||||
// Temporal metadata
|
||||
ChangeReason ChangeReason `json:"change_reason"`
|
||||
ParentVersion *int `json:"parent_version,omitempty"`
|
||||
DecisionMeta *DecisionMetadata `json:"decision_metadata"`
|
||||
|
||||
// Evolution tracking
|
||||
ContextHash string `json:"context_hash"`
|
||||
ConfidenceScore float64 `json:"confidence_score"`
|
||||
StalenessScore float64 `json:"staleness_score"`
|
||||
|
||||
// Decision influence graph
|
||||
Influences []ucxl.Address `json:"influences"` // Addresses this decision affects
|
||||
InfluencedBy []ucxl.Address `json:"influenced_by"` // Addresses that affect this
|
||||
}
|
||||
|
||||
// DecisionPath represents a path between two decisions
|
||||
type DecisionPath struct {
|
||||
FromAddress ucxl.Address `json:"from_address"`
|
||||
ToAddress ucxl.Address `json:"to_address"`
|
||||
Path []*TemporalContextNode `json:"path"`
|
||||
HopDistance int `json:"hop_distance"`
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Intelligence Engine Interface (`pkg/slurp/intelligence/engine.go`)
|
||||
|
||||
```go
|
||||
package intelligence
|
||||
|
||||
import (
|
||||
"context"
|
||||
"github.com/your-org/bzzz/pkg/ucxl"
|
||||
slurpContext "github.com/your-org/bzzz/pkg/slurp/context"
|
||||
)
|
||||
|
||||
// IntelligenceEngine generates contextual understanding
|
||||
type IntelligenceEngine interface {
|
||||
// Analyze a filesystem path and generate context
|
||||
AnalyzeFile(ctx context.Context, filePath string, role string) (*slurpContext.ContextNode, error)
|
||||
|
||||
// Analyze directory structure for hierarchical patterns
|
||||
AnalyzeDirectory(ctx context.Context, dirPath string) ([]*slurpContext.ContextNode, error)
|
||||
|
||||
// Generate role-specific insights
|
||||
GenerateRoleInsights(ctx context.Context, baseContext *slurpContext.ContextNode, role string) ([]string, error)
|
||||
|
||||
// Assess project goal alignment
|
||||
AssessGoalAlignment(ctx context.Context, node *slurpContext.ContextNode) (float64, error)
|
||||
}
|
||||
|
||||
// ProjectGoal represents a high-level project objective
|
||||
type ProjectGoal struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
Keywords []string `json:"keywords"`
|
||||
Priority int `json:"priority"`
|
||||
Phase string `json:"phase"`
|
||||
}
|
||||
|
||||
// RoleProfile defines what context a role needs
|
||||
type RoleProfile struct {
|
||||
Role string `json:"role"`
|
||||
AccessLevel slurpContext.RoleAccessLevel `json:"access_level"`
|
||||
RelevantTags []string `json:"relevant_tags"`
|
||||
ContextScope []string `json:"context_scope"` // frontend, backend, infrastructure, etc.
|
||||
InsightTypes []string `json:"insight_types"`
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Leader Integration (`pkg/slurp/leader/manager.go`)
|
||||
|
||||
```go
|
||||
package leader
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"github.com/your-org/bzzz/pkg/election"
|
||||
"github.com/your-org/bzzz/pkg/dht"
|
||||
"github.com/your-org/bzzz/pkg/slurp/intelligence"
|
||||
slurpContext "github.com/your-org/bzzz/pkg/slurp/context"
|
||||
)
|
||||
|
||||
// ContextManager handles leader-only context generation duties
|
||||
type ContextManager struct {
|
||||
mu sync.RWMutex
|
||||
isLeader bool
|
||||
election election.Election
|
||||
dht dht.DHT
|
||||
intelligence intelligence.IntelligenceEngine
|
||||
contextResolver slurpContext.ContextResolver
|
||||
|
||||
// Context generation state
|
||||
generationQueue chan *ContextGenerationRequest
|
||||
activeJobs map[string]*ContextGenerationJob
|
||||
}
|
||||
|
||||
type ContextGenerationRequest struct {
|
||||
UCXLAddress ucxl.Address `json:"ucxl_address"`
|
||||
FilePath string `json:"file_path"`
|
||||
Priority int `json:"priority"`
|
||||
RequestedBy string `json:"requested_by"`
|
||||
Role string `json:"role"`
|
||||
}
|
||||
|
||||
type ContextGenerationJob struct {
|
||||
Request *ContextGenerationRequest
|
||||
Status JobStatus
|
||||
StartedAt time.Time
|
||||
CompletedAt *time.Time
|
||||
Result *slurpContext.ContextNode
|
||||
Error error
|
||||
}
|
||||
|
||||
type JobStatus string
|
||||
|
||||
const (
|
||||
JobPending JobStatus = "pending"
|
||||
JobRunning JobStatus = "running"
|
||||
JobCompleted JobStatus = "completed"
|
||||
JobFailed JobStatus = "failed"
|
||||
)
|
||||
|
||||
// NewContextManager creates a new leader context manager
|
||||
func NewContextManager(
|
||||
election election.Election,
|
||||
dht dht.DHT,
|
||||
intelligence intelligence.IntelligenceEngine,
|
||||
resolver slurpContext.ContextResolver,
|
||||
) *ContextManager {
|
||||
cm := &ContextManager{
|
||||
election: election,
|
||||
dht: dht,
|
||||
intelligence: intelligence,
|
||||
contextResolver: resolver,
|
||||
generationQueue: make(chan *ContextGenerationRequest, 1000),
|
||||
activeJobs: make(map[string]*ContextGenerationJob),
|
||||
}
|
||||
|
||||
// Listen for leadership changes
|
||||
go cm.watchLeadershipChanges()
|
||||
|
||||
// Process context generation requests (only when leader)
|
||||
go cm.processContextGeneration()
|
||||
|
||||
return cm
|
||||
}
|
||||
|
||||
// RequestContextGeneration queues a context generation request
|
||||
func (cm *ContextManager) RequestContextGeneration(req *ContextGenerationRequest) error {
|
||||
select {
|
||||
case cm.generationQueue <- req:
|
||||
return nil
|
||||
default:
|
||||
return errors.New("context generation queue is full")
|
||||
}
|
||||
}
|
||||
|
||||
// IsLeader returns whether this node is the current leader
|
||||
func (cm *ContextManager) IsLeader() bool {
|
||||
cm.mu.RLock()
|
||||
defer cm.mu.RUnlock()
|
||||
return cm.isLeader
|
||||
}
|
||||
```
|
||||
|
||||
## Integration with Existing BZZZ Systems
|
||||
|
||||
### 1. DHT Integration (`pkg/slurp/distribution/dht.go`)
|
||||
|
||||
```go
|
||||
package distribution
|
||||
|
||||
import (
|
||||
"github.com/your-org/bzzz/pkg/dht"
|
||||
"github.com/your-org/bzzz/pkg/crypto"
|
||||
slurpContext "github.com/your-org/bzzz/pkg/slurp/context"
|
||||
)
|
||||
|
||||
// ContextDistributor handles context distribution through DHT
|
||||
type ContextDistributor struct {
|
||||
dht dht.DHT
|
||||
crypto crypto.Crypto
|
||||
}
|
||||
|
||||
// DistributeContext encrypts and stores context in DHT for role-based access
|
||||
func (cd *ContextDistributor) DistributeContext(
|
||||
ctx *slurpContext.ContextNode,
|
||||
roles []string,
|
||||
) error {
|
||||
// For each role, encrypt the context with role-specific keys
|
||||
for _, role := range roles {
|
||||
encryptedCtx, err := cd.encryptForRole(ctx, role)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to encrypt context for role %s: %w", role, err)
|
||||
}
|
||||
|
||||
// Store in DHT with role-specific key
|
||||
key := cd.generateContextKey(ctx.UCXLAddress, role)
|
||||
if err := cd.dht.Put(key, encryptedCtx); err != nil {
|
||||
return fmt.Errorf("failed to store context in DHT: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// RetrieveContext gets context from DHT and decrypts for the requesting role
|
||||
func (cd *ContextDistributor) RetrieveContext(
|
||||
address ucxl.Address,
|
||||
role string,
|
||||
) (*slurpContext.ResolvedContext, error) {
|
||||
key := cd.generateContextKey(address, role)
|
||||
|
||||
encryptedData, err := cd.dht.Get(key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to retrieve context from DHT: %w", err)
|
||||
}
|
||||
|
||||
return cd.decryptForRole(encryptedData, role)
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Configuration Integration (`pkg/slurp/config/config.go`)
|
||||
|
||||
```go
|
||||
package config
|
||||
|
||||
import "github.com/your-org/bzzz/pkg/config"
|
||||
|
||||
// SLURPConfig extends BZZZ config with SLURP-specific settings
|
||||
type SLURPConfig struct {
|
||||
// Context generation settings
|
||||
MaxHierarchyDepth int `yaml:"max_hierarchy_depth" json:"max_hierarchy_depth"`
|
||||
ContextCacheTTL int `yaml:"context_cache_ttl" json:"context_cache_ttl"`
|
||||
GenerationConcurrency int `yaml:"generation_concurrency" json:"generation_concurrency"`
|
||||
|
||||
// Role-based access
|
||||
RoleProfiles map[string]*RoleProfile `yaml:"role_profiles" json:"role_profiles"`
|
||||
DefaultAccessLevel string `yaml:"default_access_level" json:"default_access_level"`
|
||||
|
||||
// Intelligence engine settings
|
||||
RAGEndpoint string `yaml:"rag_endpoint" json:"rag_endpoint"`
|
||||
RAGTimeout int `yaml:"rag_timeout" json:"rag_timeout"`
|
||||
ConfidenceThreshold float64 `yaml:"confidence_threshold" json:"confidence_threshold"`
|
||||
|
||||
// Project goals
|
||||
ProjectGoals []*ProjectGoal `yaml:"project_goals" json:"project_goals"`
|
||||
}
|
||||
|
||||
// LoadSLURPConfig extends the main BZZZ config loading
|
||||
func LoadSLURPConfig(configPath string) (*config.Config, error) {
|
||||
// Load base BZZZ config
|
||||
bzzzConfig, err := config.Load(configPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Load SLURP-specific extensions
|
||||
slurpConfig := &SLURPConfig{}
|
||||
if err := config.LoadSection("slurp", slurpConfig); err != nil {
|
||||
// Use defaults if SLURP config not found
|
||||
slurpConfig = DefaultSLURPConfig()
|
||||
}
|
||||
|
||||
// Merge into main config
|
||||
bzzzConfig.SLURP = slurpConfig
|
||||
return bzzzConfig, nil
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Foundation (Week 1-2)
|
||||
1. **Create base package structure** in `pkg/slurp/`
|
||||
2. **Define core interfaces and types** (`context`, `temporal`)
|
||||
3. **Integrate with existing election system** for leader duties
|
||||
4. **Basic context resolver implementation** with bounded traversal
|
||||
|
||||
### Phase 2: Encryption & Distribution (Week 3-4)
|
||||
1. **Extend crypto package** for role-based encryption
|
||||
2. **Implement DHT context distribution**
|
||||
3. **Role-based access control** integration
|
||||
4. **Context caching and retrieval**
|
||||
|
||||
### Phase 3: Intelligence Engine (Week 5-7)
|
||||
1. **File analysis and context generation**
|
||||
2. **Decision temporal graph implementation**
|
||||
3. **Project goal alignment**
|
||||
4. **RAG integration** for enhanced context
|
||||
|
||||
### Phase 4: Integration & Testing (Week 8)
|
||||
1. **End-to-end integration testing**
|
||||
2. **Performance optimization**
|
||||
3. **Documentation and examples**
|
||||
4. **Leader failover testing**
|
||||
|
||||
## Key Go Patterns Used
|
||||
|
||||
### 1. Interface-Driven Design
|
||||
All major components define clear interfaces, allowing for testing and future extensibility.
|
||||
|
||||
### 2. Context Propagation
|
||||
Using Go's `context` package for cancellation and timeouts throughout the system.
|
||||
|
||||
### 3. Concurrent Processing
|
||||
Goroutines and channels for context generation queue processing and distributed operations.
|
||||
|
||||
### 4. Error Handling
|
||||
Proper error wrapping and handling following Go best practices.
|
||||
|
||||
### 5. Configuration
|
||||
Extends existing BZZZ configuration patterns seamlessly.
|
||||
|
||||
## Migration from Python Prototypes
|
||||
|
||||
The Python prototypes provide the algorithmic foundation:
|
||||
|
||||
1. **Bounded hierarchy walking** → Go recursive traversal with depth limits
|
||||
2. **CSS-like context inheritance** → Go struct composition and merging
|
||||
3. **Decision-hop analysis** → Go graph algorithms and BFS traversal
|
||||
4. **Role-based encryption** → Integration with existing Go crypto package
|
||||
5. **Temporal versioning** → Go time handling and version management
|
||||
|
||||
## Next Steps After Restart
|
||||
|
||||
1. **Run the systems-engineer agent** to create the Go package structure
|
||||
2. **Implement core interfaces** starting with `pkg/slurp/context/`
|
||||
3. **Integrate with existing BZZZ systems** step by step
|
||||
4. **Test each component** as it's implemented
|
||||
5. **Build up to full Leader-coordinated context generation**
|
||||
|
||||
This design ensures the SLURP system feels like a native part of BZZZ while providing the sophisticated contextual intelligence capabilities we designed.
|
||||
Reference in New Issue
Block a user