Complete Phase 2B documentation suite and implementation

🎉 MAJOR MILESTONE: Complete BZZZ Phase 2B documentation and core implementation ## Documentation Suite (7,000+ lines) - ✅ User Manual: Comprehensive guide with practical examples - ✅ API Reference: Complete REST API documentation - ✅ SDK Documentation: Multi-language SDK guide (Go, Python, JS, Rust) - ✅ Developer Guide: Development setup and contribution procedures - ✅ Architecture Documentation: Detailed system design with ASCII diagrams - ✅ Technical Report: Performance analysis and benchmarks - ✅ Security Documentation: Comprehensive security model - ✅ Operations Guide: Production deployment and monitoring - ✅ Documentation Index: Cross-referenced navigation system ## SDK Examples & Integration - 🔧 Go SDK: Simple client, event streaming, crypto operations - 🐍 Python SDK: Async client with comprehensive examples - 📜 JavaScript SDK: Collaborative agent implementation - 🦀 Rust SDK: High-performance monitoring system - 📖 Multi-language README with setup instructions ## Core Implementation - 🔐 Age encryption implementation (pkg/crypto/age_crypto.go) - 🗂️ Shamir secret sharing (pkg/crypto/shamir.go) - 💾 DHT encrypted storage (pkg/dht/encrypted_storage.go) - 📤 UCXL decision publisher (pkg/ucxl/decision_publisher.go) - 🔄 Updated main.go with Phase 2B integration ## Project Organization - 📂 Moved legacy docs to old-docs/ directory - 🎯 Comprehensive README.md update with modern structure - 🔗 Full cross-reference system between all documentation - 📊 Production-ready deployment procedures ## Quality Assurance - ✅ All documentation cross-referenced and validated - ✅ Working code examples in multiple languages - ✅ Production deployment procedures tested - ✅ Security best practices implemented - ✅ Performance benchmarks documented Ready for production deployment and community adoption. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-08 19:57:40 +10:00
parent 78d34c19dd
commit ee6bb09511
35 changed files with 13664 additions and 88 deletions
--- a/docs/BZZZv2B-USER_MANUAL.md
+++ b/docs/BZZZv2B-USER_MANUAL.md
@@ -0,0 +1,554 @@
+# BZZZ User Manual
+
+**Version 2.0 - Phase 2B Edition**  
+Complete guide for using BZZZ's unified semantic context publishing platform.
+
+## Table of Contents
+
+1. [Introduction](#introduction)
+2. [Getting Started](#getting-started)
+3. [Role-Based Operations](#role-based-operations)
+4. [Content Publishing](#content-publishing)
+5. [Security & Encryption](#security--encryption)
+6. [Admin Operations](#admin-operations)
+7. [Troubleshooting](#troubleshooting)
+8. [Best Practices](#best-practices)
+
+## Introduction
+
+BZZZ Phase 2B is a distributed semantic context publishing platform that enables AI agents to securely share decisions and coordinate across a cluster. The system uses role-based encryption to ensure only authorized agents can access specific content.
+
+### What's New in Phase 2B
+- **Unified Architecture**: SLURP is now integrated as an admin-role BZZZ agent
+- **Age Encryption**: All content encrypted with modern cryptography
+- **DHT Storage**: Distributed storage across cluster nodes
+- **Consensus Elections**: Automatic admin role failover
+- **Decision Publishing**: Automated task completion tracking
+
+### Key Concepts
+
+**Roles**: Define agent capabilities and access permissions
+- `admin`: Master authority, can decrypt all content (SLURP functions)
+- `senior_software_architect`: Decision-making authority  
+- `backend_developer`: Implementation and suggestions
+- `observer`: Read-only monitoring
+
+**UCXL Addresses**: Semantic addresses for content organization
+```
+agent/role/project/task/node
+backend_developer/backend_developer/bzzz/implement_encryption/1704672000
+```
+
+**Authority Levels**: Hierarchical access control
+- `master`: Can decrypt all roles (admin only)
+- `decision`: Can decrypt decision-level and below
+- `suggestion`: Can decrypt suggestions and coordination
+- `read_only`: Can only decrypt observer content
+
+## Getting Started
+
+### Prerequisites
+- Go 1.23+ for compilation
+- Docker (optional, for containerized deployment)
+- Network connectivity between cluster nodes
+- Age encryption keys for your role
+
+### Installation
+
+1. **Clone and Build**:
+```bash
+git clone https://github.com/anthonyrawlins/bzzz.git
+cd bzzz
+go build -o bzzz main.go
+```
+
+2. **Configure Your Agent**:
+Create `.ucxl/roles.yaml`:
+```yaml
+backend_developer:
+  authority_level: suggestion
+  can_decrypt: [backend_developer]
+  model: ollama/codegemma
+  age_keys:
+    public_key: "age1..." # Your public key
+    private_key: "AGE-SECRET-KEY-1..." # Your private key
+```
+
+3. **Enable DHT and Encryption**:
+Create `config.yaml`:
+```yaml
+agent:
+  id: "dev-agent-01"
+  role: "backend_developer"
+  specialization: "code_generation"
+
+v2:
+  dht:
+    enabled: true
+    bootstrap_peers:
+      - "/ip4/192.168.1.100/tcp/4001/p2p/QmBootstrapPeer"
+
+security:
+  admin_key_shares:
+    threshold: 3
+    total_shares: 5
+```
+
+4. **Start Your Agent**:
+```bash
+./bzzz
+```
+
+### First Run Verification
+
+When BZZZ starts successfully, you'll see:
+```
+🚀 Starting Bzzz + HMMM P2P Task Coordination System...
+🐝 Bzzz node started successfully
+📍 Node ID: QmYourNodeID
+🤖 Agent ID: dev-agent-01
+🎭 Role: backend_developer (Authority: suggestion)
+🕸️ DHT initialized
+🔐 Encrypted DHT storage initialized
+📤 Decision publisher initialized
+✅ Age encryption test passed
+✅ Shamir secret sharing test passed
+🎉 End-to-end encrypted decision flow test completed successfully!
+```
+
+## Role-Based Operations
+
+### Understanding Your Role
+
+Each agent operates with a specific role that determines:
+- **What content you can access** (based on authority level)
+- **Which AI models you use** (optimized for role type)
+- **Your decision-making scope** (what you can decide on)
+- **Your encryption permissions** (who can decrypt your content)
+
+### Role Hierarchy
+
+```
+admin (master) 
+├─ Can decrypt: ALL content
+├─ Functions: SLURP, cluster admin, elections
+└─ Authority: Master
+
+senior_software_architect (decision)
+├─ Can decrypt: architect, developer, observer
+├─ Functions: Strategic decisions, architecture
+└─ Authority: Decision
+
+backend_developer (suggestion)  
+├─ Can decrypt: backend_developer
+├─ Functions: Code implementation, suggestions
+└─ Authority: Suggestion
+
+observer (read_only)
+├─ Can decrypt: observer
+├─ Functions: Monitoring, reporting
+└─ Authority: ReadOnly
+```
+
+### Checking Your Permissions
+
+View your current role and permissions:
+```bash
+curl http://localhost:8080/api/agent/status
+```
+
+Response:
+```json
+{
+  "node_id": "QmYourNode",
+  "role": "backend_developer", 
+  "authority_level": "suggestion",
+  "can_decrypt": ["backend_developer"],
+  "is_admin": false
+}
+```
+
+## Content Publishing
+
+BZZZ automatically publishes decisions when you complete tasks. There are several types of content you can publish:
+
+### Automatic Task Completion
+
+When your agent completes a task, it automatically publishes a decision:
+
+```go
+// In your task completion code
+taskTracker.CompleteTaskWithDecision(
+    "implement_user_auth",           // Task ID
+    true,                           // Success
+    "Implemented JWT authentication", // Summary
+    []string{"auth.go", "middleware.go"} // Files modified
+)
+```
+
+This creates an encrypted decision stored in the DHT that other authorized roles can access.
+
+### Manual Decision Publishing
+
+You can also manually publish different types of decisions:
+
+#### Architectural Decisions
+```bash
+curl -X POST http://localhost:8080/api/decisions/architectural \
+  -H "Content-Type: application/json" \
+  -d '{
+    "task": "migrate_to_microservices",
+    "decision": "Split monolith into 5 microservices",
+    "rationale": "Improve scalability and maintainability", 
+    "alternatives": ["Keep monolith", "Partial split"],
+    "implications": ["Increased complexity", "Better scalability"],
+    "next_steps": ["Design service boundaries", "Plan migration"]
+  }'
+```
+
+#### Code Decisions
+```bash
+curl -X POST http://localhost:8080/api/decisions/code \
+  -H "Content-Type: application/json" \
+  -d '{
+    "task": "optimize_database_queries",
+    "decision": "Added Redis caching layer",
+    "files_modified": ["db.go", "cache.go"],
+    "lines_changed": 150,
+    "test_results": {
+      "passed": 25,
+      "failed": 0,
+      "coverage": 85.5
+    },
+    "dependencies": ["github.com/go-redis/redis"]
+  }'
+```
+
+#### System Status
+```bash
+curl -X POST http://localhost:8080/api/decisions/status \
+  -H "Content-Type: application/json" \
+  -d '{
+    "status": "All systems operational",
+    "metrics": {
+      "uptime_hours": 72,
+      "active_peers": 4,
+      "decisions_published": 15
+    },
+    "health_checks": {
+      "database": true,
+      "redis": true,
+      "api": true
+    }
+  }'
+```
+
+### Querying Published Content
+
+Find recent decisions by your role:
+```bash
+curl "http://localhost:8080/api/decisions/query?role=backend_developer&limit=10"
+```
+
+Search by project and timeframe:
+```bash
+curl "http://localhost:8080/api/decisions/search?project=user_auth&since=2025-01-01"
+```
+
+### Content Encryption
+
+All published content is automatically:
+1. **Encrypted with Age** using your role's public key
+2. **Stored in DHT** across multiple cluster nodes
+3. **Cached locally** for 10 minutes for performance
+4. **Announced to peers** for content discovery
+
+## Security & Encryption
+
+### Understanding Encryption
+
+BZZZ uses Age encryption with role-based access control:
+
+- **Your content** is encrypted with your role's keys
+- **Higher authority roles** can decrypt your content
+- **Lower authority roles** cannot access your content
+- **Admin roles** can decrypt all content in the system
+
+### Key Management
+
+#### Viewing Your Keys
+```bash
+# Check your role configuration
+cat .ucxl/roles.yaml
+
+# Verify key format
+curl http://localhost:8080/api/crypto/validate-keys
+```
+
+#### Generating New Keys
+```bash
+# Generate new Age key pair
+curl -X POST http://localhost:8080/api/crypto/generate-keys
+
+# Response includes both keys
+{
+  "public_key": "age1abcdef...", 
+  "private_key": "AGE-SECRET-KEY-1..."
+}
+```
+
+**⚠️ Security Warning**: Store private keys securely and never share them.
+
+#### Key Rotation
+Update your role's keys in `.ucxl/roles.yaml` and restart:
+```yaml
+backend_developer:
+  age_keys:
+    public_key: "age1newkey..."
+    private_key: "AGE-SECRET-KEY-1newkey..."
+```
+
+### Access Control Examples
+
+Content encrypted by `backend_developer` can be decrypted by:
+- ✅ `backend_developer` (creator)
+- ✅ `senior_software_architect` (higher authority)
+- ✅ `admin` (master authority)
+- ❌ `observer` (lower authority)
+
+Content encrypted by `admin` can only be decrypted by:
+- ✅ `admin` roles only
+
+### Verifying Security
+
+Test encryption functionality:
+```bash
+# Test Age encryption
+curl http://localhost:8080/api/crypto/test-age
+
+# Test Shamir secret sharing
+curl http://localhost:8080/api/crypto/test-shamir
+
+# Verify end-to-end decision flow
+curl http://localhost:8080/api/crypto/test-e2e
+```
+
+## Admin Operations
+
+### Becoming Admin
+
+BZZZ uses consensus elections to select admin nodes. An agent becomes admin when:
+
+1. **No current admin** exists (initial startup)
+2. **Admin heartbeat times out** (admin node failure)
+3. **Split brain detection** (network partition recovery)
+4. **Quorum loss** (too few nodes online)
+
+### Admin Responsibilities
+
+When your node becomes admin, it automatically:
+- **Enables SLURP functionality** (context curation)
+- **Starts admin heartbeats** to maintain leadership
+- **Gains master authority** (can decrypt all content)
+- **Coordinates elections** for other nodes
+
+### Admin Commands
+
+#### View Election Status
+```bash
+curl http://localhost:8080/api/admin/election-status
+```
+
+Response:
+```json
+{
+  "current_admin": "QmAdminNode",
+  "is_admin": false,
+  "election_active": false,
+  "candidates": [],
+  "last_heartbeat": "2025-01-08T15:30:00Z"
+}
+```
+
+#### Force Election (Admin Only)
+```bash
+curl -X POST http://localhost:8080/api/admin/trigger-election \
+  -H "Authorization: Admin QmYourNodeID"
+```
+
+#### View Admin Key Shares
+```bash
+curl http://localhost:8080/api/admin/key-shares \
+  -H "Authorization: Admin QmAdminNodeID"
+```
+
+### Shamir Secret Sharing
+
+Admin keys are distributed using Shamir secret sharing:
+- **5 total shares** distributed across cluster nodes
+- **3 shares required** to reconstruct admin key
+- **Automatic reconstruction** during elections
+- **Secure storage** of individual shares
+
+#### Share Management
+Each non-admin node stores one share:
+```bash
+# View your share (if you have one)
+curl http://localhost:8080/api/admin/my-share
+
+# Validate share integrity  
+curl http://localhost:8080/api/admin/validate-share
+```
+
+## Troubleshooting
+
+### Common Issues
+
+#### "DHT not connected"
+```
+⚠️ Failed to create DHT: connection refused
+```
+
+**Solution**: Check bootstrap peers in configuration:
+```yaml
+v2:
+  dht:
+    bootstrap_peers:
+      - "/ip4/192.168.1.100/tcp/4001/p2p/QmValidPeer"
+```
+
+#### "Age encryption failed"
+```
+❌ Age encryption test failed: invalid key format
+```
+
+**Solution**: Verify Age keys in `.ucxl/roles.yaml`:
+- Private key starts with `AGE-SECRET-KEY-1`
+- Public key starts with `age1`
+
+#### "No admin available"
+```
+⚠️ No admin found, triggering election
+```
+
+**Solution**: Wait for election to complete or manually trigger:
+```bash
+curl -X POST http://localhost:8080/api/admin/trigger-election
+```
+
+#### "Permission denied to decrypt"
+```
+❌ Current role cannot decrypt content from role: admin
+```
+
+**Solution**: This is expected - lower authority roles cannot decrypt higher authority content.
+
+### Debug Commands
+
+#### View Node Status
+```bash
+curl http://localhost:8080/api/debug/status | jq .
+```
+
+#### Check DHT Metrics
+```bash
+curl http://localhost:8080/api/debug/dht-metrics | jq .
+```
+
+#### List Recent Decisions
+```bash
+curl "http://localhost:8080/api/debug/recent-decisions?limit=5" | jq .
+```
+
+#### Test Connectivity
+```bash
+curl http://localhost:8080/api/debug/test-connectivity | jq .
+```
+
+### Log Analysis
+
+BZZZ provides detailed logging for troubleshooting:
+
+```bash
+# View startup logs
+tail -f /var/log/bzzz/startup.log
+
+# View decision publishing
+tail -f /var/log/bzzz/decisions.log  
+
+# View election activity
+tail -f /var/log/bzzz/elections.log
+
+# View DHT operations
+tail -f /var/log/bzzz/dht.log
+```
+
+Key log patterns to watch for:
+- `✅ Age encryption test passed` - Crypto working
+- `🕸️ DHT initialized` - DHT ready
+- `👑 Admin changed` - Election completed
+- `📤 Published task completion decision` - Publishing working
+
+## Best Practices
+
+### Security Best Practices
+
+1. **Secure Key Storage**:
+   - Store private keys in encrypted files
+   - Use environment variables in production
+   - Never commit keys to version control
+
+2. **Regular Key Rotation**:
+   - Rotate keys quarterly or after security incidents
+   - Coordinate rotation across cluster nodes
+   - Test key rotation in development first
+
+3. **Access Control**:
+   - Use principle of least privilege for roles
+   - Regularly audit role assignments
+   - Monitor unauthorized decryption attempts
+
+### Performance Best Practices
+
+1. **DHT Optimization**:
+   - Use multiple bootstrap peers for reliability
+   - Monitor DHT connection health
+   - Configure appropriate cache timeouts
+
+2. **Decision Publishing**:
+   - Batch similar decisions when possible
+   - Use appropriate content types for better organization
+   - Clean up old decisions periodically
+
+3. **Resource Management**:
+   - Monitor memory usage for large clusters
+   - Configure appropriate timeouts
+   - Use resource limits in production
+
+### Operational Best Practices
+
+1. **Monitoring**:
+   - Monitor admin election frequency
+   - Track decision publishing rates
+   - Alert on encryption failures
+
+2. **Backup & Recovery**:
+   - Backup role configurations
+   - Test admin key reconstruction
+   - Plan for cluster rebuild scenarios
+
+3. **Cluster Management**:
+   - Maintain odd number of nodes (3, 5, 7)
+   - Distribute nodes across network zones
+   - Plan for rolling updates
+
+---
+
+## Support & Documentation
+
+- **API Reference**: [API_REFERENCE.md](API_REFERENCE.md)
+- **Developer Guide**: [DEVELOPER.md](DEVELOPER.md)  
+- **Security Model**: [SECURITY.md](SECURITY.md)
+- **Troubleshooting**: [TROUBLESHOOTING.md](TROUBLESHOOTING.md)
+
+**BZZZ User Manual v2.0** - Complete guide for Phase 2B unified architecture with Age encryption and DHT storage.