🎉 MAJOR MILESTONE: Complete BZZZ Phase 2B documentation and core implementation ## Documentation Suite (7,000+ lines) - ✅ User Manual: Comprehensive guide with practical examples - ✅ API Reference: Complete REST API documentation - ✅ SDK Documentation: Multi-language SDK guide (Go, Python, JS, Rust) - ✅ Developer Guide: Development setup and contribution procedures - ✅ Architecture Documentation: Detailed system design with ASCII diagrams - ✅ Technical Report: Performance analysis and benchmarks - ✅ Security Documentation: Comprehensive security model - ✅ Operations Guide: Production deployment and monitoring - ✅ Documentation Index: Cross-referenced navigation system ## SDK Examples & Integration - 🔧 Go SDK: Simple client, event streaming, crypto operations - 🐍 Python SDK: Async client with comprehensive examples - 📜 JavaScript SDK: Collaborative agent implementation - 🦀 Rust SDK: High-performance monitoring system - 📖 Multi-language README with setup instructions ## Core Implementation - 🔐 Age encryption implementation (pkg/crypto/age_crypto.go) - 🗂️ Shamir secret sharing (pkg/crypto/shamir.go) - 💾 DHT encrypted storage (pkg/dht/encrypted_storage.go) - 📤 UCXL decision publisher (pkg/ucxl/decision_publisher.go) - 🔄 Updated main.go with Phase 2B integration ## Project Organization - 📂 Moved legacy docs to old-docs/ directory - 🎯 Comprehensive README.md update with modern structure - 🔗 Full cross-reference system between all documentation - 📊 Production-ready deployment procedures ## Quality Assurance - ✅ All documentation cross-referenced and validated - ✅ Working code examples in multiple languages - ✅ Production deployment procedures tested - ✅ Security best practices implemented - ✅ Performance benchmarks documented Ready for production deployment and community adoption. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
1072 lines
29 KiB
Markdown
1072 lines
29 KiB
Markdown
# BZZZ Developer Guide
|
|
|
|
**Version 2.0 - Phase 2B Edition**
|
|
Complete developer documentation for contributing to and extending BZZZ.
|
|
|
|
## Table of Contents
|
|
|
|
1. [Development Environment](#development-environment)
|
|
2. [Architecture Deep Dive](#architecture-deep-dive)
|
|
3. [Code Organization](#code-organization)
|
|
4. [Building & Testing](#building--testing)
|
|
5. [Extending BZZZ](#extending-bzzz)
|
|
6. [Debugging & Profiling](#debugging--profiling)
|
|
7. [Contributing Guidelines](#contributing-guidelines)
|
|
8. [Advanced Topics](#advanced-topics)
|
|
|
|
## Development Environment
|
|
|
|
### Prerequisites
|
|
|
|
**Required**:
|
|
- Go 1.23+ (for Go modules and generics support)
|
|
- Git (for version control)
|
|
- Make (for build automation)
|
|
|
|
**Optional but Recommended**:
|
|
- Docker & Docker Compose (for integration testing)
|
|
- golangci-lint (for code quality)
|
|
- delve (for debugging)
|
|
|
|
### Environment Setup
|
|
|
|
1. **Clone Repository**:
|
|
```bash
|
|
git clone https://github.com/anthonyrawlins/bzzz.git
|
|
cd bzzz
|
|
```
|
|
|
|
2. **Install Dependencies**:
|
|
```bash
|
|
go mod download
|
|
go mod verify
|
|
```
|
|
|
|
3. **Install Development Tools**:
|
|
```bash
|
|
# Install linter
|
|
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.54.2
|
|
|
|
# Install debugger
|
|
go install github.com/go-delve/delve/cmd/dlv@latest
|
|
|
|
# Install test coverage tools
|
|
go install golang.org/x/tools/cmd/cover@latest
|
|
```
|
|
|
|
4. **Verify Installation**:
|
|
```bash
|
|
make test
|
|
make lint
|
|
```
|
|
|
|
### IDE Configuration
|
|
|
|
#### VS Code Settings
|
|
Create `.vscode/settings.json`:
|
|
```json
|
|
{
|
|
"go.lintTool": "golangci-lint",
|
|
"go.lintFlags": [
|
|
"--fast"
|
|
],
|
|
"go.testFlags": ["-v"],
|
|
"go.testTimeout": "60s",
|
|
"go.coverageOptions": "showUncoveredCodeOnly",
|
|
"go.toolsManagement.checkForUpdates": "local"
|
|
}
|
|
```
|
|
|
|
#### GoLand Configuration
|
|
- Enable Go modules support
|
|
- Configure golangci-lint as external tool
|
|
- Set up run configurations for different components
|
|
|
|
**Cross-Reference**: Development setup in `docs/CONTRIBUTING.md`
|
|
|
|
## Architecture Deep Dive
|
|
|
|
### System Components
|
|
|
|
BZZZ Phase 2B consists of several interconnected components:
|
|
|
|
```
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ Main App │────│ Election Mgr │────│ Admin Roles │
|
|
│ (main.go) │ │ (election/) │ │ (config/) │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
│ │ │
|
|
▼ ▼ ▼
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ P2P Network │────│ DHT Storage │────│ Age Crypto │
|
|
│ (p2p/) │ │ (dht/) │ │ (crypto/) │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
│ │ │
|
|
▼ ▼ ▼
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ PubSub │────│ Decision Pub │────│ UCXL Protocol │
|
|
│ (pubsub/) │ │ (ucxl/) │ │ (ucxi/) │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
```
|
|
|
|
### Data Flow Architecture
|
|
|
|
#### Decision Publishing Flow
|
|
```
|
|
Task Completion → Decision Publisher → Age Encryption → DHT Storage
|
|
↓ ↓ ↓ ↓
|
|
Task Tracker → UCXL Address → Role Keys → P2P DHT
|
|
↓ ↓ ↓ ↓
|
|
Metadata Gen → Content Format → Multi-Recip → Cache + Announce
|
|
```
|
|
|
|
**Implementation Files**:
|
|
- `main.go:CompleteTaskWithDecision()` - Task completion trigger
|
|
- `pkg/ucxl/decision_publisher.go` - Decision publishing logic
|
|
- `pkg/crypto/age_crypto.go` - Encryption implementation
|
|
- `pkg/dht/encrypted_storage.go` - DHT storage with encryption
|
|
|
|
#### Election & Admin Flow
|
|
```
|
|
Heartbeat Timeout → Election Trigger → Consensus Vote → Admin Selection
|
|
↓ ↓ ↓ ↓
|
|
Monitor Admin → Candidate List → Raft Protocol → Key Reconstruction
|
|
↓ ↓ ↓ ↓
|
|
Health Checks → Score Calculation → Share Collection → SLURP Enable
|
|
```
|
|
|
|
**Implementation Files**:
|
|
- `pkg/election/election.go` - Complete election system
|
|
- `pkg/crypto/shamir.go` - Key reconstruction
|
|
- `pkg/config/roles.go` - Role and authority management
|
|
|
|
### Security Architecture
|
|
|
|
#### Role-Based Access Control
|
|
```
|
|
Content Creation → Authority Check → Key Selection → Multi-Recipient Encryption
|
|
↓ ↓ ↓ ↓
|
|
Role Definition → Authority Level → Age Key Pairs → Encrypted Content
|
|
↓ ↓ ↓ ↓
|
|
Config System → Hierarchy Rules → Crypto Module → DHT Storage
|
|
```
|
|
|
|
#### Key Management System
|
|
```
|
|
Admin Key → Shamir Split → Distribute Shares → Consensus → Reconstruction
|
|
↓ ↓ ↓ ↓ ↓
|
|
Master Key → 5 Shares → Node Storage → Election → Admin Access
|
|
↓ ↓ ↓ ↓ ↓
|
|
Age Identity → Threshold 3 → Secure Store → Validate → Enable SLURP
|
|
```
|
|
|
|
**Cross-Reference**: Security model details in `docs/SECURITY.md`
|
|
|
|
## Code Organization
|
|
|
|
### Package Structure
|
|
|
|
```
|
|
bzzz/
|
|
├── main.go # Application entry point
|
|
├── go.mod # Go module definition
|
|
├── go.sum # Dependency checksums
|
|
├── Makefile # Build automation
|
|
│
|
|
├── api/ # HTTP API handlers
|
|
│ ├── handlers.go # Request handlers
|
|
│ ├── middleware.go # Authentication, logging
|
|
│ └── server.go # HTTP server setup
|
|
│
|
|
├── pkg/ # Core packages
|
|
│ ├── config/ # Configuration management
|
|
│ │ ├── config.go # Main configuration
|
|
│ │ ├── roles.go # Role definitions
|
|
│ │ └── defaults.go # Default values
|
|
│ │
|
|
│ ├── crypto/ # Cryptographic operations
|
|
│ │ ├── age_crypto.go # Age encryption
|
|
│ │ └── shamir.go # Secret sharing
|
|
│ │
|
|
│ ├── dht/ # DHT storage
|
|
│ │ └── encrypted_storage.go # Encrypted DHT operations
|
|
│ │
|
|
│ ├── election/ # Admin elections
|
|
│ │ └── election.go # Election management
|
|
│ │
|
|
│ ├── ucxl/ # UCXL protocol
|
|
│ │ ├── address.go # Address parsing
|
|
│ │ └── decision_publisher.go # Decision publishing
|
|
│ │
|
|
│ └── ucxi/ # UCXI interface
|
|
│ ├── server.go # UCXI REST server
|
|
│ └── storage.go # Content storage
|
|
│
|
|
├── docs/ # Documentation
|
|
│ ├── README.md # Documentation index
|
|
│ ├── USER_MANUAL.md # User guide
|
|
│ ├── API_REFERENCE.md # API documentation
|
|
│ └── DEVELOPER.md # This file
|
|
│
|
|
└── tests/ # Test files
|
|
├── integration/ # Integration tests
|
|
├── unit/ # Unit tests
|
|
└── fixtures/ # Test data
|
|
```
|
|
|
|
### Coding Standards
|
|
|
|
#### Go Style Guidelines
|
|
|
|
**Package Naming**:
|
|
- Use short, lowercase names
|
|
- Avoid underscores or mixed caps
|
|
- Be descriptive but concise
|
|
|
|
```go
|
|
// Good
|
|
package crypto
|
|
package election
|
|
|
|
// Avoid
|
|
package cryptoOperations
|
|
package election_management
|
|
```
|
|
|
|
**Function Documentation**:
|
|
```go
|
|
// EncryptForRole encrypts content for a specific role using Age encryption.
|
|
//
|
|
// This function encrypts the provided content using the Age public key
|
|
// associated with the specified role. The encrypted content can only be
|
|
// decrypted by agents with the corresponding private key and appropriate
|
|
// authority level.
|
|
//
|
|
// Parameters:
|
|
// content: Raw content bytes to encrypt
|
|
// roleName: Target role name for encryption
|
|
//
|
|
// Returns:
|
|
// []byte: Encrypted content in Age format
|
|
// error: Any error during encryption process
|
|
//
|
|
// Cross-references:
|
|
// - DecryptWithRole(): Corresponding decryption function
|
|
// - pkg/config/roles.go: Role definitions and key management
|
|
// - docs/SECURITY.md: Security model and threat analysis
|
|
func (ac *AgeCrypto) EncryptForRole(content []byte, roleName string) ([]byte, error) {
|
|
// Implementation...
|
|
}
|
|
```
|
|
|
|
**Error Handling**:
|
|
```go
|
|
// Good - Wrap errors with context
|
|
if err != nil {
|
|
return fmt.Errorf("failed to encrypt content for role %s: %w", roleName, err)
|
|
}
|
|
|
|
// Good - Use custom error types for specific cases
|
|
type InvalidRoleError struct {
|
|
Role string
|
|
}
|
|
|
|
func (e InvalidRoleError) Error() string {
|
|
return fmt.Sprintf("invalid role: %s", e.Role)
|
|
}
|
|
```
|
|
|
|
#### Configuration Management
|
|
|
|
All configuration uses the centralized config system:
|
|
|
|
```go
|
|
// Reading configuration
|
|
cfg, err := config.LoadConfig("config.yaml")
|
|
if err != nil {
|
|
return fmt.Errorf("failed to load config: %w", err)
|
|
}
|
|
|
|
// Accessing role information
|
|
role, exists := config.GetPredefinedRoles()[cfg.Agent.Role]
|
|
if !exists {
|
|
return fmt.Errorf("role %s not found", cfg.Agent.Role)
|
|
}
|
|
|
|
// Checking permissions
|
|
canDecrypt, err := cfg.CanDecryptRole(targetRole)
|
|
if err != nil {
|
|
return fmt.Errorf("permission check failed: %w", err)
|
|
}
|
|
```
|
|
|
|
**Cross-Reference**: Configuration system in `pkg/config/config.go`
|
|
|
|
### Testing Conventions
|
|
|
|
#### Unit Tests
|
|
```go
|
|
func TestAgeCrypto_EncryptForRole(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
content []byte
|
|
roleName string
|
|
wantErr bool
|
|
errorContains string
|
|
}{
|
|
{
|
|
name: "successful encryption",
|
|
content: []byte("test content"),
|
|
roleName: "backend_developer",
|
|
wantErr: false,
|
|
},
|
|
{
|
|
name: "invalid role",
|
|
content: []byte("test content"),
|
|
roleName: "nonexistent_role",
|
|
wantErr: true,
|
|
errorContains: "role 'nonexistent_role' not found",
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
// Test implementation
|
|
cfg := &config.Config{/* test config */}
|
|
ac := NewAgeCrypto(cfg)
|
|
|
|
encrypted, err := ac.EncryptForRole(tt.content, tt.roleName)
|
|
|
|
if tt.wantErr {
|
|
require.Error(t, err)
|
|
if tt.errorContains != "" {
|
|
assert.Contains(t, err.Error(), tt.errorContains)
|
|
}
|
|
return
|
|
}
|
|
|
|
require.NoError(t, err)
|
|
assert.NotEmpty(t, encrypted)
|
|
assert.NotEqual(t, tt.content, encrypted)
|
|
})
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Integration Tests
|
|
```go
|
|
func TestEndToEndDecisionFlow(t *testing.T) {
|
|
// Set up test cluster
|
|
cluster := setupTestCluster(t, 3)
|
|
defer cluster.Shutdown()
|
|
|
|
// Test decision publishing
|
|
decision := &ucxl.TaskDecision{
|
|
Task: "test_task",
|
|
Decision: "test decision content",
|
|
Success: true,
|
|
}
|
|
|
|
err := cluster.PublishDecision("backend_developer", decision)
|
|
require.NoError(t, err)
|
|
|
|
// Test content retrieval
|
|
retrieved, err := cluster.RetrieveDecision(decision.UCXLAddress())
|
|
require.NoError(t, err)
|
|
|
|
assert.Equal(t, decision.Decision, retrieved.Decision)
|
|
}
|
|
```
|
|
|
|
**Cross-Reference**: Test examples in `tests/` directory
|
|
|
|
## Building & Testing
|
|
|
|
### Build System
|
|
|
|
#### Makefile Targets
|
|
```make
|
|
# Build binary
|
|
build:
|
|
go build -o bin/bzzz main.go
|
|
|
|
# Run tests
|
|
test:
|
|
go test -v ./...
|
|
|
|
# Run tests with coverage
|
|
test-coverage:
|
|
go test -v -coverprofile=coverage.out ./...
|
|
go tool cover -html=coverage.out -o coverage.html
|
|
|
|
# Lint code
|
|
lint:
|
|
golangci-lint run
|
|
|
|
# Format code
|
|
fmt:
|
|
go fmt ./...
|
|
goimports -w .
|
|
|
|
# Clean build artifacts
|
|
clean:
|
|
rm -rf bin/ coverage.out coverage.html
|
|
|
|
# Development server with hot reload
|
|
dev:
|
|
air -c .air.toml
|
|
|
|
# Integration tests
|
|
test-integration:
|
|
go test -v -tags=integration ./tests/integration/
|
|
|
|
# Generate documentation
|
|
docs:
|
|
go doc -all ./... > docs/GENERATED_API_DOCS.md
|
|
```
|
|
|
|
#### Build Configuration
|
|
```go
|
|
// Build with version information
|
|
go build -ldflags "-X main.version=$(git describe --tags) -X main.commit=$(git rev-parse --short HEAD)" -o bzzz main.go
|
|
```
|
|
|
|
### Testing Strategy
|
|
|
|
#### Test Categories
|
|
|
|
1. **Unit Tests** (`*_test.go` files)
|
|
- Individual function testing
|
|
- Mock external dependencies
|
|
- Fast execution (< 1s per test)
|
|
|
|
2. **Integration Tests** (`tests/integration/`)
|
|
- Component interaction testing
|
|
- Real dependencies where possible
|
|
- Moderate execution time (< 30s per test)
|
|
|
|
3. **End-to-End Tests** (`tests/e2e/`)
|
|
- Full system testing
|
|
- Multiple nodes, real network
|
|
- Longer execution time (< 5min per test)
|
|
|
|
#### Running Tests
|
|
|
|
```bash
|
|
# All tests
|
|
make test
|
|
|
|
# Specific package
|
|
go test -v ./pkg/crypto/
|
|
|
|
# Specific test
|
|
go test -v ./pkg/crypto/ -run TestAgeCrypto_EncryptForRole
|
|
|
|
# With race detection
|
|
go test -race ./...
|
|
|
|
# With coverage
|
|
make test-coverage
|
|
|
|
# Integration tests only
|
|
make test-integration
|
|
|
|
# Benchmark tests
|
|
go test -bench=. ./pkg/crypto/
|
|
```
|
|
|
|
#### Test Data Management
|
|
|
|
```go
|
|
// Use testdata directory for test fixtures
|
|
func loadTestConfig(t *testing.T) *config.Config {
|
|
configPath := filepath.Join("testdata", "test_config.yaml")
|
|
cfg, err := config.LoadConfig(configPath)
|
|
require.NoError(t, err)
|
|
return cfg
|
|
}
|
|
|
|
// Generate test data deterministically
|
|
func generateTestKeys(t *testing.T) *config.AgeKeyPair {
|
|
// Use fixed seed for reproducible tests
|
|
oldRand := rand.Reader
|
|
defer func() { rand.Reader = oldRand }()
|
|
|
|
seed := make([]byte, 32)
|
|
copy(seed, "test-seed-for-deterministic-keys")
|
|
rand.Reader = bytes.NewReader(seed)
|
|
|
|
keyPair, err := crypto.GenerateAgeKeyPair()
|
|
require.NoError(t, err)
|
|
return keyPair
|
|
}
|
|
```
|
|
|
|
## Extending BZZZ
|
|
|
|
### Adding New Decision Types
|
|
|
|
1. **Define Decision Structure**:
|
|
```go
|
|
// pkg/ucxl/decision_types.go
|
|
type CustomDecision struct {
|
|
TaskDecision // Embed base decision
|
|
CustomField1 string `json:"custom_field_1"`
|
|
CustomField2 map[string]int `json:"custom_field_2"`
|
|
CustomField3 []CustomSubType `json:"custom_field_3"`
|
|
}
|
|
|
|
type CustomSubType struct {
|
|
Name string `json:"name"`
|
|
Value int `json:"value"`
|
|
}
|
|
```
|
|
|
|
2. **Add Publisher Method**:
|
|
```go
|
|
// pkg/ucxl/decision_publisher.go
|
|
func (dp *DecisionPublisher) PublishCustomDecision(
|
|
taskName string,
|
|
customField1 string,
|
|
customField2 map[string]int,
|
|
customField3 []CustomSubType,
|
|
) error {
|
|
decision := &CustomDecision{
|
|
TaskDecision: TaskDecision{
|
|
Task: taskName,
|
|
Decision: "Custom decision type",
|
|
Success: true,
|
|
Context: map[string]interface{}{
|
|
"decision_type": "custom",
|
|
"node_id": dp.nodeID,
|
|
},
|
|
},
|
|
CustomField1: customField1,
|
|
CustomField2: customField2,
|
|
CustomField3: customField3,
|
|
}
|
|
|
|
return dp.PublishTaskDecision(&decision.TaskDecision)
|
|
}
|
|
```
|
|
|
|
3. **Add API Endpoint**:
|
|
```go
|
|
// api/handlers.go
|
|
func (h *Handlers) HandleCustomDecision(w http.ResponseWriter, r *http.Request) {
|
|
var req struct {
|
|
Task string `json:"task"`
|
|
CustomField1 string `json:"custom_field_1"`
|
|
CustomField2 map[string]int `json:"custom_field_2"`
|
|
CustomField3 []CustomSubType `json:"custom_field_3"`
|
|
}
|
|
|
|
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
|
http.Error(w, "Invalid request body", http.StatusBadRequest)
|
|
return
|
|
}
|
|
|
|
err := h.decisionPublisher.PublishCustomDecision(
|
|
req.Task,
|
|
req.CustomField1,
|
|
req.CustomField2,
|
|
req.CustomField3,
|
|
)
|
|
|
|
if err != nil {
|
|
http.Error(w, err.Error(), http.StatusInternalServerError)
|
|
return
|
|
}
|
|
|
|
w.WriteHeader(http.StatusCreated)
|
|
json.NewEncoder(w).Encode(map[string]string{
|
|
"status": "published",
|
|
"type": "custom",
|
|
})
|
|
}
|
|
```
|
|
|
|
4. **Register Route**:
|
|
```go
|
|
// api/server.go
|
|
func (s *HTTPServer) setupRoutes() {
|
|
// ... existing routes
|
|
s.router.HandleFunc("/api/decisions/custom", s.handlers.HandleCustomDecision).Methods("POST")
|
|
}
|
|
```
|
|
|
|
### Adding New Role Types
|
|
|
|
1. **Define Role in Configuration**:
|
|
```yaml
|
|
# .ucxl/roles.yaml
|
|
data_scientist:
|
|
authority_level: decision
|
|
can_decrypt: [data_scientist, backend_developer, observer]
|
|
model: ollama/llama3.1
|
|
decision_scope: [data, analytics, ml_models]
|
|
special_functions: [data_analysis, model_training]
|
|
age_keys:
|
|
public_key: "age1..."
|
|
private_key: "AGE-SECRET-KEY-1..."
|
|
```
|
|
|
|
2. **Add Role Logic**:
|
|
```go
|
|
// pkg/config/roles.go
|
|
func GetDataScienceDefaults() RoleDefinition {
|
|
return RoleDefinition{
|
|
AuthorityLevel: AuthorityDecision,
|
|
CanDecrypt: []string{"data_scientist", "backend_developer", "observer"},
|
|
Model: "ollama/llama3.1",
|
|
DecisionScope: []string{"data", "analytics", "ml_models"},
|
|
SpecialFunctions: []string{"data_analysis", "model_training"},
|
|
AgeKeys: AgeKeyPair{}, // Will be populated at runtime
|
|
}
|
|
}
|
|
|
|
// Add to role factory
|
|
func GetPredefinedRoles() map[string]RoleDefinition {
|
|
return map[string]RoleDefinition{
|
|
// ... existing roles
|
|
"data_scientist": GetDataScienceDefaults(),
|
|
}
|
|
}
|
|
```
|
|
|
|
3. **Add Specialized Methods**:
|
|
```go
|
|
// pkg/ucxl/data_science_publisher.go
|
|
func (dp *DecisionPublisher) PublishModelTrainingDecision(
|
|
modelName string,
|
|
trainingResults *ModelTrainingResults,
|
|
datasetInfo *DatasetInfo,
|
|
) error {
|
|
decision := &TaskDecision{
|
|
Task: fmt.Sprintf("train_model_%s", modelName),
|
|
Decision: fmt.Sprintf("Trained %s model with accuracy %.2f", modelName, trainingResults.Accuracy),
|
|
Success: trainingResults.Accuracy > 0.8,
|
|
Context: map[string]interface{}{
|
|
"decision_type": "model_training",
|
|
"model_name": modelName,
|
|
"training_results": trainingResults,
|
|
"dataset_info": datasetInfo,
|
|
"node_id": dp.nodeID,
|
|
},
|
|
}
|
|
|
|
return dp.PublishTaskDecision(decision)
|
|
}
|
|
```
|
|
|
|
### Adding New Encryption Methods
|
|
|
|
1. **Extend Crypto Interface**:
|
|
```go
|
|
// pkg/crypto/interfaces.go
|
|
type CryptoProvider interface {
|
|
EncryptForRole(content []byte, role string) ([]byte, error)
|
|
DecryptWithRole(encrypted []byte) ([]byte, error)
|
|
GenerateKeyPair() (KeyPair, error)
|
|
ValidateKey(key string, isPrivate bool) error
|
|
}
|
|
|
|
type KeyPair interface {
|
|
PublicKey() string
|
|
PrivateKey() string
|
|
Type() string
|
|
}
|
|
```
|
|
|
|
2. **Implement New Provider**:
|
|
```go
|
|
// pkg/crypto/nacl_crypto.go
|
|
type NaClCrypto struct {
|
|
config *config.Config
|
|
}
|
|
|
|
func NewNaClCrypto(cfg *config.Config) *NaClCrypto {
|
|
return &NaClCrypto{config: cfg}
|
|
}
|
|
|
|
func (nc *NaClCrypto) EncryptForRole(content []byte, role string) ([]byte, error) {
|
|
// NaCl/libsodium implementation
|
|
// ...
|
|
}
|
|
```
|
|
|
|
3. **Add to Configuration**:
|
|
```yaml
|
|
# config.yaml
|
|
crypto:
|
|
provider: "age" # or "nacl", "pgp", etc.
|
|
age:
|
|
# Age-specific settings
|
|
nacl:
|
|
# NaCl-specific settings
|
|
```
|
|
|
|
## Debugging & Profiling
|
|
|
|
### Debug Configuration
|
|
|
|
#### Development Build
|
|
```bash
|
|
# Build with debug information
|
|
go build -gcflags="all=-N -l" -o bzzz-debug main.go
|
|
|
|
# Run with debugger
|
|
dlv exec ./bzzz-debug
|
|
```
|
|
|
|
#### Runtime Debugging
|
|
```go
|
|
// Add debug endpoints
|
|
func (s *HTTPServer) setupDebugRoutes() {
|
|
if s.config.Debug.Enabled {
|
|
s.router.HandleFunc("/debug/pprof/", pprof.Index)
|
|
s.router.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
|
|
s.router.HandleFunc("/debug/pprof/profile", pprof.Profile)
|
|
s.router.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
|
|
s.router.HandleFunc("/debug/pprof/trace", pprof.Trace)
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Structured Logging
|
|
```go
|
|
// Use structured logging throughout
|
|
import "github.com/rs/zerolog/log"
|
|
|
|
func (ac *AgeCrypto) EncryptForRole(content []byte, roleName string) ([]byte, error) {
|
|
logger := log.With().
|
|
Str("component", "age_crypto").
|
|
Str("operation", "encrypt_for_role").
|
|
Str("role", roleName).
|
|
Int("content_size", len(content)).
|
|
Logger()
|
|
|
|
logger.Debug().Msg("Starting encryption")
|
|
|
|
// ... encryption logic
|
|
|
|
logger.Info().
|
|
Int("encrypted_size", len(encrypted)).
|
|
Dur("duration", time.Since(start)).
|
|
Msg("Encryption completed successfully")
|
|
|
|
return encrypted, nil
|
|
}
|
|
```
|
|
|
|
### Performance Profiling
|
|
|
|
#### CPU Profiling
|
|
```bash
|
|
# Generate CPU profile
|
|
go test -cpuprofile=cpu.prof -bench=. ./pkg/crypto/
|
|
|
|
# Analyze profile
|
|
go tool pprof cpu.prof
|
|
(pprof) top10
|
|
(pprof) list EncryptForRole
|
|
(pprof) web
|
|
```
|
|
|
|
#### Memory Profiling
|
|
```bash
|
|
# Generate memory profile
|
|
go test -memprofile=mem.prof -bench=. ./pkg/crypto/
|
|
|
|
# Analyze memory usage
|
|
go tool pprof mem.prof
|
|
(pprof) top10 -cum
|
|
(pprof) list NewAgeCrypto
|
|
```
|
|
|
|
#### Benchmarking
|
|
```go
|
|
// pkg/crypto/age_crypto_bench_test.go
|
|
func BenchmarkAgeCrypto_EncryptForRole(b *testing.B) {
|
|
cfg := &config.Config{/* test config */}
|
|
ac := NewAgeCrypto(cfg)
|
|
content := make([]byte, 1024) // 1KB test content
|
|
|
|
b.ResetTimer()
|
|
b.ReportAllocs()
|
|
|
|
for i := 0; i < b.N; i++ {
|
|
_, err := ac.EncryptForRole(content, "backend_developer")
|
|
if err != nil {
|
|
b.Fatal(err)
|
|
}
|
|
}
|
|
}
|
|
|
|
func BenchmarkDHTStorage_StoreRetrieve(b *testing.B) {
|
|
storage := setupTestDHTStorage(b)
|
|
content := generateTestContent(1024)
|
|
|
|
b.Run("Store", func(b *testing.B) {
|
|
for i := 0; i < b.N; i++ {
|
|
addr := fmt.Sprintf("test/role/project/task/%d", i)
|
|
err := storage.StoreUCXLContent(addr, content, "test_role", "benchmark")
|
|
if err != nil {
|
|
b.Fatal(err)
|
|
}
|
|
}
|
|
})
|
|
|
|
b.Run("Retrieve", func(b *testing.B) {
|
|
// Pre-populate content
|
|
for i := 0; i < b.N; i++ {
|
|
addr := fmt.Sprintf("test/role/project/task/%d", i)
|
|
_, _, err := storage.RetrieveUCXLContent(addr)
|
|
if err != nil {
|
|
b.Fatal(err)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
```
|
|
|
|
## Contributing Guidelines
|
|
|
|
### Code Submission Process
|
|
|
|
1. **Fork & Branch**:
|
|
```bash
|
|
git clone https://github.com/your-username/bzzz.git
|
|
cd bzzz
|
|
git checkout -b feature/your-feature-name
|
|
```
|
|
|
|
2. **Development Workflow**:
|
|
```bash
|
|
# Make changes
|
|
# Add tests
|
|
make test
|
|
make lint
|
|
|
|
# Commit changes
|
|
git add .
|
|
git commit -m "feat: add custom decision type support
|
|
|
|
- Add CustomDecision struct with validation
|
|
- Implement PublishCustomDecision method
|
|
- Add API endpoint with proper error handling
|
|
- Include comprehensive tests and documentation
|
|
|
|
Cross-references:
|
|
- pkg/ucxl/decision_publisher.go: Core implementation
|
|
- api/handlers.go: HTTP API integration
|
|
- docs/API_REFERENCE.md: API documentation update"
|
|
```
|
|
|
|
3. **Pull Request**:
|
|
```bash
|
|
git push origin feature/your-feature-name
|
|
# Create PR via GitHub interface
|
|
```
|
|
|
|
### Commit Message Format
|
|
|
|
Use conventional commits format:
|
|
```
|
|
<type>[optional scope]: <description>
|
|
|
|
[optional body]
|
|
|
|
[optional footer(s)]
|
|
```
|
|
|
|
**Types**:
|
|
- `feat`: New feature
|
|
- `fix`: Bug fix
|
|
- `docs`: Documentation changes
|
|
- `refactor`: Code refactoring
|
|
- `test`: Test additions/modifications
|
|
- `chore`: Build system, dependencies
|
|
|
|
**Cross-References**:
|
|
Always include relevant cross-references:
|
|
```
|
|
Cross-references:
|
|
- pkg/crypto/age_crypto.go:123: Implementation detail
|
|
- docs/SECURITY.md: Security implications
|
|
- tests/integration/crypto_test.go: Test coverage
|
|
```
|
|
|
|
### Code Review Checklist
|
|
|
|
#### For Authors
|
|
- [ ] All tests pass (`make test`)
|
|
- [ ] Code is formatted (`make fmt`)
|
|
- [ ] Linting passes (`make lint`)
|
|
- [ ] Documentation updated
|
|
- [ ] Cross-references added
|
|
- [ ] Performance considered
|
|
- [ ] Security implications reviewed
|
|
|
|
#### for Reviewers
|
|
- [ ] Code follows established patterns
|
|
- [ ] Error handling is comprehensive
|
|
- [ ] Tests cover edge cases
|
|
- [ ] Documentation is accurate
|
|
- [ ] Security model is maintained
|
|
- [ ] Performance impact is acceptable
|
|
|
|
**Cross-Reference**: Full contributing guide in `docs/CONTRIBUTING.md`
|
|
|
|
## Advanced Topics
|
|
|
|
### Custom Consensus Algorithms
|
|
|
|
To implement alternative consensus algorithms:
|
|
|
|
1. **Define Interface**:
|
|
```go
|
|
// pkg/election/consensus.go
|
|
type ConsensusProvider interface {
|
|
ProposeCandidate(candidate *ElectionCandidate) error
|
|
Vote(candidateID string, vote bool) error
|
|
GetWinner() (*ElectionCandidate, error)
|
|
IsComplete() bool
|
|
}
|
|
```
|
|
|
|
2. **Implement Algorithm**:
|
|
```go
|
|
// pkg/election/pbft_consensus.go
|
|
type PBFTConsensus struct {
|
|
nodes map[string]*Node
|
|
proposals map[string]*Proposal
|
|
votes map[string]map[string]bool
|
|
threshold int
|
|
}
|
|
|
|
func (p *PBFTConsensus) ProposeCandidate(candidate *ElectionCandidate) error {
|
|
// PBFT implementation
|
|
}
|
|
```
|
|
|
|
3. **Register in Factory**:
|
|
```go
|
|
// pkg/election/factory.go
|
|
func NewConsensusProvider(algorithm string, config *Config) ConsensusProvider {
|
|
switch algorithm {
|
|
case "raft":
|
|
return NewRaftConsensus(config)
|
|
case "pbft":
|
|
return NewPBFTConsensus(config)
|
|
default:
|
|
return NewRaftConsensus(config) // Default
|
|
}
|
|
}
|
|
```
|
|
|
|
### Custom Storage Backends
|
|
|
|
Implement alternative storage backends:
|
|
|
|
1. **Storage Interface**:
|
|
```go
|
|
// pkg/storage/interfaces.go
|
|
type StorageProvider interface {
|
|
Store(key string, content []byte, metadata *Metadata) error
|
|
Retrieve(key string) ([]byte, *Metadata, error)
|
|
Search(query *SearchQuery) ([]*Metadata, error)
|
|
Delete(key string) error
|
|
GetMetrics() *StorageMetrics
|
|
}
|
|
```
|
|
|
|
2. **Implementation Example**:
|
|
```go
|
|
// pkg/storage/redis_storage.go
|
|
type RedisStorage struct {
|
|
client redis.Client
|
|
crypto *crypto.AgeCrypto
|
|
}
|
|
|
|
func (rs *RedisStorage) Store(key string, content []byte, metadata *Metadata) error {
|
|
// Encrypt content
|
|
encrypted, err := rs.crypto.EncryptForRole(content, metadata.CreatorRole)
|
|
if err != nil {
|
|
return fmt.Errorf("encryption failed: %w", err)
|
|
}
|
|
|
|
// Store in Redis
|
|
return rs.client.Set(context.Background(), key, encrypted, metadata.TTL).Err()
|
|
}
|
|
```
|
|
|
|
### Plugin System
|
|
|
|
Implement a plugin system for extensibility:
|
|
|
|
1. **Plugin Interface**:
|
|
```go
|
|
// pkg/plugins/interface.go
|
|
type Plugin interface {
|
|
Name() string
|
|
Version() string
|
|
Initialize(config map[string]interface{}) error
|
|
Shutdown() error
|
|
}
|
|
|
|
type DecisionPlugin interface {
|
|
Plugin
|
|
ProcessDecision(decision *ucxl.TaskDecision) (*ucxl.TaskDecision, error)
|
|
}
|
|
|
|
type CryptoPlugin interface {
|
|
Plugin
|
|
Encrypt(content []byte, key string) ([]byte, error)
|
|
Decrypt(content []byte, key string) ([]byte, error)
|
|
}
|
|
```
|
|
|
|
2. **Plugin Manager**:
|
|
```go
|
|
// pkg/plugins/manager.go
|
|
type Manager struct {
|
|
plugins map[string]Plugin
|
|
hooks map[string][]Plugin
|
|
}
|
|
|
|
func (m *Manager) LoadPlugin(path string) error {
|
|
// Load plugin from shared library
|
|
plug, err := plugin.Open(path)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
// Get plugin symbol
|
|
symPlugin, err := plug.Lookup("BzzzPlugin")
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
// Type assert and register
|
|
bzzzPlugin, ok := symPlugin.(Plugin)
|
|
if !ok {
|
|
return fmt.Errorf("invalid plugin interface")
|
|
}
|
|
|
|
m.plugins[bzzzPlugin.Name()] = bzzzPlugin
|
|
return bzzzPlugin.Initialize(nil)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Cross-References
|
|
|
|
- **User Manual**: [USER_MANUAL.md](USER_MANUAL.md) - End-user guide
|
|
- **API Reference**: [API_REFERENCE.md](API_REFERENCE.md) - Complete API docs
|
|
- **Security Model**: [SECURITY.md](SECURITY.md) - Security architecture
|
|
- **Configuration**: [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md) - Config options
|
|
- **Contributing**: [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines
|
|
|
|
**BZZZ Developer Guide v2.0** - Complete development documentation for Phase 2B unified architecture with Age encryption and DHT storage. |