bzzz/deployments/internal-docker/README-HCFS-Integration.md

HCFS-Integrated Development Environment

This directory contains Docker configurations for creating HCFS-enabled development environments that provide AI agents with persistent, context-aware workspaces.

🎯 Overview

Instead of using temporary directories that are lost when containers stop, this system integrates with HCFS (Hierarchical Context File System) to provide:

• Persistent Workspaces: Agent work is stored in HCFS and survives container restarts
• Context Sharing: Multiple agents can access and build upon each other's work
• Intelligent Artifact Collection: Important files are automatically stored in HCFS
• Role-Based Access: Agents can access context relevant to their specialization
• Feedback Learning: The RL Context Curator learns from agent success/failure patterns

🏗️ Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Bzzz Agents    │    │ HCFS-Enabled    │    │ HCFS Core       │
│                 │    │ Containers      │    │                 │
│ • CLI Agents    │◄──►│                 │◄──►│ • Context API   │
│ • Ollama Models │    │ • Python Dev    │    │ • RL Curator    │
│ • Reasoning     │    │ • Node.js Dev   │    │ • Storage       │
│ • Code Review   │    │ • Go Dev        │    │ • Search        │
└─────────────────┘    │ • Generic Base  │    └─────────────────┘
                       └─────────────────┘

🐳 Available Images

Base Image: bzzz-hcfs-base

• Ubuntu 22.04 with HCFS integration
• Standard development tools (git, make, curl, etc.)
• HCFS workspace management scripts
• Agent user with proper permissions
• FUSE support for HCFS mounting

Language-Specific Images:

bzzz-hcfs-python

• Python 3.10 with comprehensive ML/AI packages
• Jupyter Lab/Notebook support
• Popular frameworks: Flask, FastAPI, Django
• Data science stack: NumPy, Pandas, scikit-learn
• Deep learning: PyTorch, Transformers
• Ports: 8888 (Jupyter), 8000, 5000, 8080

bzzz-hcfs-nodejs

• Node.js 20 with modern JavaScript/TypeScript tools
• Package managers: npm, yarn
• Build tools: Webpack, Vite, Rollup
• Testing: Jest, Mocha, Cypress
• Ports: 3000, 8080, 8000, 9229 (debugger)

bzzz-hcfs-go

• Go 1.21 with standard development tools
• Popular frameworks: Gin, Echo, Fiber
• Development tools: Delve debugger, Air live reload
• Ports: 8080, 8000, 9000, 2345 (debugger)

🚀 Quick Start

1. Build the Images

cd /home/tony/AI/projects/Bzzz/docker
./build-hcfs-images.sh build

2. Start the HCFS Ecosystem

docker-compose -f docker-compose.hcfs.yml up -d

3. Access Development Environments

Python Development:

# Interactive shell
docker exec -it agent-python-dev bash

# Jupyter Lab
open http://localhost:8888

Node.js Development:

# Interactive shell  
docker exec -it agent-nodejs-dev bash

# Start development server
docker exec -it agent-nodejs-dev npm run dev

Go Development:

# Interactive shell
docker exec -it agent-go-dev bash

# Build and run
docker exec -it agent-go-dev make build run

🔧 Configuration

Environment Variables

Required for HCFS Integration:

• AGENT_ID: Unique identifier for the agent
• TASK_ID: Task identifier for workspace context
• HCFS_API_URL: HCFS API endpoint (default: http://host.docker.internal:8000)
• HCFS_ENABLED: Enable/disable HCFS integration (default: true)

Optional:

• GIT_USER_NAME: Git configuration
• GIT_USER_EMAIL: Git configuration
• SETUP_PYTHON_VENV: Create Python virtual environment
• NODE_ENV: Node.js environment mode

HCFS Configuration

Each container includes /etc/hcfs/hcfs-agent.yaml with:

• API endpoints and timeouts
• Workspace settings
• Artifact collection patterns
• Security configurations
• Logging preferences

💾 Workspace Management

Automatic Features

1. Workspace Initialization: Creates HCFS context for agent workspace

2. Continuous Sync: Background daemon syncs workspace state every 30 seconds

3. Artifact Collection: Automatically stores important files:

   • Log files (*.log)
   • Documentation (.md, README)
   • Configuration (*.json, *.yaml)
   • Build outputs (build/, output/)
   • Results (results/*)

4. Graceful Shutdown: Collects final artifacts when container stops

Manual Commands

# Sync current workspace state
/opt/hcfs/hcfs-workspace.sh sync

# Collect and store artifacts
/opt/hcfs/hcfs-workspace.sh collect

# Finalize workspace (run on completion)
/opt/hcfs/hcfs-workspace.sh finalize

# Check workspace status
/opt/hcfs/hcfs-workspace.sh status

🔄 Integration with Bzzz Agents

Updated Sandbox Creation

The Bzzz sandbox system now supports HCFS workspaces:

// Create HCFS-enabled sandbox
sandbox, err := CreateSandboxWithHCFS(ctx, taskImage, agentConfig, agentID, taskID)

// Check if using HCFS
if sandbox.IsUsingHCFS() {
    workspace := sandbox.GetHCFSWorkspace()
    fmt.Printf("Using HCFS workspace: %s\n", workspace.HCFSPath)
}

Configuration in Bzzz

Add HCFS configuration to your Bzzz agent config:

hcfs:
  enabled: true
  api_url: "http://localhost:8000"
  mount_path: "/tmp/hcfs-workspaces"
  store_artifacts: true
  idle_cleanup_interval: "15m"
  max_idle_time: "1h"

📊 Monitoring and Debugging

Service Health Checks

# Check HCFS API
curl http://localhost:8000/health

# Check RL Tuner
curl http://localhost:8001/health

# View container logs
docker-compose -f docker-compose.hcfs.yml logs -f hcfs-api

Workspace Status

# View workspace metadata
cat /home/agent/work/.hcfs-workspace

# Check sync daemon status
ps aux | grep hcfs-workspace

# View HCFS logs
tail -f /var/log/hcfs/workspace.log

🛠️ Development Workflows

Python ML Development

# Start Python environment
docker exec -it agent-python-dev bash

# Create new project
cd /home/agent/work
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Start Jupyter for data exploration
jupyter lab --ip=0.0.0.0 --port=8888

# Artifacts automatically collected:
# - *.ipynb notebooks
# - model files in models/
# - results in output/

Node.js Web Development

# Start Node.js environment
docker exec -it agent-nodejs-dev bash

# Initialize project
cd /home/agent/work
cp package.json.template package.json
npm install

# Start development server
npm run dev

# Artifacts automatically collected:
# - package*.json
# - build output in dist/
# - logs in logs/

Go Microservices

# Start Go environment
docker exec -it agent-go-dev bash

# Initialize project
cd /home/agent/work
cp go.mod.template go.mod
cp main.go.template main.go
go mod tidy

# Build and run
make build
make run

# Artifacts automatically collected:
# - go.mod, go.sum
# - binary in bin/
# - test results

🔒 Security Considerations

Container Security

• Agents run as non-root agent user
• Limited sudo access only for FUSE mounts
• Network restrictions block sensitive ports
• Read-only access to system directories

HCFS Security

• Context access controlled by agent roles
• Workspace isolation between agents
• Artifact encryption (optional)
• Audit logging of all operations

🔄 Backup and Recovery

Workspace Persistence

Agent workspaces are stored in named Docker volumes:

• python-workspace: Python development files
• nodejs-workspace: Node.js development files
• go-workspace: Go development files

HCFS Data

Core HCFS data is stored in:

• hcfs-data: Main context database
• hcfs-rl-data: RL Context Curator data

Backup Script

# Backup all workspace data
docker run --rm -v python-workspace:/data -v /backup:/backup alpine \
  tar czf /backup/python-workspace-$(date +%Y%m%d).tar.gz -C /data .

🐛 Troubleshooting

Common Issues

HCFS API Not Available:

# Check if HCFS container is running
docker ps | grep hcfs-api

# Check network connectivity
docker exec agent-python-dev curl -f http://hcfs-api:8000/health

FUSE Mount Failures:

# Check FUSE support
docker exec agent-python-dev ls -la /dev/fuse

# Check mount permissions
docker exec agent-python-dev mount | grep fuse

Workspace Sync Issues:

# Restart sync daemon
docker exec agent-python-dev pkill -f hcfs-workspace
docker exec agent-python-dev /opt/hcfs/hcfs-workspace.sh daemon &

# Manual sync
docker exec agent-python-dev /opt/hcfs/hcfs-workspace.sh sync

Log Locations

• HCFS API: docker logs hcfs-api
• Agent containers: docker logs agent-python-dev
• Workspace sync: /var/log/hcfs/workspace.log (inside container)

📚 Additional Resources

• HCFS Documentation
• Bzzz Agent Configuration
• RL Context Curator Guide
• Docker Compose Reference

🎯 Next Steps

1. Deploy to Production: Use Docker Swarm or Kubernetes
2. Scale Horizontally: Add more agent instances
3. Custom Images: Create domain-specific development environments
4. Monitoring: Add Prometheus/Grafana for metrics
5. CI/CD Integration: Automate testing and deployment