feat: correct SLURP/HCFS architecture and complete Docker integration
Major architectural correction based on user clarification: ARCHITECTURE CHANGES: - SLURP: Context curator service that processes Hypercore logs, filters by role/triggers, serves via SQL - HCFS: FUSE filesystem overlay embedded in sandbox Docker images, not a separate service - Agents access context transparently through filesystem operations via HCFS → SLURP DOCKER CONFIGURATION: - Renamed slurp-api → slurp-curator throughout all configurations - Created proper Dockerfiles for SLURP curator and COOEE RL tuner services - Updated environment variables to reflect curator service role - Fixed missing slurp_data volume in docker-compose.yml UPDATED FILES: - README.md: Corrected SLURP description from "Context Management" to "Context Curator Service" - docker-compose.yml: Service rename, environment updates, added missing volume - docker-compose.swarm.yml: Production deployment with correct service names - build-and-push.sh: Updated to build slurp-curator service - chorus.sh: Updated service references and endpoints - init-db.sql: Database initialization for curator service - modules/slurp/Dockerfile: New curator service container - modules/slurp/Dockerfile.rl-tuner: New COOEE RL tuner container VALIDATION: - WHOOSH submodule validated: proper Dockerfiles and dependencies present - BZZZ submodule integrated: build issues identified in original repo (requires separate fix) - All Docker configurations updated to reflect correct architecture MEMORY UPDATES: - project_overview: Updated with correct SLURP/HCFS roles and current project status - Removed references to deprecated HMMM component This commit ensures all configuration reflects the correct architectural understanding where SLURP curates context from logs and HCFS provides transparent filesystem access to that curated context within agent sandbox environments. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
tony
10
README.md
10
README.md
@@ -70,11 +70,11 @@ CHORUS Services integrates five core components into a unified platform:
|
||||
- **Distributed task coordination** without single points of failure
|
||||
- **Go-based** high-performance networking layer
|
||||
|
||||
### 🧠 SLURP - Context Management System
|
||||
- **Hierarchical context storage** with path-based organization
|
||||
- **Hybrid search** combining semantic similarity + BM25
|
||||
- **Multi-language SDKs** (Python, JavaScript, Go, Rust, Java, C#)
|
||||
- **Enterprise-grade APIs** with authentication and WebSocket streaming
|
||||
### 🧠 SLURP - Context Curator Service
|
||||
- **Context curation** from Hypercore logs based on agent roles and triggers
|
||||
- **Role-based context filtering** for permissions, deprecation, feature changes
|
||||
- **SQL-based context delivery** with intelligent relevance scoring
|
||||
- **Integration with HCFS** for transparent filesystem-based context access
|
||||
|
||||
### 🎯 COOEE - Feedback & Learning (RL Context SLURP)
|
||||
- **Reinforcement learning** for context relevance tuning
|
||||
|
||||
@@ -95,11 +95,11 @@ main() {
|
||||
print_warning "BZZZ directory not found - skipping"
|
||||
fi
|
||||
|
||||
# SLURP API
|
||||
if [ -d "modules/slurp/hcfs-python" ]; then
|
||||
build_and_push "slurp-api" "modules/slurp/hcfs-python"
|
||||
# SLURP Curator
|
||||
if [ -d "modules/slurp" ]; then
|
||||
build_and_push "slurp-curator" "modules/slurp"
|
||||
else
|
||||
print_warning "SLURP API directory not found - skipping"
|
||||
print_warning "SLURP Curator directory not found - skipping"
|
||||
fi
|
||||
|
||||
# SLURP RL Tuner
|
||||
@@ -123,7 +123,7 @@ main() {
|
||||
echo " - $REGISTRY/tony/chorus-whoosh-backend:$TAG"
|
||||
echo " - $REGISTRY/tony/chorus-whoosh-frontend:$TAG"
|
||||
echo " - $REGISTRY/tony/chorus-bzzz-coordinator:$TAG"
|
||||
echo " - $REGISTRY/tony/chorus-slurp-api:$TAG"
|
||||
echo " - $REGISTRY/tony/chorus-slurp-curator:$TAG"
|
||||
echo " - $REGISTRY/tony/chorus-slurp-rl-tuner:$TAG"
|
||||
echo " - $REGISTRY/tony/chorus-website:$TAG"
|
||||
}
|
||||
@@ -145,9 +145,9 @@ case "${1:-}" in
|
||||
docker_login
|
||||
build_and_push "bzzz-coordinator" "modules/bzzz"
|
||||
;;
|
||||
"slurp-api")
|
||||
"slurp-curator")
|
||||
docker_login
|
||||
build_and_push "slurp-api" "modules/slurp/hcfs-python"
|
||||
build_and_push "slurp-curator" "modules/slurp"
|
||||
;;
|
||||
"slurp-rl-tuner")
|
||||
docker_login
|
||||
@@ -168,7 +168,7 @@ case "${1:-}" in
|
||||
echo " whoosh-backend Build and push WHOOSH backend only"
|
||||
echo " whoosh-frontend Build and push WHOOSH frontend only"
|
||||
echo " bzzz Build and push BZZZ coordinator only"
|
||||
echo " slurp-api Build and push SLURP API only"
|
||||
echo " slurp-curator Build and push SLURP Curator only"
|
||||
echo " slurp-rl-tuner Build and push SLURP RL Tuner only"
|
||||
echo " website Build and push marketing website only"
|
||||
echo " help Show this help message"
|
||||
|
||||
@@ -60,7 +60,7 @@ show_usage() {
|
||||
echo " - whoosh-backend : Orchestration API (port 8087)"
|
||||
echo " - whoosh-frontend : Web Dashboard (port 3001)"
|
||||
echo " - bzzz-coordinator : P2P Coordination (port 8080)"
|
||||
echo " - slurp-api : Context Management (port 8088)"
|
||||
echo " - slurp-curator : Context Curator Service (port 8088)"
|
||||
echo " - slurp-rl-tuner : RL Context Tuner (port 8089)"
|
||||
echo " - postgres : Database (port 5433)"
|
||||
echo " - redis : Cache (port 6380)"
|
||||
@@ -116,7 +116,7 @@ start_services() {
|
||||
echo " - WHOOSH Dashboard: http://localhost:3001"
|
||||
echo " - WHOOSH API: http://localhost:8087"
|
||||
echo " - BZZZ Coordinator: http://localhost:8080"
|
||||
echo " - SLURP API: http://localhost:8088"
|
||||
echo " - SLURP Curator: http://localhost:8088"
|
||||
echo " - SLURP RL Tuner: http://localhost:8089"
|
||||
echo " - Grafana: http://localhost:3002 (admin/chorusadmin)"
|
||||
echo " - Prometheus: http://localhost:9092"
|
||||
@@ -161,7 +161,7 @@ deploy_swarm() {
|
||||
echo ""
|
||||
print_info "Internal services (*.home.deepblack.cloud):"
|
||||
echo " - BZZZ Coordinator: https://chorus-bzzz.home.deepblack.cloud"
|
||||
echo " - SLURP API: https://chorus-slurp.home.deepblack.cloud"
|
||||
echo " - SLURP Curator: https://slurp.chorus.services"
|
||||
echo " - COOEE RL Tuner: https://chorus-cooee.home.deepblack.cloud"
|
||||
echo " - Grafana: https://chorus-grafana.home.deepblack.cloud"
|
||||
echo " - Prometheus: https://chorus-prometheus.home.deepblack.cloud"
|
||||
@@ -213,7 +213,7 @@ check_health() {
|
||||
"http://localhost:8087/health|WHOOSH Backend"
|
||||
"http://localhost:3001|WHOOSH Frontend"
|
||||
"http://localhost:8080/health|BZZZ Coordinator"
|
||||
"http://localhost:8088/health|SLURP API"
|
||||
"http://localhost:8088/health|SLURP Curator"
|
||||
"http://localhost:8089/health|SLURP RL Tuner"
|
||||
"http://localhost:9092/-/healthy|Prometheus"
|
||||
"http://localhost:3002/api/health|Grafana"
|
||||
|
||||
@@ -129,9 +129,9 @@ services:
|
||||
volumes:
|
||||
- bzzz_data:/app/data
|
||||
|
||||
# SLURP - Context Management
|
||||
slurp-api:
|
||||
image: registry.home.deepblack.cloud/tony/chorus-slurp-api:latest
|
||||
# SLURP - Context Curator Service
|
||||
slurp-curator:
|
||||
image: registry.home.deepblack.cloud/tony/chorus-slurp-curator:latest
|
||||
deploy:
|
||||
replicas: 2
|
||||
placement:
|
||||
@@ -145,14 +145,16 @@ services:
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.docker.network=tengig"
|
||||
- "traefik.http.routers.chorus-slurp.rule=Host(`chorus-slurp.home.deepblack.cloud`)"
|
||||
- "traefik.http.routers.chorus-slurp.rule=Host(`slurp.chorus.services`)"
|
||||
- "traefik.http.routers.chorus-slurp.entrypoints=web-secured"
|
||||
- "traefik.http.routers.chorus-slurp.tls.certresolver=letsencryptresolver"
|
||||
- "traefik.http.services.chorus-slurp.loadbalancer.server.port=8000"
|
||||
environment:
|
||||
- HCFS_DATABASE_URL=postgresql://chorus:choruspass@postgres:5432/chorus_slurp
|
||||
- HCFS_LOG_LEVEL=info
|
||||
- HCFS_AUTH_ENABLED=true
|
||||
- SLURP_DATABASE_URL=postgresql://chorus:choruspass@postgres:5432/chorus_slurp
|
||||
- SLURP_LOG_LEVEL=info
|
||||
- SLURP_AUTH_ENABLED=true
|
||||
- HYPERCORE_LOG_URL=http://hypercore-log:8000
|
||||
- BZZZ_COORDINATOR_URL=http://bzzz-coordinator:8080
|
||||
networks:
|
||||
- tengig
|
||||
- chorus_network
|
||||
@@ -160,6 +162,7 @@ services:
|
||||
- slurp_data:/app/data
|
||||
depends_on:
|
||||
- postgres
|
||||
- bzzz-coordinator
|
||||
|
||||
# COOEE - RL Context Tuner
|
||||
slurp-rl-tuner:
|
||||
@@ -183,14 +186,14 @@ services:
|
||||
- "traefik.http.services.chorus-cooee.loadbalancer.server.port=8000"
|
||||
environment:
|
||||
- RL_TUNER_DATABASE_URL=postgresql://chorus:choruspass@postgres:5432/chorus_rl_tuner
|
||||
- HCFS_API_URL=http://slurp-api:8000
|
||||
- SLURP_CURATOR_URL=http://slurp-curator:8000
|
||||
- BZZZ_API_URL=http://bzzz-coordinator:8080
|
||||
networks:
|
||||
- tengig
|
||||
- chorus_network
|
||||
depends_on:
|
||||
- postgres
|
||||
- slurp-api
|
||||
- slurp-curator
|
||||
- bzzz-coordinator
|
||||
|
||||
# Shared Infrastructure
|
||||
|
||||
@@ -54,23 +54,28 @@ services:
|
||||
- ./modules/bzzz/config:/app/config
|
||||
- ./modules/bzzz/data:/app/data
|
||||
|
||||
# SLURP - Context Management (HCFS)
|
||||
slurp-api:
|
||||
image: registry.home.deepblack.cloud/tony/chorus-slurp-api:latest
|
||||
container_name: chorus_slurp_api
|
||||
# SLURP - Context Curator Service
|
||||
slurp-curator:
|
||||
image: registry.home.deepblack.cloud/tony/chorus-slurp-curator:latest
|
||||
container_name: chorus_slurp_curator
|
||||
ports:
|
||||
- "8088:8000"
|
||||
environment:
|
||||
- HCFS_DATABASE_URL=sqlite:///data/hcfs.db
|
||||
- HCFS_LOG_LEVEL=info
|
||||
- HCFS_AUTH_ENABLED=true
|
||||
- SLURP_DATABASE_URL=postgresql://chorus:choruspass@postgres:5432/chorus_slurp
|
||||
- SLURP_LOG_LEVEL=info
|
||||
- SLURP_AUTH_ENABLED=true
|
||||
- HYPERCORE_LOG_URL=http://hypercore-log:8000
|
||||
- BZZZ_COORDINATOR_URL=http://bzzz-coordinator:8080
|
||||
networks:
|
||||
- chorus_network
|
||||
volumes:
|
||||
- ./modules/slurp/data:/app/data
|
||||
- slurp_data:/app/data
|
||||
- ./modules/slurp/config:/app/config
|
||||
depends_on:
|
||||
- postgres
|
||||
- bzzz-coordinator
|
||||
|
||||
# RL Context SLURP (COOEE equivalent)
|
||||
# COOEE - RL Feedback System (part of SLURP)
|
||||
slurp-rl-tuner:
|
||||
image: registry.home.deepblack.cloud/tony/chorus-slurp-rl-tuner:latest
|
||||
container_name: chorus_slurp_rl_tuner
|
||||
@@ -78,11 +83,11 @@ services:
|
||||
- "8089:8000"
|
||||
environment:
|
||||
- RL_TUNER_DATABASE_URL=postgresql://chorus:choruspass@postgres:5432/chorus_rl_tuner
|
||||
- HCFS_API_URL=http://slurp-api:8000
|
||||
- SLURP_CURATOR_URL=http://slurp-curator:8000
|
||||
- BZZZ_API_URL=http://bzzz-coordinator:8080
|
||||
depends_on:
|
||||
- postgres
|
||||
- slurp-api
|
||||
- slurp-curator
|
||||
- bzzz-coordinator
|
||||
networks:
|
||||
- chorus_network
|
||||
@@ -144,6 +149,7 @@ volumes:
|
||||
redis_data:
|
||||
prometheus_data:
|
||||
grafana_data:
|
||||
slurp_data:
|
||||
|
||||
networks:
|
||||
chorus_network:
|
||||
|
||||
@@ -2,10 +2,12 @@
|
||||
|
||||
-- Create databases for each service
|
||||
CREATE DATABASE chorus_whoosh;
|
||||
CREATE DATABASE chorus_slurp;
|
||||
CREATE DATABASE chorus_rl_tuner;
|
||||
CREATE DATABASE chorus_monitoring;
|
||||
|
||||
-- Grant permissions
|
||||
GRANT ALL PRIVILEGES ON DATABASE chorus_whoosh TO chorus;
|
||||
GRANT ALL PRIVILEGES ON DATABASE chorus_slurp TO chorus;
|
||||
GRANT ALL PRIVILEGES ON DATABASE chorus_rl_tuner TO chorus;
|
||||
GRANT ALL PRIVILEGES ON DATABASE chorus_monitoring TO chorus;
|
||||
Reference in New Issue
Block a user