HCFS/hcfs-python/README.md

HCFS - Context-Aware Hierarchical Context File System

A virtual filesystem that maps hierarchical paths to context blobs, enabling AI agents to navigate and manage context at different scopes.

Features

• Virtual Filesystem: FUSE-based filesystem with context navigation
• Context Storage: SQLite-based storage with versioning and metadata
• Semantic Search: Embedding-based similarity search with hybrid BM25+semantic ranking
• REST API: FastAPI-based service for programmatic access
• CLI Tools: Command-line interface for context management

Installation

# Clone and install
cd hcfs-python
pip install -e .

# Development installation
pip install -e ".[dev]"

Quick Start

1. Initialize Database

hcfs init --db-path ./hcfs_context.db

2. Mount Filesystem

# Create mount point
mkdir /tmp/hcfs_mount

# Mount HCFS (runs in foreground)
hcfs mount -m /tmp/hcfs_mount -d ./hcfs_context.db -f

3. Use Virtual Files

# Navigate context scope
cd /tmp/hcfs_mount/projects/my_project/

# View current context
cat .context

# List contexts at current path  
cat .context_list

# Push new context
echo "This is context for my project" > .context_push

4. API Server

# Start API server
hcfs serve --db-path ./hcfs_context.db --port 8000

# API docs at http://localhost:8000/docs

5. CLI Operations

# Push context via CLI
hcfs push "/projects/my_project" "Project context content" --author "me"

# Get context with inheritance
hcfs get "/projects/my_project" --depth 2

# Search contexts
hcfs search "machine learning" --search-type hybrid --top-k 5

Architecture

Core Components

• ContextDatabase (hcfs.core.context_db): SQLite-based context storage
• HCFSFilesystem (hcfs.core.filesystem): FUSE filesystem implementation
• EmbeddingManager (hcfs.core.embeddings): Semantic search and embeddings
• ContextAPI (hcfs.api.server): REST API server

Virtual Files

Each directory in HCFS contains three virtual files:

• .context - Read aggregated context for current path + parents
• .context_list - List all contexts at current path
• .context_push - Write to this file to add context

API Endpoints

• GET /context/{path} - Get contexts with inheritance
• POST /context - Create new context
• PUT /context/{id} - Update context
• DELETE /context/{id} - Delete context
• POST /search - Semantic/hybrid search
• GET /similar/{id} - Find similar contexts

Development

Project Structure

hcfs-python/
├── hcfs/
│   ├── core/           # Core components
│   │   ├── context_db.py   # Database layer
│   │   ├── filesystem.py   # FUSE filesystem
│   │   └── embeddings.py   # ML/embeddings
│   ├── api/            # REST API
│   │   └── server.py
│   └── cli.py          # Command line interface
├── tests/              # Test suite
├── pyproject.toml      # Project configuration
└── README.md

Running Tests

pytest tests/

Code Formatting

black hcfs/
isort hcfs/

Research Context

This implements the Context-Aware Hierarchical Context File System as described in the project plan, combining:

• Semantic file systems (Gifford et al., 1991)
• LLM-based semantic filesystems (LSFS, ICLR 2025)
• Path-structure embeddings for hierarchical context
• Hybrid retrieval (BM25 + semantic embeddings)

The system enables AI agents to navigate context hierarchically while maintaining semantic relationships between context blobs.

License

MIT License