143 lines
3.6 KiB
Markdown
143 lines
3.6 KiB
Markdown
# HCFS - Context-Aware Hierarchical Context File System
|
|
|
|
A virtual filesystem that maps hierarchical paths to context blobs, enabling AI agents to navigate and manage context at different scopes.
|
|
|
|
## Features
|
|
|
|
- **Virtual Filesystem**: FUSE-based filesystem with context navigation
|
|
- **Context Storage**: SQLite-based storage with versioning and metadata
|
|
- **Semantic Search**: Embedding-based similarity search with hybrid BM25+semantic ranking
|
|
- **REST API**: FastAPI-based service for programmatic access
|
|
- **CLI Tools**: Command-line interface for context management
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
# Clone and install
|
|
cd hcfs-python
|
|
pip install -e .
|
|
|
|
# Development installation
|
|
pip install -e ".[dev]"
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
### 1. Initialize Database
|
|
```bash
|
|
hcfs init --db-path ./hcfs_context.db
|
|
```
|
|
|
|
### 2. Mount Filesystem
|
|
```bash
|
|
# Create mount point
|
|
mkdir /tmp/hcfs_mount
|
|
|
|
# Mount HCFS (runs in foreground)
|
|
hcfs mount -m /tmp/hcfs_mount -d ./hcfs_context.db -f
|
|
```
|
|
|
|
### 3. Use Virtual Files
|
|
```bash
|
|
# Navigate context scope
|
|
cd /tmp/hcfs_mount/projects/my_project/
|
|
|
|
# View current context
|
|
cat .context
|
|
|
|
# List contexts at current path
|
|
cat .context_list
|
|
|
|
# Push new context
|
|
echo "This is context for my project" > .context_push
|
|
```
|
|
|
|
### 4. API Server
|
|
```bash
|
|
# Start API server
|
|
hcfs serve --db-path ./hcfs_context.db --port 8000
|
|
|
|
# API docs at http://localhost:8000/docs
|
|
```
|
|
|
|
### 5. CLI Operations
|
|
```bash
|
|
# Push context via CLI
|
|
hcfs push "/projects/my_project" "Project context content" --author "me"
|
|
|
|
# Get context with inheritance
|
|
hcfs get "/projects/my_project" --depth 2
|
|
|
|
# Search contexts
|
|
hcfs search "machine learning" --search-type hybrid --top-k 5
|
|
```
|
|
|
|
## Architecture
|
|
|
|
### Core Components
|
|
|
|
- **ContextDatabase** (`hcfs.core.context_db`): SQLite-based context storage
|
|
- **HCFSFilesystem** (`hcfs.core.filesystem`): FUSE filesystem implementation
|
|
- **EmbeddingManager** (`hcfs.core.embeddings`): Semantic search and embeddings
|
|
- **ContextAPI** (`hcfs.api.server`): REST API server
|
|
|
|
### Virtual Files
|
|
|
|
Each directory in HCFS contains three virtual files:
|
|
|
|
- `.context` - Read aggregated context for current path + parents
|
|
- `.context_list` - List all contexts at current path
|
|
- `.context_push` - Write to this file to add context
|
|
|
|
### API Endpoints
|
|
|
|
- `GET /context/{path}` - Get contexts with inheritance
|
|
- `POST /context` - Create new context
|
|
- `PUT /context/{id}` - Update context
|
|
- `DELETE /context/{id}` - Delete context
|
|
- `POST /search` - Semantic/hybrid search
|
|
- `GET /similar/{id}` - Find similar contexts
|
|
|
|
## Development
|
|
|
|
### Project Structure
|
|
```
|
|
hcfs-python/
|
|
├── hcfs/
|
|
│ ├── core/ # Core components
|
|
│ │ ├── context_db.py # Database layer
|
|
│ │ ├── filesystem.py # FUSE filesystem
|
|
│ │ └── embeddings.py # ML/embeddings
|
|
│ ├── api/ # REST API
|
|
│ │ └── server.py
|
|
│ └── cli.py # Command line interface
|
|
├── tests/ # Test suite
|
|
├── pyproject.toml # Project configuration
|
|
└── README.md
|
|
```
|
|
|
|
### Running Tests
|
|
```bash
|
|
pytest tests/
|
|
```
|
|
|
|
### Code Formatting
|
|
```bash
|
|
black hcfs/
|
|
isort hcfs/
|
|
```
|
|
|
|
## Research Context
|
|
|
|
This implements the Context-Aware Hierarchical Context File System as described in the project plan, combining:
|
|
|
|
- **Semantic file systems** (Gifford et al., 1991)
|
|
- **LLM-based semantic filesystems** (LSFS, ICLR 2025)
|
|
- **Path-structure embeddings** for hierarchical context
|
|
- **Hybrid retrieval** (BM25 + semantic embeddings)
|
|
|
|
The system enables AI agents to navigate context hierarchically while maintaining semantic relationships between context blobs.
|
|
|
|
## License
|
|
|
|
MIT License |