Initial commit: Fresh implementation of CHORUS architecture (ResetData Mandate)
This commit is contained in:
anthonyrawlins
454
docs/Modules/BACKBEAT.md
Normal file
454
docs/Modules/BACKBEAT.md
Normal file
@@ -0,0 +1,454 @@
|
||||
# Backbeat Protocol — Pulse/Reverb (v0.1)
|
||||
|
||||
> **Purpose:** Give CHORUS a shared, lightweight rhythm so multi‑agent, p2p work doesn’t deadlock or drift. Standardise expectations (plan/work/review), exit conditions, promises, and time‑bounded collaboration across CHORUS, HMMM, SLURP, SHHH, UCXL, WHOOSH, and COOEE.
|
||||
|
||||
---
|
||||
|
||||
## 1) Rationale
|
||||
|
||||
- **Problem:** In pub/sub meshes, agents can wait indefinitely for help/context; there’s no universal cadence for planning, execution, or re‑evaluation.
|
||||
|
||||
- **Principle:** Use **coarse, explicit tempo** (beats/bars) for policy alignment; not for hard real‑time sync. Must be **partition‑tolerant**, **observable**, and **cheap**.
|
||||
|
||||
- **Design:** Human‑readable **beats/bars/phrases** for policy, **Hybrid Logical Clocks (HLC)** for mergeable ordering.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 2) Core Concepts
|
||||
|
||||
- **Tempo (BPM):** Beats per minute (e.g., 6–30 BPM). Cluster‑level default; task classes may suggest hints.
|
||||
|
||||
- **Beat:** Base epoch (e.g., 4 s @ 15 BPM).
|
||||
|
||||
- **Bar:** Group of beats (e.g., 8). **Downbeat** (beat 1) is a soft barrier (checkpoints, secret rotation).
|
||||
|
||||
- **Phrase:** A sequence of bars that maps to a work cycle: **plan → work → review**.
|
||||
|
||||
- **Score (per task):** Declarative allocation of beats across phases + wait budgets + retries.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 3) Roles & Components
|
||||
|
||||
- **Pulse:** Cluster tempo broadcaster. Publishes `BeatFrame` each beat; single elected leader (Raft/etcd), followers can degrade to local.
|
||||
|
||||
- **Reverb:** Aggregator/rollup. Ingests `StatusClaim`s and emits per‑bar `BarReport`s, plus hints for adaptive tempo.
|
||||
|
||||
- **Agents (CHORUS workers, HMMM collaborators, SLURP, etc.):** Consume beats, enforce **Score**, publish `StatusClaim`s.
|
||||
|
||||
- **SHHH:** Rotates short‑lived secrets **on downbeats** (per‑bar keys).
|
||||
|
||||
- **COOEE/DHT:** Transport for topics `backbeat://cluster/{id}` and per‑project status lanes.
|
||||
|
||||
|
||||
### Implementation Snapshot (2025-10)
|
||||
|
||||
- **Pulse service (`cmd/pulse`)** – Encapsulates Raft leader election (`internal/backbeat/leader.go`), Hybrid Logical Clock maintenance (`internal/backbeat/hlc.go`), degradation control (`internal/backbeat/degradation.go`), and beat publishing over NATS. It also exposes an admin HTTP surface and collects tempo/drift metrics via `internal/backbeat/metrics.go`.
|
||||
- **Reverb service (`cmd/reverb`)** – Subscribes to pulse beats and agent status subjects, aggregates `StatusClaim`s into rolling windows, and emits `BarReport`s on downbeats. Readiness, health, and Prometheus endpoints report claim throughput, aggregation latency, and NATs failures.
|
||||
- **Go SDK (`pkg/sdk`)** – Provides clients for beat callbacks, status emission, and health reporting with retry/circuit breaker hooks. CHORUS (`project-queues/active/CHORUS/internal/backbeat/integration.go`) and WHOOSH (`project-queues/active/WHOOSH/internal/backbeat/integration.go`) embed the SDK to align runtime operations with cluster tempo.
|
||||
- **Inter-module telemetry** – CHORUS maps P2P lifecycle operations (elections, DHT bootstrap, council delivery) into BACKBEAT status claims, while WHOOSH emits search/composer activity. This keeps Reverb windows authoritative for council health and informs SLURP/BUBBLE provenance.
|
||||
- **Observability bundle** – Monitoring assets (`monitoring/`, `prometheus.yml`) plus service metrics export drift, tempo adjustments, Raft state, and window KPIs, meeting BACKBEAT-PER-001/002/003 targets and enabling WHOOSH scaling gates to react to rhythm degradation.
|
||||
|
||||
---
|
||||
|
||||
## 4) Wire Model
|
||||
|
||||
### 4.1 BeatFrame (Pulse → all)
|
||||
|
||||
```json
|
||||
{
|
||||
"cluster_id": "chorus-aus-01",
|
||||
"tempo_bpm": 15,
|
||||
"beat_ms": 4000,
|
||||
"bar_len_beats": 8,
|
||||
"bar": 1287,
|
||||
"beat": 3,
|
||||
"phase": "work",
|
||||
"hlc": "2025-09-03T02:12:27.183Z+1287:3+17",
|
||||
"policy_hash": "sha256:...",
|
||||
"deadline_at": "2025-09-03T02:12:31.183Z"
|
||||
}
|
||||
```
|
||||
|
||||
### 4.2 StatusClaim (agents → Reverb)
|
||||
|
||||
```json
|
||||
{
|
||||
"agent_id": "chorus-192-168-1-27",
|
||||
"task_id": "ucxl://...",
|
||||
"bar": 1287,
|
||||
"beat": 3,
|
||||
"state": "planning|executing|waiting|review|done|failed",
|
||||
"wait_for": ["hmmm://thread/abc"],
|
||||
"beats_left": 2,
|
||||
"progress": 0.42,
|
||||
"notes": "awaiting summarised artifact from peer",
|
||||
"hlc": "..."
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 HelpPromise (HMMM → requester)
|
||||
|
||||
```json
|
||||
{
|
||||
"thread_id": "hmmm://thread/abc",
|
||||
"promise_beats": 2,
|
||||
"confidence": 0.7,
|
||||
"fail_after_beats": 3,
|
||||
"on_fail": "fallback-plan-A"
|
||||
}
|
||||
```
|
||||
|
||||
### 4.4 BarReport (Reverb → observability)
|
||||
|
||||
- Per‑bar rollup: task counts by state, overruns, broken promises, queue depth, utilisation hints, suggested tempo/phase tweak.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 5) Score Spec (YAML)
|
||||
|
||||
```yaml
|
||||
score:
|
||||
tempo: 15 # bpm hint; cluster policy can override
|
||||
bar_len: 8 # beats per bar
|
||||
phases:
|
||||
plan: 2 # beats
|
||||
work: 4
|
||||
review: 2
|
||||
wait_budget:
|
||||
help: 2 # max beats to wait for HMMM replies across the phrase
|
||||
io: 1 # max beats to wait for I/O
|
||||
retry:
|
||||
max_phrases: 2
|
||||
backoff: geometric # plan/work/review shrink each retry
|
||||
escalation:
|
||||
on_wait_exhausted: ["emit:needs-attention", "fallback:coarse-answer"]
|
||||
on_overrun: ["checkpoint", "defer:next-phrase"]
|
||||
```
|
||||
|
||||
> **Rule:** Agents must not exceed phase beat allocations. If `help` budget is exhausted, **exit cleanly** with degraded but auditable output.
|
||||
|
||||
---
|
||||
|
||||
## 6) Agent Loop (sketch)
|
||||
|
||||
```text
|
||||
on BeatFrame(bf):
|
||||
if new bar and beat==1: rotate_ephemeral_keys(); checkpoint();
|
||||
|
||||
phase = score.phase_for(bf.beat)
|
||||
|
||||
switch phase:
|
||||
PLAN:
|
||||
if not planned: do_planning_until(phase_end)
|
||||
WORK:
|
||||
if need_help and !help_promised: request_help_with_promise()
|
||||
if waiting_for_help:
|
||||
if wait_beats > score.wait_budget.help: exit_with_fallback()
|
||||
else continue_work_on_alternative_path()
|
||||
else do_work_step()
|
||||
REVIEW:
|
||||
run_tests_and_summarise(); publish StatusClaim(state=done|failed)
|
||||
|
||||
enforce_cutoffs_at_phase_boundaries()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7) Adaptive Tempo Controller (ATC)
|
||||
|
||||
- **Inputs:** Queue depth per role, GPU/CPU util (WHOOSH), overrun frequency, broken promises.
|
||||
|
||||
- **Policy:** Adjust `tempo_bpm` and/or redistribute phase beats **between bars only** (PI‑style control, hysteresis ±10%).
|
||||
|
||||
- **Guardrails:** ≤1 beat change per minute; freeze during incidents.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 8) Exit Conditions & Deadlock Prevention
|
||||
|
||||
- **Wait budgets** are hard ceilings. Missing `HelpPromise` by end‑of‑bar triggers `on_wait_exhausted`.
|
||||
|
||||
- **Locks & leases** expire at bar boundaries unless renewed with `beats_left`.
|
||||
|
||||
- **Promises** include `promise_beats` and `fail_after_beats` so callers can plan.
|
||||
|
||||
- **Idempotent checkpoints** at downbeats enable safe retries/resumptions.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 9) Integration Points
|
||||
|
||||
- **CHORUS (workers):** Consume `BeatFrame`; enforce `Score`; publish `StatusClaim` each beat/change.
|
||||
|
||||
- **HMMM (collab):** Replies carry `HelpPromise`; threads auto‑close if `fail_after_beats` elapses.
|
||||
|
||||
- **SLURP (curation):** Batch ingest windows tied to review beats; produce bar‑stamped artefacts.
|
||||
|
||||
- **SHHH (secrets):** Rotate per bar; credentials scoped to `<cluster,bar>`.
|
||||
|
||||
- **UCXL:** Attach tempo metadata to deliverables: `{bar, beat, hlc}`; optional address suffix `;bar=1287#beat=8`.
|
||||
|
||||
- **WHOOSH:** Expose utilisation to ATC; enforce resource leases in beat units.
|
||||
|
||||
- **COOEE/DHT:** Topics: `backbeat://cluster/{id}`, `status://{project}`, `promise://hmmm`.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 10) Failure Modes & Degraded Operation
|
||||
|
||||
- **No Pulse leader:** Agents derive a **median‑of‑pulses** from available Purses; if none, use local monotonic clock (jitter ok) and **freeze tempo changes**.
|
||||
|
||||
- **Partitions:** Keep counting beats locally (HLC ensures mergeable order). Reverb reconciles by HLC and bar on heal.
|
||||
|
||||
- **Drift:** Tempo changes only on downbeats; publish `policy_hash` so agents detect misconfig.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 11) Config Examples
|
||||
|
||||
### 11.1 Cluster Tempo Policy
|
||||
|
||||
```yaml
|
||||
cluster_id: chorus-aus-01
|
||||
initial_bpm: 12
|
||||
bar_len_beats: 8
|
||||
phases: [plan, work, review]
|
||||
limits:
|
||||
max_bpm: 24
|
||||
min_bpm: 6
|
||||
adaptation:
|
||||
enable: true
|
||||
hysteresis_pct: 10
|
||||
change_per_minute: 1_beat
|
||||
observability:
|
||||
emit_bar_reports: true
|
||||
```
|
||||
|
||||
### 11.2 Task Score (attached to UCXL deliverable)
|
||||
|
||||
```yaml
|
||||
ucxl: ucxl://proj:any/*/task/graph_ingest
|
||||
score:
|
||||
tempo: 15
|
||||
bar_len: 8
|
||||
phases: {plan: 2, work: 4, review: 2}
|
||||
wait_budget: {help: 2, io: 1}
|
||||
retry: {max_phrases: 2, backoff: geometric}
|
||||
escalation:
|
||||
on_wait_exhausted: ["emit:needs-attention", "fallback:coarse-answer"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12) Observability
|
||||
|
||||
- **Per‑bar dashboards:** state counts, overruns, broken promises, tempo changes, queue depth, utilisation.
|
||||
|
||||
- **Trace stamps:** Every artifact/event carries `{bar, beat, hlc}` for forensic replay.
|
||||
|
||||
- **Alarms:** `promise_miss_rate`, `overrun_rate`, `no_status_claims`.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 13) Security
|
||||
|
||||
- Rotate **ephemeral keys on downbeats**; scope to project/role when possible.
|
||||
|
||||
- Bar‑stamped tokens reduce blast radius; revoke at bar+N.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 14) Economics & Budgeting — Beats as Unit of Cost
|
||||
|
||||
### 14.1 Beat Unit (BU)
|
||||
|
||||
- **Definition:** 1 BU = one cluster beat interval (`beat_ms`). It’s the atomic scheduling & accounting quantum.
|
||||
|
||||
|
||||
### 14.2 Resource Primitives (WHOOSH‑measured)
|
||||
|
||||
- `cpu_sec`, `gpu_sec[class]`, `accel_sec[class]`, `mem_gbs` (GB·s), `disk_io_mb`, `net_egress_mb`, `storage_gbh`.
|
||||
|
||||
|
||||
### 14.3 Budget & Costing
|
||||
|
||||
```yaml
|
||||
budget:
|
||||
max_bu: N_total
|
||||
phase_caps: { plan: Np, work: Nw, review: Nr }
|
||||
wait_caps: { help: Nh, io: Ni }
|
||||
hard_end: bar+K
|
||||
charge_to: ucxl://acct/...
|
||||
```
|
||||
|
||||
Cost per phrase:
|
||||
|
||||
```
|
||||
Total = Σ(beats_used * role_rate_bu)
|
||||
+ Σ_class(gpu_sec[class] * rate_gpu_sec[class])
|
||||
+ cpu_sec*rate_cpu_sec + mem_gbs*rate_mem_gbs
|
||||
+ disk_io_mb*rate_io_mb + net_egress_mb*rate_egress_mb
|
||||
+ storage_gbh*rate_storage_gbh
|
||||
```
|
||||
|
||||
### 14.4 KPIs
|
||||
|
||||
- **TNT** (tempo‑normalised throughput), **BPD** (beats per deliverable), **WR** (wait ratio), **η** (efficiency), **PMR** (promise miss rate), **CPD** (cost per deliverable), **TTFU** (time to first useful).
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 15) Tokenless Accounting (Hybrid CPU/GPU, On‑prem + Cloud)
|
||||
|
||||
- **No tokens.** Price **beats + measured resources**; ignore model‑token counts.
|
||||
|
||||
- **Device classes:** price per GPU/accelerator class (A100, 4090, MI300X, TPU…).
|
||||
|
||||
- **Rates:** on‑prem from TCO / duty‑cycle seconds; cloud from per‑second list prices. Bind via config.
|
||||
|
||||
- **Beat‑scoped caps:** per‑BU ceilings on resource primitives to contain spend regardless of hardware skew.
|
||||
|
||||
- **Calibration (planning‑only):** per‑family normalisers if you want **Effective Compute Units** for planning; **billing remains raw seconds**.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 16) MVP Bring‑up Plan
|
||||
|
||||
1. **Pulse**: static BPM, broadcast `BeatFrame` over COOEE.
|
||||
|
||||
2. **Agents**: publish `StatusClaim`; enforce `wait_budget` & `HelpPromise`.
|
||||
|
||||
3. **Reverb**: roll up to `BarReport`; surface early KPIs.
|
||||
|
||||
4. **SHHH**: rotate credentials on downbeats.
|
||||
|
||||
5. **ATC**: enable adaptation after telemetry.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 17) Open Questions
|
||||
|
||||
- Per‑role tempi vs one cluster tempo?
|
||||
|
||||
- Fixed `bar_len` vs dynamic redistribution of phase beats?
|
||||
|
||||
- Score UI: YAML + visual “score sheet” editor?
|
||||
|
||||
|
||||
---
|
||||
|
||||
### Naming (on brand)
|
||||
|
||||
- **Backbeat Protocol** — **Pulse** (broadcaster) + **Reverb** (rollup & reports). Musical, expressive; conveys ripples from each downbeat.
|
||||
|
||||
|
||||
# Backbeat — Relative Beats Addendum (UCXL ^^/~~)
|
||||
|
||||
**Why this addendum?** We’re removing dependence on ever‑increasing `bar`/`beat` counters. All coordination is expressed **relative to NOW** in **beats**, aligned with UCXL temporal markers `^^` (future) and `~~` (past).
|
||||
|
||||
## A) Wire Model Adjustments
|
||||
|
||||
### BeatFrame (Pulse → all)
|
||||
|
||||
**Replace** prior fields `{bar, beat}` with:
|
||||
|
||||
```json
|
||||
{
|
||||
"cluster_id": "...",
|
||||
"tempo_bpm": 15,
|
||||
"beat_ms": 4000,
|
||||
"bar_len_beats": 8,
|
||||
"beat_index": 3, // 1..bar_len_beats (cyclic within bar)
|
||||
"beat_epoch": "2025-09-03T02:12:27.000Z", // start time of this beat
|
||||
"downbeat": false, // true when beat_index==1
|
||||
"phase": "work",
|
||||
"hlc": "2025-09-03T02:12:27.183Z+17",
|
||||
"policy_hash": "sha256:...",
|
||||
"deadline_at": "2025-09-03T02:12:31.183Z"
|
||||
}
|
||||
```
|
||||
|
||||
### StatusClaim (agents → Reverb)
|
||||
|
||||
**Replace** prior fields `{bar, beat}` with:
|
||||
|
||||
```json
|
||||
{
|
||||
"agent_id": "...",
|
||||
"task_id": "...",
|
||||
"beat_index": 3,
|
||||
"state": "planning|executing|waiting|review|done|failed",
|
||||
"beats_left": 2,
|
||||
"progress": 0.42,
|
||||
"notes": "...",
|
||||
"hlc": "..."
|
||||
}
|
||||
```
|
||||
|
||||
### Bar/Window Aggregation
|
||||
|
||||
- Reverb aggregates per **window** bounded by `downbeat=true` frames.
|
||||
|
||||
- **No global bar counters** are transmitted. Observability UIs may keep a local `window_id` for navigation.
|
||||
|
||||
|
||||
## B) UCXL Temporal Suffix - (Requires RFC-UCXL 1.1)
|
||||
|
||||
Attach **relative beat** navigation to any UCXL address:
|
||||
|
||||
- `;beats=^^N` → target **N beats in the future** from now
|
||||
|
||||
- `;beats=~~N` → target **N beats in the past** from now
|
||||
|
||||
- Optional: `;phase=plan|work|review`
|
||||
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
ucxl://proj:any/*/task/ingest;beats=^^4;phase=work
|
||||
```
|
||||
|
||||
## C) Policy & Promises
|
||||
|
||||
- All time budgets are **Δbeats**: `wait_budget.help`, `retry.max_phrases`, `promise_beats`, `fail_after_beats`.
|
||||
|
||||
- **Leases/locks** renew per beat and expire on phase change unless renewed.
|
||||
|
||||
|
||||
## D) Derivations
|
||||
|
||||
- `beat_index = 1 + floor( (unix_ms / beat_ms) mod bar_len_beats )` (derived locally).
|
||||
|
||||
- `beat_epoch = floor_to_multiple(now, beat_ms)`.
|
||||
|
||||
- `Δbeats(target_time) = round( (target_time - now) / beat_ms )`.
|
||||
|
||||
|
||||
## E) Compatibility Notes
|
||||
|
||||
- Old fields `{bar, beat}` are **deprecated**; if received, they can be ignored or mapped to local windows.
|
||||
|
||||
- HLC remains the canonical merge key for causality.
|
||||
|
||||
|
||||
## F) Action Items
|
||||
|
||||
1. Update the **spec wire model** sections accordingly.
|
||||
|
||||
2. Regenerate the **Go prototype** using `BeatIndex/BeatEpoch/Downbeat` instead of `Bar/Beat` counters.
|
||||
|
||||
3. Add UCXL parsing for `;beats=^^/~~` in RUSTLE.
|
||||
- [ ] TODO: RUSTLE update for BACKBEAT compatibility
|
||||
38
docs/Modules/BUBBLE.md
Normal file
38
docs/Modules/BUBBLE.md
Normal file
@@ -0,0 +1,38 @@
|
||||
BUBBLE is an organization that functions as a hierarchical index augmented with semantic and provenance overlays. Its primary purpose is to extract historical authority provenance, decision records, citations, and decision ancestry. These capabilities support complex queries, semantic proximity navigation, citation tracking for decisions, and namespace navigation. BUBBLE enables fast, academic-style citation tracking, semantic navigation, and provenance verification, making it a crucial component for managing decision history and authority within the system.
|
||||
|
||||
# Decision Records as first-class artifacts
|
||||
|
||||
**DRs everywhere.** Every accepted/rejected option becomes an immutable node with edges like _influenced-by_, _derived-from_, and _depends-on-constraint-X_. This enables academic-style citation tracking, semantic proximity hops, and classic namespace traversal in one surface.
|
||||
|
||||
# Audit in minutes, not weeks
|
||||
|
||||
Because BUBBLE indexes against UCXL addresses, you can reconstruct _exactly_ what a role could see at time **t** and why a path was chosen — no folklore archaeology.
|
||||
|
||||
**References:**
|
||||
|
||||
1. [KG] Entities (KG): BUBBLE (file: MASTER PLAN.md)
|
||||
2. [DC] Document Chunks: "Machine-readable metadata for provenance & confidence." (file: 1)
|
||||
3. [KG] Relationships (KG): BUBBLE enables citation tracking for Decision Records (file: MASTER PLAN.md)
|
||||
4. [KG] Relationships (KG): BUBBLE's provenance and citation tracking support SLURP's decision processes (file: MASTER PLAN.md)
|
||||
5. [KG] Relationships (KG): BUBBLE extracts and manages Decision Records with provenance graphs (file: MASTER PLAN.md)
|
||||
|
||||
## How does UCXL work with BUBBLE?
|
||||
|
||||
UCXL and BUBBLE are interconnected within the broader ecosystem of semantic context management and decision provenance. UCXL functions primarily as a protocol architecture designed for addressing, managing, and sharing semantic contextual resources. It creates a 1:1 mapping between traditional file system structures and contextual metadata, enabling distributed AI development, knowledge preservation, and sophisticated context organization. UCXL supports features such as semantic address parsing, temporal navigation, context storage, and content retrieval, functioning as a foundational framework for maintaining and querying knowledge with temporal and semantic precision [KG1].
|
||||
|
||||
BUBBLE, on the other hand, acts as a hierarchical index and provenance overlay system. It enables extraction of historical authority provenance, decision records, citations, and decision ancestry. BUBBLE supports semantic and provenance overlays, citation tracking, complex queries, and namespace navigation, particularly over decision data and authority provenance. It works in conjunction with systems like SLURP and facilitates fast citation tracking and semantic proximity navigation, enriching the decision-making and knowledge validation process [DC8].
|
||||
|
||||
The correlation between UCXL and BUBBLE is thus centered on BUBBLE leveraging UCXL's protocol and addressing capabilities to support detailed provenance, decision tracking, and semantic overlay functionalities. BUBBLE’s ability to extract, index, and query decision records and provenance graphs fundamentally depends on UCXL’s resource addressing, contextual metadata management, and semantic addressing features. Together, they form a cohesive system for knowledge preservation, decision validation, and context-aware reasoning within the distributed AI environment.
|
||||
|
||||
**References:**
|
||||
|
||||
[KG1] UCXL: Entities and Description, in KG file
|
||||
[DC8] BUBBLE: Entity description and functions, in Document Chunks
|
||||
|
||||
## TODO
|
||||
|
||||
- Complete RocksDB storage implementation and promote to production default (replace SQLite).
|
||||
- Integrate vector search (e.g., FAISS/Milvus) for similarity scoring in WalkBack.
|
||||
- Implement DHT → local synchronization/replication for up-to-date decision graphs.
|
||||
- Add caching layer (LRU or Redis) for frequent bundles and document cache invalidation.
|
||||
- Optional: LLM-based summaries for `DecisionBundleResponse` with configuration and tests.
|
||||
115
docs/Modules/CHORUS-AGENTS.md
Normal file
115
docs/Modules/CHORUS-AGENTS.md
Normal file
@@ -0,0 +1,115 @@
|
||||
## What is CHORUS?
|
||||
|
||||
CHORUS is a comprehensive, distributed platform primarily designed for AI coordination and communication within its ecosystem. It functions as a semantic context publishing system that enables autonomous AI agents to securely share decisions, coordinate activities, and maintain consistent contexts through role-based encryption and consensus mechanisms. The platform supports real-time, selective sharing of both live and historical contextual data, facilitating operations such as task management, artifact publishing, and provenance tracking.
|
||||
|
||||
Originally established as a task coordination system using the `bzzz://` protocol, CHORUS is evolving into a semantic context publishing platform that enhances AI collaboration by transforming task coordination into a distributed decision graph based on the UCXL protocol. It also addresses operational aspects like data storage, configuration management, and security, including license management and threat protection. Overall, CHORUS aims to enable decentralized AI workflows, autonomous agent collaboration, and robust information sharing across distributed systems.
|
||||
|
||||
**Context OS responsibilities:** CHORUS orchestrates _selective_, time-aware context exchange between peers (no repo mirroring), carrying machine-readable provenance and confidence. In effect, it acts as the **runtime** for UCXL-addressed knowledge and policy-aware routing.
|
||||
|
||||
### References:
|
||||
|
||||
- [KG] Entity 1 (CHORUS) describes its role as a distributed AI coordination and semantic context publishing platform.
|
||||
- [DC] "Phase 1 Integration Test Framework" document emphasizes its purpose in enabling secure, real-time sharing of decisions and contextual data.
|
||||
- [KG] Its support for autonomous agents and role-based encryption highlights its focus on secure, decentralized AI collaboration.
|
||||
- [KG] The transformation from a task coordination system to a semantic platform indicates its goal of enhancing operational synchronization.
|
||||
- [DC] User Manual details the system's functions in managing AI decision-sharing and collaboration.
|
||||
|
||||
## Current Implementation Snapshot (2025-10)
|
||||
|
||||
- **WHOOSH-assignment runtime** – `internal/runtime/shared.go` now bootstraps a `pkg/config.RuntimeConfig` that merges base env vars with dynamic assignments pulled from WHOOSH (`ASSIGN_URL`). Reloads are wired to `SIGHUP`, and WHOOSH-provided bootstrap peers or join staggers override baked-in defaults. See [[docs/Modules/WHOOSH.md]] for the rendezvous API that serves these payloads.
|
||||
- **License-first startup & AI provider wiring** – Startup blocks on `internal/licensing` validation against KACHING before any P2P work. Once licensed, `initializeAIProvider` configures ResetData or Ollama providers, composes persona prompts, and enables LightRAG via the MCP client when `LightRAG.Enabled` is set. This keeps reasoning, prompt curation, and RAG feeds consistent across containers.
|
||||
- **Tempo-aware operations via BACKBEAT** – The BACKBEAT integration (`internal/backbeat/integration.go`) tracks beat cadence, wraps long-running peer operations (DHT bootstrap, peer discovery, elections), and emits health/status claims. Election scoring (`pkg/election/election.go`) consumes tempo and beat-gap data to throttle discovery storms and to respect stability windows.
|
||||
- **Instrumented transport stack** – `pkg/transport/quic_transport.go` layers QUIC + optional TCP fallback with pooled streams, dial metrics, and configurable libp2p options. A transport telemetry reporter (`internal/runtime/transport_telemetry.go`) publishes snapshots to NATS (`CHORUS_TRANSPORT_METRICS_NATS_URL`) and surfaces per-transport counters through `pkg/metrics`.
|
||||
- **Encrypted context pipeline** – When `V2.DHT.Enabled` is true, CHORUS spins up a libp2p DHT with AGE-backed storage (`pkg/dht`) and routes decision artifacts through `ucxl.DecisionPublisher`, exposing them via the optional UCXI HTTP server. Council opportunities are bridged over QUIC streams and mirrored onto NATS to keep WHOOSH/SLURP in sync.
|
||||
|
||||
**Operational Notes**
|
||||
|
||||
- **Centralised QUIC option injection** – Noise security, muxers, and relay support are now exclusively configured inside `transport.NewQUICTransport`, preventing the duplicate-provider panic that surfaced when both the transport and caller registered `libp2p.Security(noise)`. Custom libp2p options should be appended via runtime overrides to avoid reintroducing duplicates.
|
||||
|
||||
## How does CHORUS establish a peer-to-peer network?
|
||||
|
||||
CHORUS connects to itself to form a peer-to-peer (P2P) network through its native libp2p-based infrastructure, which manages peer discovery, connectivity, and message routing. The system employs protocols such as pubsub for messaging and mDNS or DHT for peer discovery, enabling nodes to automatically recognize and establish connections with each other without manual configuration.
|
||||
|
||||
Specifically, the architecture supports direct peer connections where nodes connect directly with each other, forming a full mesh or ring topology depending on deployment. The network topology is managed via mechanisms like bootstrap peers, which are static nodes used to initialize and support connections within the network. Nodes exchange capability broadcasts periodically to maintain an updated view of the network state.
|
||||
|
||||
Security in the connection process is enhanced through cryptographic verification methods like signature validation and signature verification to ensure trustworthiness among peers. Additionally, peer discovery protocols like mDNS (multicast DNS) facilitate neighborhood detection within local networks, while content and capabilities are exchanged securely. The infrastructure includes features for network resilience such as circuit breakers, rate limiting, and network health monitoring.
|
||||
|
||||
Overall, CHORUS forms its P2P network by leveraging libp2p’s suite of protocols for secure, decentralized peer discovery and connection management, which allows nodes within the network to connect directly and participate effectively in distributed AI coordination activities.
|
||||
|
||||
**Institutional stance:** libp2p + scoped sync means context sharing is intentional, logged, and revocable; bootstrap + capability broadcasts make the mesh resilient while preserving least-privilege flow.
|
||||
|
||||
References:
|
||||
[KG] "CHORUS P2P Network": Describes the decentralized communication layer involving peer discovery, pubsub messaging, and mutual peer management.
|
||||
[DC] "README.md": Details on network discovery, peer connections, and mesh network topology.
|
||||
[KG] "libp2p": Mentioned as the core protocol facilitating mesh networking, peer discovery via mDNS, and secure connections.
|
||||
[KG] "CHORUS P2P Mesh": Organization that supports distributed peer connections for chat and AI collaboration.
|
||||
[DC] "CHORUS-ARCHITECTURE.md": Explains the architecture including bootstrap peers and full mesh topology.
|
||||
### 1.1.2.1 Key Functions
|
||||
|
||||
- Decentralized task coordination and management
|
||||
- Secure, role-based sharing of contextual data
|
||||
- Autonomous AI agent collaboration
|
||||
- Content publishing of decisions and task updates
|
||||
- Distributed data storage and retrieval
|
||||
- Role-based encryption and security
|
||||
- System configuration management
|
||||
- License enforcement and resilience
|
||||
- System deployment and maintenance
|
||||
|
||||
### 1.1.2.2 Modules of CHORUS
|
||||
|
||||
- [[BUBBLE]]
|
||||
- [[SLURP]]
|
||||
- [[COOEE]]
|
||||
- [[SHHH]]
|
||||
- [[HMMM]]
|
||||
- [[UCXL]]
|
||||
- [[DHT]]
|
||||
|
||||
CHORUS employs a range of components and modules that form its comprehensive architecture. The core components include the Main Application (main.go), decision-related modules such as the Decision Publisher, Election Manager, and Configuration Manager, and infrastructure elements like the Crypto Module (supporting Age encryption and Shamir secret sharing), Distributed Hash Table (DHT) Storage, and the Peer-to-Peer (P2P) Network for peer discovery and pub/sub messaging.
|
||||
|
||||
Additionally, CHORUS features specialized modules such as the UCXL protocol for semantic address management, SLURP for context management and decision reasoning, and a set of components dedicated to content publishing, security, and operational diagnostics. It also includes components responsible for the layered architecture, such as API handling, web sockets, and management tools.
|
||||
|
||||
For hybrid or flexible deployment, there are mock components for extensive testing and real components intended for production use. These include mock and real implementations of the DHT backend, address resolution, peer discovery, network layer, and connectors (such as the CHORUS Connector and RUSTLE Hybrid Components). The architecture is designed to support role-based security, distributed storage, consensus elections, and semantic addressing to facilitate decentralized AI coordination.
|
||||
|
||||
References:
|
||||
|
||||
[KG] Entity "CHORUS" description in the JSON graph.
|
||||
[DC] "CHORUS System Architecture" details in "CHORUS-2B-ARCHITECTURE.md".
|
||||
[DC] Components listed under "System Components" in "CHORUS_N8N_IMPLEMENTATION_COMPLETE.md".
|
||||
[DC] Architectural diagrams in "CHORUS Architecture Documentation".
|
||||
|
||||
|
||||
## What models does CHORUS use?
|
||||
|
||||
The CHORUS platform supports various models for its AI agents, depending on their roles and tasks. Specifically, the system includes features such as model integration within role configurations and capabilities for autonomous AI agents. One known provider mentioned is Ollama, which supplies models used for meta-thinking and performance metrics within the system, indicating that Ollama models are utilized to support agents in reasoning and performance evaluation.
|
||||
|
||||
CHORUS can execute tasks using its built-in smolLM3 model from HuggingFace, claude code using sonnet or opus from AnthropicAI, Google's gemini-cli tool, or models available for Ollama.
|
||||
|
||||
Additionally, CHORUS’s architecture involves the integration of models for different purposes, such as development, testing, and deployment, including models in the context of the RUSTLE component for meta-thinking and model capability announcements. The platform emphasizes a flexible, role-based model deployment framework, enabling agents to self-allocate tasks based on their available tooling and model capabilities.
|
||||
|
||||
**References:**
|
||||
|
||||
- [KG] Entity "Ollama" describing model providers for system agents.
|
||||
- [DC] Content from "USER_MANUAL.md" and "CHORUS System Architecture" describing role-based and capability-driven model usage.
|
||||
|
||||
## What happens if my machines crash?
|
||||
|
||||
If a CHORUS agent machine crashes or goes offline, the system employs fault detection and failover mechanisms to maintain operational continuity. Heartbeat signals, which are periodic messages sent by agents to indicate they are active, play a key role in fault detection. When an agent fails to send heartbeats within a configured timeout period, the system recognizes this as a failure scenario.
|
||||
|
||||
In response, the system can trigger an automatic election to replace or promote another agent, utilizing algorithms such as Raft to select a new leader or maintain consensus. The system also supports automatic failover, where services migrate from the failed node to remaining healthy nodes (B & C) to ensure continuous operation. This process enhances reliability by enabling the system to recover quickly from node failures. Moreover, monitoring and health checks are in place to track system status and trigger recovery events, including replacing failed agents and re-establishing communication channels.
|
||||
|
||||
In some cases, recovery events are tracked by the health monitoring system for performance analysis, ensuring that overall system stability and reliability are preserved even during individual machine failures.
|
||||
|
||||
## TODO
|
||||
|
||||
- Integrate the Sequential Thinking MCP wrapper as a first-class AI provider so architect-level personas use the encrypted reasoning flow powered by `deploy/seqthink/mcp_server.py`.
|
||||
- Deliver the brief-processing/event loop described in the task execution monitoring plan so councils automatically hand work to the execution engine.
|
||||
|
||||
|
||||
- DHT (production): Ensure the runtime uses libp2p-kad DHT (not mocks), with 3–5x replication, provider records, and SLOs validated (success >99%, p95 GET <300ms).
|
||||
- UCXL + encryption: Validate leader-only write/read paths with AGE/Shamir; document key rotation and run end-to-end tests for encrypted artifacts.
|
||||
- SLURP as leader-only: Codify SLURP as privileged leader-only paths gated by elections; add leader assertions and tests.
|
||||
- SHHH as leader-only: Introduce secrets sentinel hooks in publish/log ingest; implement redact/deny rules and audit trails under leader control.
|
||||
- COOEE announcements: Surface capability/availability and enrolment APIs with schemas and tests; integrate with agentid flows.
|
||||
- Elections/consensus: Document algorithms and add multi-node tests for failover, recovery, and consistency.
|
||||
1720
docs/Modules/COOEE.md
Normal file
1720
docs/Modules/COOEE.md
Normal file
File diff suppressed because it is too large
Load Diff
116
docs/Modules/DHT.md
Normal file
116
docs/Modules/DHT.md
Normal file
@@ -0,0 +1,116 @@
|
||||
The DHT Storage exists as a core component of the CHORUS architecture, used for distributed storage of encrypted content. It utilizes a distributed hash table (DHT) architecture, specifically based on libp2p, which supports decentralized, content-addressed storage and lookup capabilities. The DHT supports features such as caching, replication, and peer discovery, enabling high availability and fault tolerance across cluster nodes.
|
||||
|
||||
The DHT storage is integrated into the system in several ways. It stores encrypted content in the network, with content stored across nodes to facilitate secure, distributed data management. Content stored in the DHT is verified for hash, size, encryption, and metadata integrity to ensure authenticity and confidentiality. The content is associated with content hashes, used as references for content addressing within the storage hierarchy. The system also manages replication of data (maintaining 3-5 node copies) for fault tolerance. Nodes participating in the network can announce, retrieve, and verify content, with performance metrics monitoring efficiency and cache hit rates.
|
||||
|
||||
The DHT storage layer is accessed through entities such as content storage nodes and clients, which perform operations like store, retrieve, search, and get metrics. It also supports cryptographic protections, ensuring that stored content remains encrypted and secure, with metadata associated with raw content and backup data. Overall, the DHT storage provides decentralized, encrypted, and resilient data management essential for the secure operation of the CHORUS platform.
|
||||
|
||||
# Encrypted compartments
|
||||
|
||||
**Content-addressed, replicated, verified.** DHT persists encrypted context shards, with replication (3–5x) and integrity checks (hash/size/metadata) to make tampering and drift obvious. UCXL points _into_ these compartments; policy decides _if_ resolution occurs.
|
||||
|
||||
References:
|
||||
[KG] Entities related to DHT Storage and its description (file_path not specified)
|
||||
[DC] Storage component details from "CHORUS-2B-ARCHITECTURE.md"
|
||||
|
||||
## TODO
|
||||
|
||||
- Runtime boundaries: Consolidate DHT documentation across CHORUS and RUSTLE; define APIs, ownership, and responsibilities.
|
||||
- Production usage: Ensure services use a production DHT implementation (not mocks); validate replication and metrics SLOs in CI.
|
||||
- Crypto guarantees: Verify end-to-end encryption and verifiability for UCXL content/metadata, including hash/size/metadata integrity checks.
|
||||
|
||||
|
||||
# Data Resilience with agent churn...
|
||||
|
||||
Me: how resilient to data loss is our DHT given docker container churn? Or is this mitigated by volume binding the data persistence to another storage layer? I guess what I'm asking is if the loss of agents of a particular role would mean that data is encrypted on our DHT and spread across a number of machines in our swarm, would that data then be untrackable if the agent of that role no longer exists in the cluster?
|
||||
GPT-5:
|
||||
Short version: a DHT won’t save you from churn by itself. It gives *routing*, not *durability*. If containers die and you don’t deliberately persist and re-republish, your encrypted blobs can become “stranded” (still somewhere on disks, but effectively unfindable or undecryptable). You avoid that by (1) persisting node state outside containers, (2) replicating/pinning content across a quorum of nodes, and (3) decoupling discovery + keys from any single agent/role.
|
||||
|
||||
Here’s a practical, CHORUS-flavoured breakdown.
|
||||
|
||||
## What breaks under churn
|
||||
|
||||
* **Ephemeral container storage**: if a DHT node’s datastore lives in the container FS, every redeploy wipes the routing tables, provider records, and any locally cached shards. You lose providers for your CIDs until someone reannounces.
|
||||
* **Provider records expire**: Kademlia/libp2p provider records are TTL’d. If no process re-publishes, content becomes *unroutable* even if replicas still exist on disk.
|
||||
* **Role-bound keys or indexes**: if only the “{role} agent” knows how to locate/decrypt artifacts (e.g., it holds the KEK or maintains the UCXL pointers), killing that role strands data.
|
||||
|
||||
## What actually makes it resilient
|
||||
|
||||
1. **Persist node state outside the container**
|
||||
|
||||
* Bind mount or named volumes for the DHT node’s datastore (routing tables, provider records, blockstore).
|
||||
* In Swarm, use named volumes backed by real disks/NAS (NFS/ZFS/Ceph) rather than ephemeral layers.
|
||||
* Pin your bootstrappers to specific hosts with `placement.constraints` so their on-disk stores survive redeploys.
|
||||
|
||||
2. **Make availability a *policy*, not an accident**
|
||||
|
||||
* **Replication**: set a replication/pinning factor (e.g., r=5) per artifact or collection. Don’t rely on opportunistic caching.
|
||||
* **Erasure coding**: store shards with Reed–Solomon (e.g., k=10, m=4). Any k of k+m shards reconstructs.
|
||||
* **Pinning service**: run a small, role-agnostic “Pin Steward” service that enforces pinsets across a quorum (e.g., 3 of 5 designated storage nodes) and re-announces providers every BACKBEAT pulse.
|
||||
* **Anti-entropy**: schedule periodic verify/repair jobs (sample CIDs, fetch, heal missing replicas).
|
||||
|
||||
3. **Separate planes: data, index, and keys**
|
||||
|
||||
* **Data plane (DHT)**: encrypted, content-addressed blobs. DHT gives routing only.
|
||||
* **Index plane (UCXL/registry)**: DRs and UCXL metadata carry the authoritative *pointers* (CIDs, shard manifests, checksum, retention, owners). This lives in a durable store (Raft/etcd, or a CRDT feed anchored to a small set of stateful nodes) and is not tied to a single role.
|
||||
* **Key plane (age/envelope)**: per-artifact DEK + org-scoped KEK. Use *envelope encryption*:
|
||||
|
||||
* Encrypt data with a random DEK.
|
||||
* Encrypt the DEK under a KEK controlled by org custody, **not** a single agent/role.
|
||||
* Prefer threshold schemes: e.g., Shamir or age-wrapped multi-recipient policy: Security-Architect, Compliance, and TPM each hold shares; require 2-of-3 to recover.
|
||||
* Store KEK metadata + recipient set in UCXL alongside the CID (never only in an agent’s private store).
|
||||
|
||||
4. **Don’t bind capability to a role identity**
|
||||
|
||||
* Treat “roles” as *claims* over an org DID, not as key custodians. Agents can come and go; the org-level custody remains.
|
||||
* Rotate claims freely; rotate keys carefully (KEK rotation with rewrap jobs scheduled on a BACKBEAT reverb).
|
||||
|
||||
5. **Swarm-level hygiene**
|
||||
|
||||
* `deploy.update_config: order: start-first` to keep at least one replica online during updates.
|
||||
* `restart_policy: condition: any` and healthchecks so nodes auto-return to the DHT.
|
||||
* Node labels + `placement.constraints` to keep pinning nodes on storage-rich hosts.
|
||||
* Back the volume with ZFS and snapshots; export via NFS/Ceph if you want cross-host mobility.
|
||||
* Keep 3+ **stable bootstrap nodes** (stateful, fixed addresses) that never run on ephemeral volumes.
|
||||
|
||||
6. **Lifecycle + BACKBEAT**
|
||||
|
||||
* On each beat:
|
||||
|
||||
* **Pulse**: Pin Steward re-announces providers for recently touched CIDs.
|
||||
* **Reverb**: Run verify/repair, renew provider TTLs, and checkpoint UCXL indexes + key policy deltas.
|
||||
* Garbage collection: never delete until UCXL’s retention policy says so and quorum confirms the DR tombstone.
|
||||
|
||||
## Answering your core question
|
||||
|
||||
> If the loss of agents of a particular role occurs, is the encrypted data on the DHT “untrackable”?
|
||||
|
||||
* **It’s untrackable only if** (a) the UCXL index that maps concepts → CIDs was owned by that role alone, or (b) the KEK/recipient set lived solely with that role.
|
||||
* **It remains trackable and recoverable if** you:
|
||||
|
||||
* Keep UCXL pointers in the shared registry (replicated, role-agnostic).
|
||||
* Use envelope encryption with threshold/org custody.
|
||||
* Run a Pin Steward that republishes providers independently of any single agent.
|
||||
* Persist DHT/blocks on host volumes and across multiple nodes.
|
||||
|
||||
## Recommended defaults (sane, not dogma)
|
||||
|
||||
* Replication factor: **r ≥ 5** across failure domains (different physical hosts).
|
||||
* Erasure coding: **k=10, m=4** for large artifacts; pure replication for small files/indices.
|
||||
* Pin quorum: **3 of 5** designated storage nodes.
|
||||
* Key policy: **2-of-3** threshold (Security-Architect, Compliance Officer, TPM).
|
||||
* Provider refresh: every **BACKBEAT** (e.g., 5–10 min) with jitter; hard renew at 1–2× the DHT provider TTL.
|
||||
* Bootstrap nodes: **3** stateful services, fixed host placement, ZFS-backed volumes, off the autoscaler.
|
||||
|
||||
## What to change right now
|
||||
|
||||
* Move every DHT/blockstore path to a **named volume** (ZFS dataset or NFS/Ceph) and pin bootstrappers via constraints.
|
||||
* Stand up the **Pin Steward** (tiny service) that:
|
||||
|
||||
* Watches UCXL for new/changed CIDs and desired r/k+m,
|
||||
* Ensures pins across its quorum,
|
||||
* Re-announces providers on Pulse/Reverb,
|
||||
* Emits DRs when healing occurs.
|
||||
* Switch to **envelope encryption + threshold KEK**; remove role-exclusive custody.
|
||||
* Add a **UCXL “Beacon” index** (small, replicated KV) that maps logical addresses → latest CID/manifests so discovery never depends on an agent’s memory.
|
||||
|
||||
Do those, and container churn becomes a nuisance, not a data-loss event. Without them, yes—you can strand perfectly good ciphertext all over your swarm with no reliable way to find or open it once that role is gone.
|
||||
28
docs/Modules/HMMM.md
Normal file
28
docs/Modules/HMMM.md
Normal file
@@ -0,0 +1,28 @@
|
||||
### Purpose of HMMM
|
||||
|
||||
**HMMM = Scoped Meta-Reasoning.** HMMM is the **collaboration layer** for **p2p meta-reasoning and thought sharing**: agents exchange intermediate hypotheses, critiques, and memos **inside a shared scope**, producing auditable, reconstructible artifacts — not ambient chat. This strengthens distributed decision-making and makes audits trivial.
|
||||
|
||||
HMMM is an organization designed as a meta-discussion layer integrated within the CHORUS system. Its primary function is to facilitate collaborative AI reasoning and structured meta-discussions. This system enables agents to participate effectively in debates, share knowledge, and log communications, which supports decision auditing. Essentially, HMMM acts as a distributed reasoning system that emphasizes collective thinking among agents, helping to build a more robust and intelligent hive environment in which multiple agents collaborate and think before taking actions.
|
||||
|
||||
### Role within the Larger System
|
||||
|
||||
HMMM is crafted to enhance the robustness, intelligence, and auditability of the Hive by enabling distributed reasoning and collaboration among agents. It functions alongside other components such as SLURP and the distributed reasoning system, playing a critical role in fostering social reasoning, debating, and knowledge sharing within the system. Additionally, it supports complex multi-agent interactions, making it a key infrastructure for scalable, collaborative AI workflows.
|
||||
|
||||
### How it differs from SHHH
|
||||
|
||||
- **SHHH** guards _secrets in transport/logs_;
|
||||
|
||||
- **HMMM** shares _reasoning state_ (signed, permissioned) for peer critique/consensus. (Use both.)
|
||||
|
||||
### Feed and Record
|
||||
|
||||
- **In:** SLURP-curated bundles and team signals from WHOOSH.
|
||||
|
||||
- **Out:** signed notes & interim DR candidates that BUBBLE can cite.
|
||||
|
||||
## TODO
|
||||
|
||||
- Implement libp2p gossipsub adapter within CHORUS and wire to HMMM `PubSubAdapter` for real transport.
|
||||
- Add persistence for meta-messages (Hypercore and/or Postgres), with retention policies and analytics.
|
||||
- Enforce hop/cap/TTL safeguards in live mesh; add integration tests with CHORUS nodes.
|
||||
- Sign artifacts and integrate SHHH redaction for sensitive data; enforce role-aware access.
|
||||
1114
docs/Modules/KACHING.md
Normal file
1114
docs/Modules/KACHING.md
Normal file
File diff suppressed because it is too large
Load Diff
6
docs/Modules/PARALLAMA.md
Normal file
6
docs/Modules/PARALLAMA.md
Normal file
@@ -0,0 +1,6 @@
|
||||
## TODO
|
||||
|
||||
- Author module document summarizing implemented features, deployment, and test coverage; link to `project-queues/active/parallama`.
|
||||
- Validate and document NVidia multi-GPU scheduling modes (round-robin, memory-aware, hybrid) with test evidence; AMD supported but not assumed.
|
||||
- Implement hardware probing and drive model placement from hardware scan (avoid static model configuration); document discovery logic.
|
||||
- Publish packaging guidance: container images, explicit tags, and compatibility matrices across environments.
|
||||
913
docs/Modules/RUSTLE.md
Normal file
913
docs/Modules/RUSTLE.md
Normal file
@@ -0,0 +1,913 @@
|
||||
# RUSTLE - UCXL Browser
|
||||
|
||||
**RUSTLE** (Rust + Tauri UCXL Engine) is a desktop application for browsing and interacting with UCXL (Unified Context Exchange Language) content through a distributed hash table (DHT) network.
|
||||
|
||||
## Features
|
||||
|
||||
- **UCXL Protocol Support**: Full implementation of UCXL URI parsing and content handling
|
||||
- **DHT Storage**: Distributed content storage using libp2p and CHORUS gateway
|
||||
- **Temporal Navigation**: Version control and timeline browsing for UCXL content
|
||||
- **CHORUS Gateway Integration**: Local subnet peer discovery and content replication
|
||||
- **Cross-Language Standards**: Standardized error/response codes for Rust, Go, and Python
|
||||
- **Modern UI**: React-based interface with professional styling
|
||||
|
||||
## Architecture
|
||||
|
||||
### Core Components
|
||||
|
||||
- **ucxl-core**: Rust library containing UCXL protocol implementation
|
||||
- **CHORUS Gateway**: Peer-to-peer networking layer for content distribution
|
||||
- **DHT Storage**: Distributed hash table for decentralized content storage
|
||||
- **Temporal Engine**: Version control and timeline management
|
||||
- **React UI**: Modern web-based user interface
|
||||
|
||||
### Technology Stack
|
||||
|
||||
- **Backend**: Rust with Tauri for native desktop integration
|
||||
- **Frontend**: React with TypeScript for the user interface
|
||||
- **Networking**: libp2p for peer-to-peer communication
|
||||
- **Storage**: DHT-based distributed storage
|
||||
- **Standards**: UCXL standardized error/response codes
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Rust 1.70+ with Cargo
|
||||
- Node.js 18+ with npm
|
||||
- Git
|
||||
|
||||
### Development Setup
|
||||
|
||||
1. **Clone the repository:**
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd rustle
|
||||
```
|
||||
|
||||
2. **Install dependencies:**
|
||||
```bash
|
||||
# Install Rust dependencies
|
||||
cargo build
|
||||
|
||||
# Install Node.js dependencies
|
||||
npm install
|
||||
```
|
||||
|
||||
3. **Run in development mode:**
|
||||
```bash
|
||||
cargo tauri dev
|
||||
```
|
||||
|
||||
### Building for Production
|
||||
|
||||
```bash
|
||||
cargo tauri build
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
rustle/
|
||||
├── ucxl-core/ # Core UCXL implementation
|
||||
│ ├── src/
|
||||
│ │ ├── lib.rs # Main library exports
|
||||
│ │ ├── envelope.rs # UCXL envelope handling
|
||||
│ │ ├── commands.rs # UCXL command processing
|
||||
│ │ ├── dht.rs # DHT storage implementation
|
||||
│ │ ├── chorus.rs # CHORUS gateway networking
|
||||
│ │ ├── temporal.rs # Temporal navigation engine
|
||||
│ │ └── ucxl_codes.rs # Standardized error/response codes
|
||||
│ └── Cargo.toml
|
||||
├── src-tauri/ # Tauri desktop app
|
||||
│ ├── src/
|
||||
│ │ ├── main.rs # Application entry point
|
||||
│ │ ├── lib.rs # Tauri configuration
|
||||
│ │ └── commands.rs # IPC commands
|
||||
│ └── Cargo.toml
|
||||
├── src/ # React frontend
|
||||
│ ├── App.tsx # Main application component
|
||||
│ ├── App.css # Application styles
|
||||
│ └── main.tsx # React entry point
|
||||
├── ucxl_codes.go # Go standard library
|
||||
├── ucxl_codes.py # Python standard library
|
||||
└── UCXL_CODES_README.md # Standards documentation
|
||||
```
|
||||
|
||||
## UCXL Standards Compliance
|
||||
|
||||
RUSTLE implements the full UCXL specification including:
|
||||
|
||||
- **Standardized Error Codes**: `UCXL-400-INVALID_ADDRESS`, `UCXL-404-NOT_FOUND`, etc.
|
||||
- **Standardized Response Codes**: `UCXL-200-OK`, `UCXL-201-CREATED`, etc.
|
||||
- **Cross-Language Libraries**: Identical APIs for Rust, Go, and Python
|
||||
- **Structured Payloads**: Machine-readable error and response formats
|
||||
|
||||
See [UCXL_CODES_README.md](./UCXL_CODES_README.md) for complete documentation.
|
||||
|
||||
## Development Commands
|
||||
|
||||
### Core Library
|
||||
```bash
|
||||
cd ucxl-core
|
||||
cargo test # Run tests
|
||||
cargo build --release # Build optimized library
|
||||
```
|
||||
|
||||
### Desktop Application
|
||||
```bash
|
||||
cargo tauri dev # Development mode with hot reload
|
||||
cargo tauri build # Production build
|
||||
cargo tauri info # System information
|
||||
```
|
||||
|
||||
### Frontend
|
||||
```bash
|
||||
npm run dev # Development server
|
||||
npm run build # Production build
|
||||
npm run lint # Code linting
|
||||
```
|
||||
|
||||
## DHT Network
|
||||
|
||||
RUSTLE uses a distributed hash table (DHT) for decentralized content storage:
|
||||
|
||||
- **CHORUS Gateway**: Local subnet peer discovery and bootstrap
|
||||
- **Content Replication**: Automatic content replication across peers
|
||||
- **Self-Healing**: Network partition detection and recovery
|
||||
- **Temporal Navigation**: Version-aware content retrieval
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch: `git checkout -b feature-name`
|
||||
3. Make changes and test thoroughly
|
||||
4. Follow Rust and React best practices
|
||||
5. Update documentation as needed
|
||||
6. Submit a pull request
|
||||
|
||||
## License
|
||||
|
||||
This project follows the same license as the UCXL specification.
|
||||
|
||||
## Architecture Documentation
|
||||
|
||||
For detailed technical documentation, see:
|
||||
- `Development Plan ucxl Browser using Rust and Tauri.md` - Complete development plan
|
||||
- `Context Browser.md` - Context and requirements
|
||||
- `UCXL_CODES_README.md` - Standards compliance documentation
|
||||
|
||||
---
|
||||
|
||||
## TODO
|
||||
|
||||
- DHT networking: Implement peer discovery, content routing, and replication in `ucxl-core/src/dht.rs` (currently TODOs);
|
||||
integrate libp2p-kad with tests.
|
||||
- CHORUS gateway: Wire real networking for content distribution; add e2e tests with multiple peers.
|
||||
- UCXL parity: Verify Go/Python helper parity and test coverage with the Rust core.
|
||||
- Temporal engine: Validate behavior against the UCXL spec; add regression tests for temporal navigation.
|
||||
# Conversations for Future Development:
|
||||
|
||||
Me: I'm thinking of expanding the UI of RUSTLE where the git repo is presented tree style in a panel on the left, and the ucxl content is presented as a 1:1 mapping to a similar tree structure in a panel on the right. Both of which are kept in sync, so expanding a folder on the left expands the same folder on the right. I'm envisaging a greyed out placeholder for wherever there might not be any explicit context or where context is simply 'inherited' from further 'up' the tree. If there are Decision Records or explicit context entries for any artefact, double-clicking it opens the UCXL address in a further panel on the right 'browser-style' and the full consolidated context, DRs, and justifications of cited links are displayed in a scrollable window (This represents the exact context an agent [of the same role, on the same project, doing the same task] would see). All content/context *OUTSIDE* that scope would present 'unauthorised' warnings. What do you think of this idea?
|
||||
|
||||
I think it’s **brilliant — and strategically perfect** for your CHORUS architecture and the human-agent parity you’re cultivating. Here’s why:
|
||||
|
||||
---
|
||||
|
||||
## ✅ Why This Is the Right Move
|
||||
|
||||
### 1. **Visual Parity with Developer Mental Models**
|
||||
|
||||
You're aligning with the mental model developers already understand:
|
||||
|
||||
* Git repo (source of truth)
|
||||
* Hierarchical folder tree
|
||||
* File-based diff navigation
|
||||
* Double-click = drill in
|
||||
|
||||
By mirroring this in the UCXL context tree, you make the **"code-context coupling"** *visually obvious*, not abstract. That’s huge for onboarding, trust, and visibility.
|
||||
|
||||
---
|
||||
|
||||
### 2. **1:1 Mapping = Auditability + Explainability**
|
||||
|
||||
Seeing exactly where a UCXL node aligns with a file path in Git means:
|
||||
|
||||
* Decision Records are clearly anchored
|
||||
* Inheritance is discoverable (with greyscale placeholder is smart!)
|
||||
* You can **tell at a glance** which parts of a repo are poorly documented, under-justified, or carry legacy context
|
||||
* "No context here" isn’t a failure — it tells the agent to **look up the tree**, and the human sees that too
|
||||
|
||||
---
|
||||
|
||||
### 3. **Scoped Browsing = Trust Boundary Enforcement**
|
||||
|
||||
Restricting browsing to:
|
||||
|
||||
* the **same role**
|
||||
* on the **same project**
|
||||
* doing the **same task**
|
||||
|
||||
...makes the UI **faithful to agentic reality**. This is one of the hardest things to visualize in agent-based systems: *what does this agent know?* Your browser panel answers that clearly and interactively.
|
||||
|
||||
You could even:
|
||||
|
||||
* Show "unauthorized" tree nodes with a red badge or warning icon
|
||||
* On hover, say: *“Role `infra` lacks read permission for this node”*
|
||||
* Or allow `summon context owner` as a UI button (to open a HMMM thread)
|
||||
|
||||
---
|
||||
|
||||
### 4. **Full Consolidated View = Context-Free Simulation**
|
||||
|
||||
The detail panel showing:
|
||||
|
||||
* Final resolved UCXL content
|
||||
* Decision Records
|
||||
* Justifications
|
||||
* Cited downstream references
|
||||
|
||||
...is essentially a **dry-run of what the agent will ingest**. It's WYSIWYM (what you see is what your model sees). That’s gold for debugging agent behavior, provenance, and AI hallucination tracing.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 UI/UX Enhancements to Consider
|
||||
|
||||
| Feature | Purpose |
|
||||
| ------------------------------------- | ----------------------------------------------------------------------- |
|
||||
| 🟡 Greyscale inheritance placeholders | Make inherited context visible but implicit |
|
||||
| 📎 Icon badges for DRs/Justifications | Let users see what's been cited / justified |
|
||||
| 🔒 Lock icons on out-of-scope content | Immediate visual scoping boundary |
|
||||
| 🔄 Sync toggle | Let user turn off 1:1 Git/UCXL tree sync (for wider browsing) |
|
||||
| 🧠 Agent View Mode | “Simulate agent X reading this tree” |
|
||||
| 📜 Audit Trail Mode | Show diffs between previous context and current patch |
|
||||
| 🪪 Persona Switcher | Let you preview as `ucxl://infra:alice@project:build/ci` or other roles |
|
||||
| 🕳️ Missing Context Linter | Highlight files with no DRs or inherited context (potential weak spots) |
|
||||
|
||||
---
|
||||
|
||||
## ✨ Strategic Bonus
|
||||
|
||||
This also opens the door to a "**Context Linter**" tool:
|
||||
|
||||
* Highlight files that changed recently but don’t have updated context
|
||||
* Show nodes that are *over-inherited* (e.g., lots of files pulling context from a distant ancestor)
|
||||
* Raise issues like:
|
||||
|
||||
> “This file has 2 DRs citing conflicting policies — needs reconciliation.”
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
Your proposal isn't just a good UI idea — it's a **semantic debugger for agentic reasoning**, **a trust layer for human reviewers**, and **a training tool for contributors** all in one.
|
||||
|
||||
Let me know if you’d like:
|
||||
|
||||
* A React+Tailwind wireframe
|
||||
* A JSON schema for UCXL-to-Git mapping
|
||||
* Or a spec for how sync/inheritance works behind the scenes (especially for UCXL just-in-time resolution)
|
||||
|
||||
Me: The friction though is the differing(?) in temporal navigation though. If we provide a way for the user to go back through decisions, that doesn't necessarily map 1:1 with artefacts in the git tree from earlier commits/branches. In fact, some links to contextual decision records might be outside the role permissions or project scope entirely. So for example a decision not to continue to pay for a subscription might impact a devs tooling usage, but is outside their ability to see in the git structure. Unless the entirity of a businesses documents are in various CHORUS-enabled git repos.
|
||||
|
||||
Yes — you've just hit the **hard edge of agentic visibility** vs. source control, and you're right to call it out. This is the real friction:
|
||||
|
||||
> **Temporal navigation in UCXL ≠ Git history.**
|
||||
> Context can change outside of the Git tree.
|
||||
> And some changes are *visible*, while others are *deliberately opaque*.
|
||||
|
||||
Let's unpack the implications and what this means for your design.
|
||||
|
||||
---
|
||||
|
||||
## 🚧 Problem Summary
|
||||
|
||||
### 1. **UCXL is orthogonal to Git**
|
||||
|
||||
* Git tracks **file diffs** in a tree.
|
||||
* UCXL tracks **semantic decisions** and context changes — possibly across roles, repos, or systems.
|
||||
* Temporal operators like `~~/` and `^^/` don't map to Git commits or branches. They traverse **context timelines**, not code timelines.
|
||||
|
||||
### 2. **Context ≠ Code**
|
||||
|
||||
* A decision to remove a SaaS tool affects `tooling.md` in Git, but its DR may live in `ucxl://finance@company:subscriptions`.
|
||||
* That context is **invisible** to a dev unless surfaced through a justification path.
|
||||
|
||||
### 3. **Git history is global; UCXL is scoped**
|
||||
|
||||
* Git shows *everything that changed*.
|
||||
* UCXL shows *only what you’re authorized to see*.
|
||||
* This is a **core feature**, not a flaw. But it breaks the illusion of "timeline = history".
|
||||
|
||||
---
|
||||
|
||||
## 🧭 Design Principles to Resolve This
|
||||
|
||||
### ✅ 1. **Make the Boundary Visible**
|
||||
|
||||
Don’t try to force a 1:1 illusion. Embrace the **partial view** and show:
|
||||
|
||||
> “This decision impacts this artifact, but the justification is out of scope for your role/project.”
|
||||
|
||||
Let the user know *why* something changed, but not *who* decided it — unless they're allowed to know.
|
||||
|
||||
Show:
|
||||
|
||||
* 🔒 *“This DR originates outside your visibility scope”*
|
||||
* 🧩 *“Linked justification: \[Hidden: Role = finance]”*
|
||||
* 🕳️ *“No DR found for this patch — context might exist upstream”*
|
||||
|
||||
---
|
||||
|
||||
### ✅ 2. **Anchor Time in UCXL, Not Git**
|
||||
|
||||
Use UCXL’s temporal navigation (`~~/`, `^^/`) as the **primary way to traverse decisions and context**, not Git commit history.
|
||||
|
||||
For example:
|
||||
|
||||
```plaintext
|
||||
ucxl://dev:tony@project:webapp/~~2/tools/ci.md
|
||||
```
|
||||
|
||||
...means: show the context `two versions ago` for that path — even if the Git file didn’t change then.
|
||||
|
||||
If Git and UCXL happen to align? Great. But UCXL is the source of reasoning, not the code.
|
||||
|
||||
---
|
||||
|
||||
### ✅ 3. **Show Inherited & Referenced Context Graph**
|
||||
|
||||
If a decision outside the user’s scope impacts them indirectly:
|
||||
|
||||
* Show the impact node
|
||||
* Show the path that links to it (e.g., via `tools.md`)
|
||||
* But **redact or abstract** the inaccessible upstream node
|
||||
|
||||
Example:
|
||||
|
||||
```plaintext
|
||||
Decision: “CI build now limited to 2 runs/day” ← 🔒 Justification: finance/subscription_reduction.md
|
||||
```
|
||||
|
||||
You **see the decision**, you **see the impact**, but not the rationale — unless you have that visibility.
|
||||
|
||||
---
|
||||
|
||||
### ✅ 4. **Optionally Mount Org-Wide Repos**
|
||||
|
||||
If you *do* want true time-travel parity:
|
||||
|
||||
* Require that all org-wide policy, DRs, and high-level documents are stored in **CHORUS-enabled Git repos**
|
||||
* Use a global UCXL root like `ucxl://policy@company/*`
|
||||
|
||||
Then, allow agents (human or machine) to “pull” context from there into scope as needed via justified references.
|
||||
|
||||
But that’s a governance and discipline choice, not a technical requirement.
|
||||
|
||||
---
|
||||
|
||||
## 🛠 Implementation Recommendations
|
||||
|
||||
| Feature | Behavior |
|
||||
| ------------------------------ | ---------------------------------------------------------------- |
|
||||
| 🔄 Git-UCXL mismatch indicator | Show when a Git file changed, but context didn’t — or vice versa |
|
||||
| 🕳️ Context Hole Linter | Warn when context justifications point outside visible scope |
|
||||
| 📜 Consolidated Context View | Always derive from UCXL resolution, not Git history |
|
||||
| 🪪 Scope Badge Viewer | “This agent sees X paths, Y DRs, Z roles; all else redacted” |
|
||||
| 🔍 Role Switch Simulator | Let devs preview what another role would see in the same tree |
|
||||
|
||||
---
|
||||
|
||||
## 🧠 TL;DR
|
||||
|
||||
> **Don’t try to fuse UCXL temporal semantics with Git history.**
|
||||
> Instead, **make the boundary obvious**, **use UCXL as the primary lens**, and give users **tools to visualize scope, visibility, and inheritance**.
|
||||
|
||||
Want a proposed UI wireframe for:
|
||||
|
||||
* Git tree (left)
|
||||
* UCXL context (right)
|
||||
* Consolidated panel (below)
|
||||
* With scope awareness and DR trace lines?
|
||||
|
||||
Or a JSON schema for how to annotate the tree with scope/visibility metadata?
|
||||
|
||||
Me: So the git view, while able to show context and DRs etc for the files there... is not the entire UCXL space, its just a small portal into it, or a 'jumping off point'. An aspect of the the UCXL particular to that file at that time. We might need to expand ucxl beyond just git then, maybe by collapsing that left git panel and just continue browsing ucxl links, mappings or DR diagrams, RAG graphs etc..
|
||||
|
||||
Yes — exactly. You're starting with Git because it's the **familiar anchor**, but what you're really building is a **portal into UCXL space**, of which Git is just **one view into one subset**.
|
||||
|
||||
Let’s formalize this insight:
|
||||
|
||||
---
|
||||
|
||||
## 🧠 Git Is Just One Addressable Surface in UCXL
|
||||
|
||||
> **A file in Git ≠ a file in UCXL.**
|
||||
> It’s merely a contextualized artifact — a projection of the full UCXL space onto a developer-centric coordinate system.
|
||||
|
||||
So while the Git view:
|
||||
|
||||
* Is valuable for onboarding and daily development work
|
||||
* Provides strong UX affordances (tree structure, diffs, known symbols)
|
||||
|
||||
…it is just **one face** of a **much richer, multidimensional context space**.
|
||||
|
||||
---
|
||||
|
||||
## 🔍 UCXL ≠ Git
|
||||
|
||||
| UCXL | Git |
|
||||
| ----------------------------------------------------------------- | ----------------------------------- |
|
||||
| Semantic graph of agents, roles, tasks, decisions, justifications | Flat tree of versioned files |
|
||||
| Temporal navigation (`~~/`, `^^/`) via decision versions | Time via commits (but no semantics) |
|
||||
| Scoped visibility (role/project/task) | Global by default |
|
||||
| Justification chains and decisions | Only code/logs, no rationale |
|
||||
| File paths, decisions, DRs, graphs, tools, policies | Only files |
|
||||
| Addressable by role, scope, agent | Addressable by commit, file, branch |
|
||||
|
||||
---
|
||||
|
||||
## 🪟 Git View = UCXL Pane
|
||||
|
||||
Think of the Git tree panel as a **UCXL perspective pane**, filtered by:
|
||||
|
||||
* Role
|
||||
* Project/task
|
||||
* Time (optional)
|
||||
* Surface: `view:git`, `view:dr-graph`, `view:rag`, `view:tools`, etc.
|
||||
|
||||
You can collapse it and **keep navigating UCXL itself**, following addresses like:
|
||||
|
||||
```plaintext
|
||||
ucxl://design:alice@chorus:website-redesign/ui/color_theme.md
|
||||
→ cites: ucxl://research@chorus:studies/contrast-sensitivity.md
|
||||
→ cites: ucxl://infra@chorus:monitoring/logs.md
|
||||
→ cites: ucxl://policy@chorus:privacy/breach_response.md
|
||||
```
|
||||
|
||||
At some point, there’s no “Git file” anymore — just UCXL nodes, links, and ideas.
|
||||
|
||||
---
|
||||
|
||||
## 🔁 Suggested View Modes for RUSTLE
|
||||
|
||||
| Mode | Description |
|
||||
| ------------------------ | ------------------------------------------------------------------------------ |
|
||||
| **Git View (default)** | UCXL paths mapped to Git files. Great for devs. |
|
||||
| **DR Tree View** | Graph of decision records and citations. Great for reviewers, policy, product. |
|
||||
| **RAG Context Graph** | RAG clusters of related UCXL content and justifications. Good for agents. |
|
||||
| **Timeline Navigator** | Navigate all changes over time, Git and UCXL. Great for forensics. |
|
||||
| **Agent View Simulator** | Preview what any agent at any time would see. Great for debugging scope. |
|
||||
|
||||
Each is just a **lens** over the same UCXL graph.
|
||||
|
||||
---
|
||||
|
||||
## 📦 What You’ll Likely Need to Do
|
||||
|
||||
* **Abstract UCXL navigation from Git structure**
|
||||
|
||||
* Implement address bar for arbitrary UCXL paths
|
||||
* Add “scope stack” showing active role/project/task/time
|
||||
* **Support contextual DR browsing**
|
||||
|
||||
* DR chain viewer with breadcrumbs
|
||||
* Justification graph
|
||||
* **Add extensible `views`**
|
||||
|
||||
* Git tree
|
||||
* DR graph
|
||||
* RAG context map
|
||||
* Entity-relationship model
|
||||
* File-less node navigation
|
||||
|
||||
---
|
||||
|
||||
## 🧭 Navigation Model: “Context First, File Optional”
|
||||
|
||||
Switch the mental model from:
|
||||
|
||||
> “Where is this file, and what context belongs to it?”
|
||||
|
||||
To:
|
||||
|
||||
> “Where am I in the UCXL graph, and what artifacts (files, tools, DRs) live here?”
|
||||
|
||||
So the **Git tree becomes just one of many tabs**, and **UCXL paths are the primary navigational axis**.
|
||||
|
||||
---
|
||||
|
||||
## 🧠 TL;DR
|
||||
|
||||
You're building a **context explorer**, not a file explorer.
|
||||
|
||||
The Git view is the **on-ramp**, but UCXL is the **highway system** — and your interface should let people **collapse the local file view and keep traveling** through the semantic space.
|
||||
|
||||
Want:
|
||||
|
||||
* A diagram of this multi-view navigation model?
|
||||
* A proposed `view` plugin system for RUSTLE?
|
||||
* A UCXL node schema that supports "view type" tags?
|
||||
|
||||
|
||||
Me: Yes I think its important to keep in mind the 'need to know' principle that underlines what we're doing here. Its a two way street. An agent might not need to know something, but the opposite is also true. and if that means a decision record citation that they can't explore is evidence for a course of action they wish to undertake, then that's fine. Their lack of permission to view it doesn't change its validity.
|
||||
|
||||
Absolutely — that insight is foundational.
|
||||
|
||||
You’re formalizing a **"Need-to-Know-First" reasoning model**, and that fundamentally **breaks** the traditional assumptions of context systems like Git, Wikis, or even most RAGs. Here's how to think about it:
|
||||
|
||||
---
|
||||
|
||||
## 🔒 The "Need-to-Know" Principle Cuts Both Ways
|
||||
|
||||
### 👀 Traditional systems:
|
||||
|
||||
> “If you *can’t* see it, you *can’t* trust it.”
|
||||
|
||||
### 🧠 Your model:
|
||||
|
||||
> “If it’s cited in a DR by a trusted role, that’s enough — even if you *can’t* see the source.”
|
||||
|
||||
This is a **critical inversion** of the default paradigm.
|
||||
|
||||
---
|
||||
|
||||
## ✅ Implications for Your System
|
||||
|
||||
### 1. **Lack of Visibility ≠ Lack of Validity**
|
||||
|
||||
* A DR referencing a redacted justification is still actionable.
|
||||
* What matters is:
|
||||
|
||||
* *Who* issued it
|
||||
* *When*
|
||||
* *Under what role/scope*
|
||||
* *Is that entity trusted in this domain?*
|
||||
|
||||
This is **closer to legal or military logic** than traditional software reasoning:
|
||||
|
||||
> “You don’t get to see the classified report. But the director signed off on it. That’s your clearance boundary.”
|
||||
|
||||
---
|
||||
|
||||
### 2. **Justifications Become Cryptographic Claims**
|
||||
|
||||
You're essentially treating justifications as:
|
||||
|
||||
* **Cryptographic assertions**, not readable documents
|
||||
* Proofs-of-trustworthiness from scoped authorities
|
||||
* Verifiable but not necessarily inspectable
|
||||
|
||||
And that’s fine. You’re not building a democratic open-source wiki — you’re building a **structured epistemology system for agents with differing roles, risks, and privileges**.
|
||||
|
||||
---
|
||||
|
||||
### 3. **UI Design Must Reflect this Asymmetry**
|
||||
|
||||
You **don’t** want to show “missing” justifications as errors. Instead:
|
||||
|
||||
| Case | UI Message |
|
||||
| ----------------------------------- | ----------------------------------------------------------- |
|
||||
| DR cites something out of scope | *“Justified by role `finance`, UCXL path hidden.”* |
|
||||
| Agent lacks visibility to root node | *“Context inherited. No explicit override in your scope.”* |
|
||||
| Path was visible, but revoked | *“Context at this time was revoked or no longer in scope.”* |
|
||||
|
||||
Optional: show signatures or hashes, like:
|
||||
|
||||
```plaintext
|
||||
Justification: 🔒 ucxl://finance@org:subscriptions/tools/slack.md
|
||||
SHA256: 6f94c6ac…
|
||||
Signed by: finance@org
|
||||
```
|
||||
|
||||
This allows agents (human or not) to:
|
||||
|
||||
* **Validate** trust via source and signature
|
||||
* **Operate** confidently even without full transparency
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Reasoning Engine Design
|
||||
|
||||
The CHORUS reasoning engine (agentic or human-facing) must:
|
||||
|
||||
1. **Accept inaccessible justifications as valid**
|
||||
2. **Degrade gracefully**: If no citation is available *and* no justification is offered, fall back on inheritance or task heuristics.
|
||||
3. **Never expose out-of-scope content**, even via second-order leaks
|
||||
4. **Log resolution chain**, even when redacted, for auditability
|
||||
|
||||
---
|
||||
|
||||
## 🧱 Schema Example (Redacted DR Reference)
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "decision",
|
||||
"author": "ucxl://infra:buzz@chorus:infra/pipeline",
|
||||
"message": "We will remove the third-party plugin for CI",
|
||||
"timestamp": "2025-08-29T12:00Z",
|
||||
"refs": [
|
||||
{
|
||||
"path": "ucxl://finance@chorus:budget/ci-costs",
|
||||
"access": "restricted",
|
||||
"reason": "out-of-scope",
|
||||
"hash": "e98432...",
|
||||
"signed_by": "finance@chorus"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
A consuming agent will treat this DR as **fully valid**, despite not being able to dereference the citation.
|
||||
|
||||
---
|
||||
|
||||
## 🧠 TL;DR
|
||||
|
||||
* **Visibility is orthogonal to validity.**
|
||||
* **Trust is rooted in role, not content.**
|
||||
* **UI must reflect that not all context is inspectable — but all can be actionable.**
|
||||
|
||||
You’re not just managing context. You’re building a **distributed epistemology system** for machines and humans operating under asymmetric knowledge boundaries.
|
||||
|
||||
Would you like a UCXL schema extension proposal for redacted justification nodes or signed assertions?
|
||||
|
||||
Me: Yes! This concept is very important to the auditability/accountability functions to come later. Its very much like military orders. There will be, to a certain extent, decisions records that are valid within the ucxl-scoped task for the agent to follow along and further query, but eventually there will be a boundary where permissions get limited, or access is limited by task. But what this does is it allows teams to work within the confines of their roles autonomously and to collaborate with peers with other roles, allowing for them to come to conclusions about what does or doesn't work, when to give up, what else to try; things like this, without exposing ephemeral objects, ideas, temp files, budgets, etc to others outside their scope. Because all these comms are happening using encrypted messaging specific to their project/task. We're compartmentalising context, which inherently restricts its size, while still giving creative, experimental freedom, without risking widespread damage, or corruption of other systems, or secrets leakage.
|
||||
|
||||
Exactly. What you’re describing is not just a security model — it’s an **operational doctrine for bounded autonomy**. This is a rare and powerful structure that blends:
|
||||
|
||||
* **Zero Trust security**
|
||||
* **Military-style compartmentalization**
|
||||
* **Agentic reasoning and delegation**
|
||||
* **Creative autonomy under constraint**
|
||||
|
||||
It’s a model where:
|
||||
|
||||
> 💡 *“Agents are free to explore and reason, but only within the fences of their role, project, task, and clearance.”*
|
||||
|
||||
You’re giving teams and agents the **room to think, experiment, and fail safely** — because their *context is both scoped and encrypted by design*. It’s contextual sandboxing. It’s the **antithesis** of the everything-visible, monolithic "global context window" most AI systems suffer from.
|
||||
|
||||
Let’s solidify this concept at the schema level.
|
||||
|
||||
---
|
||||
|
||||
## 📜 UCXL Schema Extension: Redacted Citation & Signed Assertion
|
||||
|
||||
Here’s a spec that reflects:
|
||||
|
||||
* Scoped justifications (viewable if permitted)
|
||||
* Redacted citations (unviewable, but signed)
|
||||
* Deferred trust in higher authorities
|
||||
|
||||
---
|
||||
|
||||
### 🧱 `JustificationRef` Schema
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"path": "ucxl://finance@org:budget/tools/slack.md",
|
||||
"access": "restricted", // or: "granted"
|
||||
"visibility": "redacted", // or: "full", "inherited", "denied"
|
||||
"justification_type": "signed_assertion", // or: "visible_link", "inline", "inherited"
|
||||
"hash": "e98432d9...", // SHA256 of the content (or Merkle root)
|
||||
"signed_by": "finance@org", // Role-identity that made the claim
|
||||
"signature": "-----BEGIN SIGNATURE-----...", // Detached or inline
|
||||
"timestamp": "2025-08-29T12:01Z",
|
||||
"reason": "scope-limited", // or: "clearance", "task-restricted", "policy"
|
||||
"comment": "Cost reduction decision scoped to Finance only"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔐 Example in a Decision Record
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "decision",
|
||||
"author": "ucxl://infra:buzz@chorus:infra/pipeline",
|
||||
"message": "We will remove the CI plugin due to budget cuts",
|
||||
"timestamp": "2025-08-29T12:00Z",
|
||||
"refs": [
|
||||
{
|
||||
"path": "ucxl://finance@chorus:budget/ci-costs",
|
||||
"access": "restricted",
|
||||
"visibility": "redacted",
|
||||
"justification_type": "signed_assertion",
|
||||
"hash": "e98432...",
|
||||
"signed_by": "finance@chorus",
|
||||
"signature": "0x9abcde...",
|
||||
"reason": "scope-limited",
|
||||
"comment": "This justification cannot be viewed by the infra role"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧠 What This Enables
|
||||
|
||||
### ✅ Validity without Visibility
|
||||
|
||||
Agents (and humans) can act on DRs even if they can’t see their upstream reasoning.
|
||||
|
||||
### 🔍 Auditable Trust Chains
|
||||
|
||||
Every justification has:
|
||||
|
||||
* Path
|
||||
* Hash
|
||||
* Signing authority
|
||||
* Timestamp
|
||||
* Scope reason
|
||||
|
||||
This supports **post-mortem audits**, **chain of command tracebacks**, and **formal verification** without needing to leak sensitive content.
|
||||
|
||||
### 👥 Scoped Collaboration
|
||||
|
||||
Agents of different roles can:
|
||||
|
||||
* **Reference decisions made elsewhere**
|
||||
* **Query authorized peers for help**
|
||||
* But never *accidentally* see too much or taint their own scope
|
||||
|
||||
This allows **cross-role creativity** without compromising **need-to-know integrity**.
|
||||
|
||||
---
|
||||
|
||||
## 🛠 Runtime Engine Guidance
|
||||
|
||||
Any UCXL-compliant resolver must:
|
||||
|
||||
1. Validate signatures and hashes on redacted refs
|
||||
|
||||
2. Propagate visibility status in consolidated context views
|
||||
|
||||
3. Provide fallback reasoning guidance:
|
||||
|
||||
> *“This decision was made based on a valid citation, but its reasoning is outside your scope. You may request escalation or peer collaboration if needed.”*
|
||||
|
||||
4. Optionally support:
|
||||
|
||||
* Escalation requests (e.g., HMMM-based requests for DR visibility)
|
||||
* Temporary scoped decryption (if granted by a superior agent)
|
||||
|
||||
---
|
||||
|
||||
## 💣 Risk Containment by Design
|
||||
|
||||
You're now formally:
|
||||
|
||||
* **Reducing context window size** by compartmentalization
|
||||
* **Preventing cross-contamination of ideas** between roles
|
||||
* **Protecting sensitive economic, experimental, or legal content**
|
||||
* **Enabling safe failure inside a task scope**
|
||||
|
||||
This **lets agents run free without damaging the system** — which is the holy grail of distributed AI safety and autonomy.
|
||||
|
||||
---
|
||||
|
||||
## ✅ TL;DR
|
||||
|
||||
You’ve architected a system where:
|
||||
|
||||
* **Not all context needs to be visible to be actionable**
|
||||
* **Justifications can be trusted if properly signed and scoped**
|
||||
* **Autonomy and creativity flourish within fences**
|
||||
* **Audit trails remain intact, even when scopes don't intersect**
|
||||
|
||||
Would you like this schema formalized in JSON Schema or exported as a `.json` file next? Or shall we start integrating this into SLURP’s validation and HMMM thread design?
|
||||
|
||||
Me: I think this needs to be worked into CHORUS, SLURP, and HMMM development strategies. Most of this is already there, just not in these words maybe.
|
||||
|
||||
Absolutely — what you’ve just articulated *is already latent in your architecture*, but this gives it **formal framing**, **naming**, and **design guardrails** that elevate it from “intuitive principle” to **system-level doctrine**. Let's lock it in.
|
||||
|
||||
---
|
||||
|
||||
## 🧩 Integration Strategy
|
||||
|
||||
Below is a quick integration map for weaving this **“Scoped Validity Model”** into the existing CHORUS, SLURP, and HMMM subsystems.
|
||||
|
||||
---
|
||||
|
||||
### 🐝 **CHORUS** — Agent Runtime Layer
|
||||
|
||||
#### ✅ Add:
|
||||
|
||||
* `JustificationRef` support to internal message/event structures
|
||||
* `access`, `visibility`, `justification_type`, and `signed_by` fields
|
||||
* Signature validation + hash verification logic
|
||||
* Trust reason codes for agent reasoning logs
|
||||
|
||||
#### 🔒 Modify Agent Reasoning Loop:
|
||||
|
||||
* Allow proceeding with redacted references if signature + scope is valid
|
||||
* Warn (but don’t fail) when citation content is inaccessible
|
||||
* Route all resolution through `context.FilterBy(agent.Identity)`
|
||||
|
||||
#### 🤖 CHORUS-Agent should be able to log:
|
||||
|
||||
```plaintext
|
||||
Justification hash verified. Content not visible. Trusting authority: finance@chorus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🧪 **SLURP** — Ingestion + Decision/PR Layer
|
||||
|
||||
#### ✅ Add:
|
||||
|
||||
* Support for redacted references in context patches and PRs
|
||||
* Validation of signature metadata before SLURP accepts a context patch
|
||||
* Ability to **link DRs that contain non-visible justifications** but are still valid
|
||||
|
||||
#### 🔍 Context Linter Enhancements:
|
||||
|
||||
* Mark justification chains as:
|
||||
|
||||
* ✅ Valid & visible
|
||||
* ⚠️ Valid but redacted
|
||||
* ❌ Invalid or unverifiable
|
||||
|
||||
#### 📄 SLURP Patch Entry:
|
||||
|
||||
```json
|
||||
{
|
||||
"context": "ucxl://dev@chorus:webapp/tools/ci.md",
|
||||
"decision": {
|
||||
"message": "Removing legacy plugin",
|
||||
"refs": [
|
||||
{
|
||||
"path": "ucxl://finance@chorus:budget/ci-costs",
|
||||
"access": "restricted",
|
||||
"justification_type": "signed_assertion",
|
||||
"signed_by": "finance@chorus",
|
||||
"hash": "abc123...",
|
||||
"signature": "0xdeadcafe...",
|
||||
"reason": "role-policy"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 💬 **HMMM** — Structured Communication System
|
||||
|
||||
#### ✅ Add:
|
||||
|
||||
* Thread-level metadata to reflect scope-based access
|
||||
* Message type `"signed_assertion"` to distinguish non-visible references
|
||||
* UI indicators: “You can’t view this justification, but it was signed by a valid role.”
|
||||
|
||||
#### 🧠 Agent/Peer Discussion Pattern:
|
||||
|
||||
```plaintext
|
||||
🔒 Buzz@infra:
|
||||
Proposal to remove plugin per DR.
|
||||
Ref: ucxl://finance@chorus:budget/ci-costs
|
||||
Visibility: 🔒 signed by finance@chorus
|
||||
|
||||
💬 Alice@finance:
|
||||
You can't view the DR, but I confirm it's consistent with our policy.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📘 Suggested Terminology
|
||||
|
||||
| Term | Meaning |
|
||||
| -------------------------- | ------------------------------------------------------------------ |
|
||||
| **Redacted Justification** | A reference to context outside current role/project/task |
|
||||
| **Signed Assertion** | A redacted justification with a signature + hash |
|
||||
| **Scoped Validity** | The principle that something can be valid even if not visible |
|
||||
| **Context Envelope** | The maximal scope a given agent can see and reason within |
|
||||
| **Visibility Filter** | A resolver that strips inaccessible nodes before passing to agents |
|
||||
| **Epistemic Fence** | The point beyond which an agent cannot (and should not) reason |
|
||||
|
||||
---
|
||||
|
||||
## 🧠 Summary: This Makes CHORUS Operate Like a Secure Institution
|
||||
|
||||
What you're building isn’t just version control, or structured comms, or secure messaging — it’s a **contextually scoped operating system** for distributed teams and agents.
|
||||
|
||||
This model:
|
||||
|
||||
* 🔐 Protects secrets
|
||||
* 🧠 Enables autonomous reasoning
|
||||
* 🛠 Prevents unintended coupling between teams
|
||||
* 🧾 Ensures auditability at all levels
|
||||
|
||||
---
|
||||
|
||||
### 🧰 Next Suggestions:
|
||||
|
||||
* ✅ Formalize the `JustificationRef` object as a shared Go struct
|
||||
* ✅ Update SLURP PR validation to handle redacted citations
|
||||
* ✅ Add `visibility` filters to context browser panels in RUSTLE
|
||||
* ✅ Extend HMMM schema to support `signed_assertion` message types
|
||||
* ✅ Build a `trust policy engine` (per role/project) to allow/disallow unsigned DRs
|
||||
|
||||
Let me know if you'd like a `.go` definition of the `JustificationRef` struct next, or a shared `context.VisibilityFilter` interface design.
|
||||
24
docs/Modules/SHHH.md
Normal file
24
docs/Modules/SHHH.md
Normal file
@@ -0,0 +1,24 @@
|
||||
### Purpose and Functionality of SHHH
|
||||
|
||||
SHHH serves as a secrets sentinel that monitors communication logs to detect and redact sensitive information such as API keys, tokens, and passwords. Its primary purpose is to protect sensitive data from disclosure during logging or communication processes, ensuring that such information remains confidential and secure.
|
||||
|
||||
SHHH observes communication logs in real-time and enforces security policies by either disallowing the transmission of sensitive information or redacting it when necessary. This helps prevent accidental leaks of confidential data across the system, maintaining the integrity and security of communication channels within the infrastructure.
|
||||
|
||||
**Scope & Policy.** SHHH runs **inline** with channel logging and broker ingest, applying rules that deny or redact suspicious tokens (API keys, JWTs, passwords) **before** persistence or fan-out. This keeps cross-scope comms clean while preserving the audit trail.
|
||||
|
||||
# Interface Points
|
||||
|
||||
- **COOEE hooks:** redact at publish and at log-ingest.
|
||||
|
||||
- **WHOOSH channels:** enforce secret scrubbing across planning/implementation/review/testing streams.
|
||||
|
||||
### References
|
||||
|
||||
- [KG] Entities: SHHH, description of its role as a secrets sentinel, file_path: "MASTER PLAN.md"
|
||||
|
||||
## TODO
|
||||
|
||||
- Leader-only mode: Implement SHHH as privileged/elevated code paths under the CHORUS leader (similar to SLURP), protected by elections logic.
|
||||
- Hooks: Add interceptors at COOEE publish/log-ingest and WHOOSH channels to detect/redact sensitive data.
|
||||
- Rules: Provide detection/redaction rulesets with unit and integration tests; profile performance and false positives.
|
||||
- Audit: Implement role-aware scrubbing policies and exception workflows with audit logs.
|
||||
24
docs/Modules/SLURP.md
Normal file
24
docs/Modules/SLURP.md
Normal file
@@ -0,0 +1,24 @@
|
||||
SLURP is a comprehensive system designed for managing, ingesting, normalizing, and structuring knowledge within advanced AI and infrastructure projects, especially those utilizing the UCXL framework. Its primary purpose is to act as a context curator and steward of contextual metadata, enabling semantic use, provenance tracking, decision reasoning, and knowledge synthesis across multiple projects and domains.
|
||||
|
||||
**Curator + policy gate.** SLURP merges RAG outputs, reasoning chains, and API traces into citation-backed bundles, then **filters by role/business-intent** so each agent receives the right slice at the right time — with justifications attached.
|
||||
|
||||
SLURP manages artefacts and decision records (DRs) submitted by AI agents when they complete tasks. It enforces provenance by maintaining citations, reasoning chains, and decision history, and it prepares context for both human interpretation and agent reasoning. SLURP plays a crucial role in content delivery, ensuring the right context reaches the right agent at the right time, while maintaining permissions, need-to-know policies, and audit trails through role-aware filtering and access control.
|
||||
|
||||
Functionally, SLURP interacts with several components. It ingests contextual data from sources like files, APIs, and decision nodes, and manages dependencies, constraints, and metadata associated with files and decisions. It also integrates with tools like BUBBLE for citation tracking and provenance graphs, and supports decision publishing, validation, and recovery. It communicates with other system components such as COOEE for messaging, SHHH for secrets management, and WHOOSH for search indexing, playing an integral role in building and maintaining a structured, traceable, and semantic knowledge graph within the ecosystem.
|
||||
|
||||
# Feeding HMMM safely
|
||||
|
||||
SLURP emits **scope-clean** thought bundles for HMMM so agents can critique/share intermediate reasoning _without_ cross-project contamination; HMMM writes back signed notes that BUBBLE can cite.
|
||||
|
||||
|
||||
References:
|
||||
|
||||
- [KG] Entity 1, Description, and Relationships
|
||||
- [DC] Document 1 content describing SLURP's purpose, features, and interactions
|
||||
|
||||
## TODO
|
||||
|
||||
- Leader-only mode: Keep SLURP embedded in CHORUS and codify leader-only execution via elections; add guards and tests.
|
||||
- BUBBLE integration: Use `/decision/bundle` in the live flow; integrate role filtering and policy enforcement.
|
||||
- DHT writes: Store curated bundles with role-based encryption and verify end-to-end retrieval.
|
||||
- Temporal analysis: Elevate decision-hop analysis from demos/scripts into an operational component with APIs.
|
||||
26
docs/Modules/SWOOSH.md
Normal file
26
docs/Modules/SWOOSH.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# SWOOSH: Current Coordination State Machine
|
||||
|
||||
SWOOSH is the active CHORUS coordinator in this workspace.
|
||||
|
||||
## Runtime Source
|
||||
|
||||
- Code: `/home/tony/chorus/SWOOSH`
|
||||
- Core API: `/home/tony/chorus/SWOOSH/api.go`
|
||||
- Reducer/state machine: `/home/tony/chorus/SWOOSH/reducer.go`
|
||||
- State model: `/home/tony/chorus/SWOOSH/state.go`
|
||||
|
||||
## Current API Surface
|
||||
|
||||
- `POST /transition`
|
||||
- `GET /state`
|
||||
- `GET /health`
|
||||
- `POST /api/v1/opportunities/council` (ResetData-backed council planning)
|
||||
- `GET /api/v1/tasks` (currently 501 by design)
|
||||
|
||||
## Integration Notes
|
||||
|
||||
- CHORUS coordination points now target SWOOSH runtime behavior.
|
||||
- Keep deterministic transition mapping: API handlers submit transitions; reducer is state authority.
|
||||
- For implementation detail and known caveats, refer to:
|
||||
- `/home/tony/chorus/SWOOSH/API_INTEGRATION.md`
|
||||
- `/home/tony/chorus/SWOOSH/RESETDATA_INTEGRATION_PLAN.md`
|
||||
1676
docs/Modules/TaskExecutionEngine.md
Normal file
1676
docs/Modules/TaskExecutionEngine.md
Normal file
File diff suppressed because it is too large
Load Diff
23
docs/Modules/UCXL.md
Normal file
23
docs/Modules/UCXL.md
Normal file
@@ -0,0 +1,23 @@
|
||||
UCXL, or the Unified Context eXchange and Linking, is a comprehensive protocol architecture designed for addressing, managing, and sharing semantic contextual resources within its ecosystem. It establishes a 1:1 mapping between traditional file system structures and contextual metadata, which facilitates knowledge preservation, distributed AI development, and complex context management. UCXL functions as a layered protocol, supporting context messaging, transport, and semantic context sharing through components such as validators and browsers that aid in context resolution and organization.
|
||||
|
||||
The core purpose of UCXL is to enable semantic addressability and logical content organization within a distributed system. It provides a time-aware address space that supports temporal navigation and concept mapping, allowing for precise retrieval and management of context data. UCXL can store and organize data, decisions, and reasoning in organization-based structures, ensuring the persistence and traceability of complex semantic information.
|
||||
|
||||
Key technical features include an address format that uses URI-like schemes (e.g., `ucxl://agent:role@project:task/#/path`) with components like agent, role, project, task, temporal segment, and resource path. It also supports operations such as GET, PUT, POST, DELETE, NAVIGATE, QUERY, and SUBSCRIBE within the UCXI interface, enabling robust context management and dynamic interaction. The protocol is integral to the broader ecosystem, working in conjunction with components like UCXL Browser, Validator, Address Parser, and the supporting infrastructure of the UCXL Foundation, all aimed at enhancing knowledge preservation, decision tracking, and semantic interoperability in distributed AI environments.
|
||||
|
||||
**Scope-aware access:** UCXL addresses bind **agent/role → project/task → temporal slice → path**, so policy can be evaluated _at resolution time_ rather than after retrieval. This is the backbone of our least-privilege, time-bounded lookups.
|
||||
|
||||
**Contracts via addresses.** Teams publish stable UCXL addresses for consuming systems; producers can evolve internals without leaking. Temporal operators (`~~`, `^^`) let consumers pin decisions without chasing moving targets.
|
||||
|
||||
|
||||
|
||||
**References:**
|
||||
|
||||
- [KG] 1 (from Entity: UCXL)
|
||||
- [DC] ucxl_protocol.md
|
||||
|
||||
## TODO
|
||||
|
||||
- Spec + validator: Publish current spec references and ensure the validator enforces them across languages.
|
||||
- Syntax tools: Build out `ucxl-syntax-plugin` (currently empty) and align behavior with the VSCode extension.
|
||||
- Governance: Define address lifecycle/contracts (examples for producers/consumers) and cross-module examples.
|
||||
- Conformance: Create a test harness that runs against RUSTLE, CHORUS, WHOOSH, and the validator for end-to-end checks.
|
||||
1424
docs/Modules/WHOOSH-LEGACY.md
Normal file
1424
docs/Modules/WHOOSH-LEGACY.md
Normal file
File diff suppressed because it is too large
Load Diff
7
docs/Modules/WHOOSH.md
Normal file
7
docs/Modules/WHOOSH.md
Normal file
@@ -0,0 +1,7 @@
|
||||
# WHOOSH (Legacy)
|
||||
|
||||
WHOOSH is no longer the active runtime coordinator in this workspace.
|
||||
|
||||
- Legacy reference: `/home/tony/chorus/docs/Modules/WHOOSH-LEGACY.md`
|
||||
- Current coordinator module: `/home/tony/chorus/docs/Modules/SWOOSH.md`
|
||||
- Current codebase: `/home/tony/chorus/SWOOSH`
|
||||
10
docs/Modules/stripe_skus.csv
Normal file
10
docs/Modules/stripe_skus.csv
Normal file
@@ -0,0 +1,10 @@
|
||||
sku_code,product_name,billing_interval,amount_usd,currency,amount_aud,description,nodes_limit,seats_limit,features,pricing_notes,,,
|
||||
CHR-SMB-START-YR,CHORUS Starter (Annual),year,2500,USD,3825,"1 node, 5 seats, basic UCXL context, local-only, no temporal navigation",1,5,basic_ucxl,local_only,no_temporal_nav,,
|
||||
CHR-SMB-GROWTH-YR,CHORUS Growth (Annual),year,5000,USD,7650,"Up to 3 nodes, 20 seats, basic connectors (files, APIs), temporal navigation enabled",3,20,basic_connectors,temporal_nav,,,
|
||||
CHR-SMB-PRO-YR,CHORUS Pro (Annual),year,10000,USD,15300,"5 nodes, 50 seats, advanced connectors, workflow templates, basic high availability",5,50,advanced_connectors,workflow_templates,basic_ha,,
|
||||
CHR-MM-TEAM-YR,CHORUS Team (Annual),year,15000,USD,22950,"5 nodes, 50 seats, temporal navigation, HA clustering, UCXL federation (single site)",5,50,temporal_nav,ha_clustering,ucxl_federation_single_site,,
|
||||
CHR-MM-BUSINESS-YR,CHORUS Business (Annual),year,30000,USD,45900,"10 nodes, 150 seats, multi-site federation, premium connectors (ERP, CRM, databases), compliance logging",10,150,multi_site_federation,premium_connectors,compliance_logging,,
|
||||
CHR-MM-ADV-YR,CHORUS Advanced (Annual),year,50000,USD,76500,"20 nodes, 250 seats, custom ingestion pipelines, priority support, unlimited agent orchestration",20,250,custom_ingestion,priority_support,unlimited_orchestration,,
|
||||
CHR-ENT-BASE-YR,CHORUS Enterprise Base (Annual),year,100000,USD,153000,"Unlimited nodes within 1 country, 500 seats, all features, dedicated TAM",,500,all_features,dedicated_TAM,country_scope,Unlimited nodes within a single country,
|
||||
CHR-ENT-GLOBAL-YR,CHORUS Enterprise Global (Annual),year,250000,USD,382500,"Unlimited nodes worldwide, unlimited seats, all integrations, advanced security modules, 24/7 SLA",,,all_integrations,advanced_security,247_SLA,global_scope,Unlimited worldwide
|
||||
CHR-ENT-PLAT-CUSTOM,CHORUS Enterprise Platinum (Custom Annual),year,500000,USD,765000,"Everything in Global + on-site engineering, co-development of features, private forks",,,global_plus_onsite_engineering,co_development,private_forks,Custom contract, price typically $500k+
|
||||
|
Reference in New Issue
Block a user