Initial SWOOSH executor and reducer implementation

2025-10-24 18:35:13 +11:00
commit 38707dd182
9 changed files with 1931 additions and 0 deletions
--- a/SWOOSH_PLAN.md
+++ b/SWOOSH_PLAN.md
@@ -0,0 +1,424 @@
+Here’s your **Codex briefing document**, ready to hand over.
+It defines the SWOOSH architecture, constraints, coding style, and behavioral contract in a single authoritative form.
+
+**Download:** [SWOOSH_Codex_Design_Brief.md](sandbox:/mnt/data/SWOOSH_Codex_Design_Brief.md)
+
+---
+
+### 📄 **SWOOSH_Codex_Design_Brief.md**
+
+````markdown
+# SWOOSH — Deterministic Orchestrator (Codex Implementation Brief)
+
+> **Purpose:**  
+> Codex is to implement the SWOOSH orchestrator exactly as described in this brief.  
+> SWOOSH replaces the WHOOSH orchestration module with a deterministic state-machine core built in **Go**, designed for **predictable, audit-ready, chaos-resilient orchestration**.
+
+---
+
+## 1. Overview
+
+SWOOSH governs:
+- Project ingestion (DISCOVER → READY)
+- Council formation (PLAN_ROLES → READY)
+- Environment provisioning (ALLOCATE → READY)
+- Execution (PLAN → WORK → REVIEW → REVERB)
+
+It guarantees:
+- **Deterministic state transitions** (pure reducer pattern)
+- **Idempotent application** of transitions
+- **Write-ahead logging (WAL)** with append-only semantics
+- **Periodic snapshots** and deterministic replay
+- **Hybrid Logical Clock (HLC)** ordering across replicas
+- **Immutable audit evidence** (UCXL + BUBBLE Decision Records)
+
+---
+
+## 2. Language and Runtime
+
+- **Language:** Go 1.22+
+- **Dependencies:** stdlib only, except for BadgerDB (WAL store)
+- **Concurrency:** single-threaded state executor goroutine; NO mutexes on state
+- **Architecture:** pure reducer + guard providers + WAL + snapshot
+- **All state mutation must occur within the executor loop**
+
+---
+
+## 3. Core Contracts
+
+Codex must **not invent or modify** these structs, fields, or method signatures.
+
+### 3.1 OrchestratorState
+
+```go
+type OrchestratorState struct {
+    Meta struct {
+        Version       string
+        SchemaHash    string
+    }
+
+    Boot struct {
+        Licensed      bool
+        LicenseExpiry time.Time
+        NodeID        string
+    }
+
+    Ingestion struct {
+        Phase          string // DISCOVER|FETCH|VALIDATE|INDEX|READY
+        ContentHash    string
+        SourceSet      []string
+        LastError      string
+        Epoch          uint64
+    }
+
+    Council struct {
+        Phase          string // PLAN_ROLES|ELECT|TOOLING_SYNC|READY
+        PlannedRoles   []string
+        Members        []CouncilMember
+        QuorumCertHash string
+        MCPHealthGreen bool
+        Epoch          uint64
+    }
+
+    Environment struct {
+        Phase          string // ALLOCATE|PROVISION|HEALTHCHECK|READY|DEGRADED
+        CapacityOK     bool
+        Health         string // green|amber
+        Resources      []EnvResource
+        Epoch          uint64
+    }
+
+    Execution struct {
+        Phase          string // PLAN|WORK|REVIEW|REVERB
+        ActiveWindowID string
+        BeatIndex      uint64
+        PlanLocked     bool
+        Approvals      uint32
+        Epoch          uint64
+    }
+
+    Control struct {
+        Paused    bool
+        Degraded  bool
+        Recovering bool
+    }
+
+    Policy struct {
+        Quarantined bool
+        Rationale   string
+    }
+
+    HLCLast   string // last applied hlc
+    StateHash string // sha256 of canonical serialization
+}
+````
+
+---
+
+### 3.2 TransitionProposal
+
+```go
+type TransitionProposal struct {
+    CurrentStateHash string   `json:"current_state_hash"`
+    TransitionName   string   `json:"transition"`
+    InputsHash       string   `json:"inputs_hash"`
+    Signer           string   `json:"signer"`
+    IdemKey          string   `json:"idem_key"`
+    HLC              string   `json:"hlc"`
+    WindowID         string   `json:"window_id"`
+    Evidence         []string `json:"evidence"`
+}
+```
+
+---
+
+### 3.3 GuardOutcome
+
+```go
+type GuardOutcome struct {
+    LicenseOK  bool
+    BackbeatOK bool
+    QuorumOK   bool
+    PolicyOK   bool
+    MCPHealthy bool
+    Rationale  []string
+}
+```
+
+Guards are computed **outside** the reducer. Reducer consumes them as verified input.
+
+---
+
+### 3.4 WALRecord
+
+```go
+type WALRecord struct {
+    StatePreHash    string          `json:"state_pre_hash"`
+    StatePostHash   string          `json:"state_post_hash"`
+    Transition      TransitionProposal `json:"transition"`
+    Guard           GuardOutcome    `json:"guard"`
+    AppliedAtHLC    string          `json:"applied_hlc"`
+    AppliedAtUnixNs int64           `json:"applied_unix_ns"`
+    Index           uint64          `json:"index"`
+}
+```
+
+---
+
+## 4. Reducer Contract
+
+```go
+// Reduce applies a validated transition to the given state and returns a new state.
+// It MUST be deterministic, pure, and side-effect free.
+// It MUST NOT perform I/O, logging, or time reads.
+func Reduce(oldState OrchestratorState, t TransitionProposal, g GuardOutcome) (OrchestratorState, error)
+```
+
+* Reducer logic implemented as a **switch on TransitionName**.
+* Guard outcomes are already evaluated.
+* Each case:
+
+  * Updates state fields
+  * Increments relevant Epoch
+  * Computes new StateHash
+* Returns new state, no side-effects.
+
+---
+
+## 5. Executor Model
+
+Codex must implement an **Executor** struct that:
+
+* Runs as a single goroutine
+* Serializes all writes to state
+* Exposes a safe interface for transition submission
+
+```go
+type Executor struct {
+    state     OrchestratorState
+    wal       WALStore
+    snapshot  SnapshotStore
+    applyCh   chan TransitionProposal
+    resultsCh chan ApplyResult
+}
+
+type ApplyResult struct {
+    Success   bool
+    Error     error
+    NewState  OrchestratorState
+    GuardInfo GuardOutcome
+}
+```
+
+### Rules
+
+* No other goroutine may mutate `state`.
+* All writes funnel through `applyCh`.
+* Each accepted transition:
+
+  1. Validate → Guard → Reduce
+  2. Append to WAL (fsync)
+  3. Update canonical state
+  4. Snapshot periodically
+
+---
+
+## 6. WAL Layer
+
+Codex must implement:
+
+```go
+type WALStore interface {
+    Append(record WALRecord) error
+    Replay(fromIndex uint64) ([]WALRecord, error)
+    Sync() error
+    LastIndex() uint64
+}
+```
+
+Implementation details:
+
+* Backed by BadgerDB or flat append-only file.
+* Must fsync after Append or Sync().
+* Replay must yield records in index order.
+* No concurrent writers (single executor owns WAL).
+
+---
+
+## 7. Snapshot Layer
+
+```go
+type Snapshot struct {
+    State           OrchestratorState
+    LastAppliedHLC  string
+    LastAppliedIndex uint64
+}
+
+type SnapshotStore interface {
+    Save(s Snapshot) error
+    LoadLatest() (Snapshot, error)
+}
+```
+
+* Writes as atomic replace (temp → fsync → rename).
+* Snapshot frequency configurable (every N transitions or REVERB phase).
+
+---
+
+## 8. Guard Providers
+
+Codex must define interfaces only (not implementations):
+
+```go
+type GuardProvider interface {
+    Evaluate(t TransitionProposal, s OrchestratorState) (GuardOutcome, error)
+}
+```
+
+Specific guard providers (implemented later):
+
+* KACHINGGuard
+* BACKBEATGuard
+* HMMMQuorumGuard
+* SHHHPolicyGuard
+* MCPHealthGuard
+
+---
+
+## 9. API Layer
+
+Codex will expose a minimal HTTP API:
+
+| Method | Path          | Description                                             |
+| ------ | ------------- | ------------------------------------------------------- |
+| POST   | `/transition` | Submit a TransitionProposal                             |
+| GET    | `/state`      | Return current OrchestratorState (optionally projected) |
+| GET    | `/health`     | Returns readiness, WAL lag, last snapshot info          |
+
+Rules:
+
+* Transition proposals must include `idem_key`, `hlc`, `window_id`.
+* Transition acceptance is serialized by executor.
+* Response must include resulting StateHash or failure reason.
+
+---
+
+## 10. Determinism Requirements
+
+* Reducer must be **referentially transparent**.
+* No access to:
+
+  * System time
+  * Random generators
+  * Environment variables
+  * Network sockets
+* Hashing, sorting, or serialization must use deterministic algorithms.
+* WAL replay must yield identical `StatePostHash` as original execution.
+
+---
+
+## 11. Concurrency Rules
+
+* Only the **executor goroutine** writes to state.
+* All other goroutines are read-only or handle I/O.
+* Mutexes are forbidden on state objects.
+* Communication via typed channels only.
+
+---
+
+## 12. Logging and Metrics
+
+Codex may stub structured logging, but **no I/O inside reducer**.
+
+Metrics hooks (later implemented):
+
+* `state_advance_latency`
+* `retry_count`
+* `quarantine_rate`
+* `recovery_time`
+* `pending_transitions`
+
+---
+
+## 13. Testing Hooks
+
+Codex must include:
+
+* In-memory WAL and snapshot mocks for property tests.
+* `Replay()` function that:
+
+  * Loads snapshot + WAL
+  * Applies transitions deterministically
+  * Returns resulting `StateHash`
+* Determinism test: assert identical hashes across runs.
+
+---
+
+## 14. Generation Scope
+
+Codex is to generate **Go code only**, adhering to this spec.
+Modules to generate (one per iteration):
+
+1. `state.go` — definitions from §3
+2. `reducer.go` — function skeleton for `Reduce()`
+3. `wal.go` — WALStore interface + Badger impl
+4. `snapshot.go` — SnapshotStore interface + atomic file impl
+5. `executor.go` — single-goroutine executor
+6. `api.go` — HTTP handlers for `/transition` and `/state`
+
+---
+
+## 15. Non-Goals
+
+* No database migrations
+* No gRPC
+* No background workers beyond executor
+* No dynamic schema changes
+* No plugins or reflection
+
+---
+
+## 16. Style & Discipline
+
+* Idiomatic Go formatting and error handling.
+* Deterministic serialization (canonical JSON or sorted keys).
+* No use of global variables or shared mutable state.
+* Reducer logic must be explicit and exhaustive — no default fallthrough.
+
+---
+
+## 17. Example Flow (Codex reference)
+
+1. Client submits `TransitionProposal` (e.g. `"ELECT_QUORUM_CERT"`).
+2. API enqueues proposal to executor’s `applyCh`.
+3. Executor calls `GuardProvider.Evaluate()`.
+4. Guards OK → `Reduce(oldState, proposal, guard)` → new state.
+5. Append WALRecord.
+6. Update state.
+7. Broadcast new snapshot hash to metrics/log.
+
+---
+
+## 18. Acceptance Criteria
+
+Codex output is accepted when:
+
+* All interfaces and structs match this doc exactly.
+* Code compiles with `go build ./...`.
+* Reducer is pure, single-threaded, and deterministic.
+* WAL replay reproduces identical state.
+* HTTP API handles at least POST `/transition` and GET `/state`.
+
+---
+
+## 19. Keywords
+
+`deterministic`, `state-machine`, `pure reducer`, `single executor`, `idempotent`, `WAL`, `snapshot`, `HLC`, `auditability`, `no concurrency races`.
+
+---
+
+## 20. End of Brief
+
+> Codex, implement the SWOOSH orchestrator **exactly as specified**,
+> using Go and following the constraints in this document.
+> Do not add dependencies, types, or concurrency patterns not explicitly described here.