Files
SWOOSH/SWOOSH_PLAN.md
2025-10-24 18:35:13 +11:00

10 KiB
Raw Permalink Blame History

Heres your Codex briefing document, ready to hand over. It defines the SWOOSH architecture, constraints, coding style, and behavioral contract in a single authoritative form.

Download: SWOOSH_Codex_Design_Brief.md


📄 SWOOSH_Codex_Design_Brief.md

# SWOOSH — Deterministic Orchestrator (Codex Implementation Brief)

> **Purpose:**  
> Codex is to implement the SWOOSH orchestrator exactly as described in this brief.  
> SWOOSH replaces the WHOOSH orchestration module with a deterministic state-machine core built in **Go**, designed for **predictable, audit-ready, chaos-resilient orchestration**.

---

## 1. Overview

SWOOSH governs:
- Project ingestion (DISCOVER → READY)
- Council formation (PLAN_ROLES → READY)
- Environment provisioning (ALLOCATE → READY)
- Execution (PLAN → WORK → REVIEW → REVERB)

It guarantees:
- **Deterministic state transitions** (pure reducer pattern)
- **Idempotent application** of transitions
- **Write-ahead logging (WAL)** with append-only semantics
- **Periodic snapshots** and deterministic replay
- **Hybrid Logical Clock (HLC)** ordering across replicas
- **Immutable audit evidence** (UCXL + BUBBLE Decision Records)

---

## 2. Language and Runtime

- **Language:** Go 1.22+
- **Dependencies:** stdlib only, except for BadgerDB (WAL store)
- **Concurrency:** single-threaded state executor goroutine; NO mutexes on state
- **Architecture:** pure reducer + guard providers + WAL + snapshot
- **All state mutation must occur within the executor loop**

---

## 3. Core Contracts

Codex must **not invent or modify** these structs, fields, or method signatures.

### 3.1 OrchestratorState

```go
type OrchestratorState struct {
    Meta struct {
        Version       string
        SchemaHash    string
    }

    Boot struct {
        Licensed      bool
        LicenseExpiry time.Time
        NodeID        string
    }

    Ingestion struct {
        Phase          string // DISCOVER|FETCH|VALIDATE|INDEX|READY
        ContentHash    string
        SourceSet      []string
        LastError      string
        Epoch          uint64
    }

    Council struct {
        Phase          string // PLAN_ROLES|ELECT|TOOLING_SYNC|READY
        PlannedRoles   []string
        Members        []CouncilMember
        QuorumCertHash string
        MCPHealthGreen bool
        Epoch          uint64
    }

    Environment struct {
        Phase          string // ALLOCATE|PROVISION|HEALTHCHECK|READY|DEGRADED
        CapacityOK     bool
        Health         string // green|amber
        Resources      []EnvResource
        Epoch          uint64
    }

    Execution struct {
        Phase          string // PLAN|WORK|REVIEW|REVERB
        ActiveWindowID string
        BeatIndex      uint64
        PlanLocked     bool
        Approvals      uint32
        Epoch          uint64
    }

    Control struct {
        Paused    bool
        Degraded  bool
        Recovering bool
    }

    Policy struct {
        Quarantined bool
        Rationale   string
    }

    HLCLast   string // last applied hlc
    StateHash string // sha256 of canonical serialization
}

3.2 TransitionProposal

type TransitionProposal struct {
    CurrentStateHash string   `json:"current_state_hash"`
    TransitionName   string   `json:"transition"`
    InputsHash       string   `json:"inputs_hash"`
    Signer           string   `json:"signer"`
    IdemKey          string   `json:"idem_key"`
    HLC              string   `json:"hlc"`
    WindowID         string   `json:"window_id"`
    Evidence         []string `json:"evidence"`
}

3.3 GuardOutcome

type GuardOutcome struct {
    LicenseOK  bool
    BackbeatOK bool
    QuorumOK   bool
    PolicyOK   bool
    MCPHealthy bool
    Rationale  []string
}

Guards are computed outside the reducer. Reducer consumes them as verified input.


3.4 WALRecord

type WALRecord struct {
    StatePreHash    string          `json:"state_pre_hash"`
    StatePostHash   string          `json:"state_post_hash"`
    Transition      TransitionProposal `json:"transition"`
    Guard           GuardOutcome    `json:"guard"`
    AppliedAtHLC    string          `json:"applied_hlc"`
    AppliedAtUnixNs int64           `json:"applied_unix_ns"`
    Index           uint64          `json:"index"`
}

4. Reducer Contract

// Reduce applies a validated transition to the given state and returns a new state.
// It MUST be deterministic, pure, and side-effect free.
// It MUST NOT perform I/O, logging, or time reads.
func Reduce(oldState OrchestratorState, t TransitionProposal, g GuardOutcome) (OrchestratorState, error)
  • Reducer logic implemented as a switch on TransitionName.

  • Guard outcomes are already evaluated.

  • Each case:

    • Updates state fields
    • Increments relevant Epoch
    • Computes new StateHash
  • Returns new state, no side-effects.


5. Executor Model

Codex must implement an Executor struct that:

  • Runs as a single goroutine
  • Serializes all writes to state
  • Exposes a safe interface for transition submission
type Executor struct {
    state     OrchestratorState
    wal       WALStore
    snapshot  SnapshotStore
    applyCh   chan TransitionProposal
    resultsCh chan ApplyResult
}

type ApplyResult struct {
    Success   bool
    Error     error
    NewState  OrchestratorState
    GuardInfo GuardOutcome
}

Rules

  • No other goroutine may mutate state.

  • All writes funnel through applyCh.

  • Each accepted transition:

    1. Validate → Guard → Reduce
    2. Append to WAL (fsync)
    3. Update canonical state
    4. Snapshot periodically

6. WAL Layer

Codex must implement:

type WALStore interface {
    Append(record WALRecord) error
    Replay(fromIndex uint64) ([]WALRecord, error)
    Sync() error
    LastIndex() uint64
}

Implementation details:

  • Backed by BadgerDB or flat append-only file.
  • Must fsync after Append or Sync().
  • Replay must yield records in index order.
  • No concurrent writers (single executor owns WAL).

7. Snapshot Layer

type Snapshot struct {
    State           OrchestratorState
    LastAppliedHLC  string
    LastAppliedIndex uint64
}

type SnapshotStore interface {
    Save(s Snapshot) error
    LoadLatest() (Snapshot, error)
}
  • Writes as atomic replace (temp → fsync → rename).
  • Snapshot frequency configurable (every N transitions or REVERB phase).

8. Guard Providers

Codex must define interfaces only (not implementations):

type GuardProvider interface {
    Evaluate(t TransitionProposal, s OrchestratorState) (GuardOutcome, error)
}

Specific guard providers (implemented later):

  • KACHINGGuard
  • BACKBEATGuard
  • HMMMQuorumGuard
  • SHHHPolicyGuard
  • MCPHealthGuard

9. API Layer

Codex will expose a minimal HTTP API:

Method Path Description
POST /transition Submit a TransitionProposal
GET /state Return current OrchestratorState (optionally projected)
GET /health Returns readiness, WAL lag, last snapshot info

Rules:

  • Transition proposals must include idem_key, hlc, window_id.
  • Transition acceptance is serialized by executor.
  • Response must include resulting StateHash or failure reason.

10. Determinism Requirements

  • Reducer must be referentially transparent.

  • No access to:

    • System time
    • Random generators
    • Environment variables
    • Network sockets
  • Hashing, sorting, or serialization must use deterministic algorithms.

  • WAL replay must yield identical StatePostHash as original execution.


11. Concurrency Rules

  • Only the executor goroutine writes to state.
  • All other goroutines are read-only or handle I/O.
  • Mutexes are forbidden on state objects.
  • Communication via typed channels only.

12. Logging and Metrics

Codex may stub structured logging, but no I/O inside reducer.

Metrics hooks (later implemented):

  • state_advance_latency
  • retry_count
  • quarantine_rate
  • recovery_time
  • pending_transitions

13. Testing Hooks

Codex must include:

  • In-memory WAL and snapshot mocks for property tests.

  • Replay() function that:

    • Loads snapshot + WAL
    • Applies transitions deterministically
    • Returns resulting StateHash
  • Determinism test: assert identical hashes across runs.


14. Generation Scope

Codex is to generate Go code only, adhering to this spec. Modules to generate (one per iteration):

  1. state.go — definitions from §3
  2. reducer.go — function skeleton for Reduce()
  3. wal.go — WALStore interface + Badger impl
  4. snapshot.go — SnapshotStore interface + atomic file impl
  5. executor.go — single-goroutine executor
  6. api.go — HTTP handlers for /transition and /state

15. Non-Goals

  • No database migrations
  • No gRPC
  • No background workers beyond executor
  • No dynamic schema changes
  • No plugins or reflection

16. Style & Discipline

  • Idiomatic Go formatting and error handling.
  • Deterministic serialization (canonical JSON or sorted keys).
  • No use of global variables or shared mutable state.
  • Reducer logic must be explicit and exhaustive — no default fallthrough.

17. Example Flow (Codex reference)

  1. Client submits TransitionProposal (e.g. "ELECT_QUORUM_CERT").
  2. API enqueues proposal to executors applyCh.
  3. Executor calls GuardProvider.Evaluate().
  4. Guards OK → Reduce(oldState, proposal, guard) → new state.
  5. Append WALRecord.
  6. Update state.
  7. Broadcast new snapshot hash to metrics/log.

18. Acceptance Criteria

Codex output is accepted when:

  • All interfaces and structs match this doc exactly.
  • Code compiles with go build ./....
  • Reducer is pure, single-threaded, and deterministic.
  • WAL replay reproduces identical state.
  • HTTP API handles at least POST /transition and GET /state.

19. Keywords

deterministic, state-machine, pure reducer, single executor, idempotent, WAL, snapshot, HLC, auditability, no concurrency races.


20. End of Brief

Codex, implement the SWOOSH orchestrator exactly as specified, using Go and following the constraints in this document. Do not add dependencies, types, or concurrency patterns not explicitly described here.