12 council design briefs for distributed OS specification project targeting 1024-node Hopper/Grace/Blackwell GPU cluster with Weka parallel filesystem. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
375 lines
41 KiB
Markdown
375 lines
41 KiB
Markdown
# Council Design Brief: Security Model, Isolation, and Trust
|
||
|
||
**Council ID:** `council-sec`
|
||
**Mission:** Design the complete security architecture for DistOS — spanning hardware-rooted trust, capability-based access control, cryptographic identity, multi-tenant GPU isolation, zero-trust networking, and supply chain integrity — such that every resource access and inter-agent communication is authorised, attributable, and auditable, from silicon to application.
|
||
**UCXL Base Address:** `ucxl://council-sec:*@DistOS:security/*`
|
||
**Agent Count:** ~60
|
||
**Status:** Pre-formation (Constitution Phase)
|
||
**Created:** 2026-02-24
|
||
|
||
---
|
||
|
||
## 1. Scope and Responsibilities
|
||
|
||
Council-sec owns the security model of DistOS in its entirety. This is unusual among the DistOS councils: rather than designing one isolated subsystem, council-sec defines constraints and primitives that every other council must satisfy. It operates both as a producer (defining security APIs, attestation protocols, and key management) and as an authority (issuing security requirements that other councils must incorporate). No other council may finalise an architectural decision that has security implications without ratification from council-sec.
|
||
|
||
The target environment is a 1024-node Hopper/Grace/Blackwell cluster that may host multiple tenants concurrently: research groups, internal teams, and potentially external customers via the KACHING licensing model. The security model must support this multi-tenancy without hardware-level repartitioning of the cluster for each tenant.
|
||
|
||
### In Scope
|
||
|
||
- Capability-based security model: capability taxonomy, attenuation, revocation, and ambient authority elimination
|
||
- Hardware-rooted trust chain: TPM 2.0 attestation, GPU attestation (NVIDIA Hopper/Blackwell Confidential Computing), secure boot verification
|
||
- Cryptographic identity: node identity, agent identity, workload identity; DID-based identity architecture; Ed25519 signing
|
||
- Zero-trust network model: mutual TLS everywhere, SPIFFE/SPIRE workload identity, no implicit trust based on network position
|
||
- Multi-tenant GPU isolation: hardware MIG (Multi-Instance GPU) partitioning, software memory isolation between tenant workloads sharing a GPU, confidential VM boundaries
|
||
- Secure enclave design for sensitive workload data: NVIDIA Hopper Confidential Computing, TEE integration
|
||
- RBAC and ABAC policy engine: role-based and attribute-based access control for DistOS resources
|
||
- Secrets management architecture: secret lifecycle, rotation, distribution to agents, Vault integration
|
||
- Audit logging: tamper-evident audit log, log retention policy, query interface
|
||
- Supply chain security: agent code integrity verification, container image signing, runtime attestation of CHORUS agents
|
||
- Secure communication protocols: channel encryption, replay prevention, message signing for all inter-agent messages
|
||
- Threat model: formal threat model document classifying adversaries and their capabilities
|
||
|
||
### Out of Scope
|
||
|
||
- Specific network transport implementation (council-net owns this; council-sec specifies the security requirements that the transport must satisfy)
|
||
- Memory allocation algorithms (council-mem owns this; council-sec specifies isolation requirements)
|
||
- Fault recovery procedures (council-fault owns this; council-sec specifies that recovery paths must not bypass access controls)
|
||
- Physical security of the data centre nodes (assumed to be operated by resetdata.ai under a separate security programme)
|
||
|
||
---
|
||
|
||
## 2. Research Domains
|
||
|
||
### 2.1 Capability-Based Security
|
||
|
||
Object capabilities (ocaps) represent the cleanest theoretical foundation for a distributed OS security model because they unify authentication and authorisation: possession of a capability is both proof of identity and proof of authorisation. DistOS should adopt ocaps as its primary access control model, supplementing with RBAC/ABAC for human operators who cannot hold unforgeable references.
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- Dennis & Van Horn (1966), "Programming Semantics for Multiprogrammed Computations" — original capability concept; establishes that capabilities are unforgeable tokens conferring the right to perform an operation on a specific object
|
||
- Miller (2006), "Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control" (PhD thesis) — the definitive modern treatment of object capabilities; defines the confinement property, the principle of least authority (POLA), and the E programming language's ocap model; DistOS's capability system should be evaluated against Miller's taxonomy
|
||
- Shapiro et al. (1999), "EROS: A Fast Capability System" — production capability OS demonstrating that capability systems need not sacrifice performance; EROS achieves capability confinement with minimal TCB and provides the performance benchmark DistOS should target
|
||
- Watson et al. (2015), "CHERI: A Hybrid Capability-System Architecture for Scalable Software Compartmentalization" — CHERI extends the ISA with hardware capability registers, eliminating the software overhead of software capability systems; Arm Morello and future RISC-V implementations make CHERI practically relevant; DistOS should evaluate whether CHERI-capable CPUs are present in the target cluster or likely to be in a future revision
|
||
- Watson et al. (2010), "Capsicum: Practical Capabilities for UNIX" — retrofits capability semantics onto FreeBSD without requiring application rewrite; demonstrates that capability confinement is deployable in production UNIX-derived systems; DistOS should evaluate a Capsicum-inspired capability layer for POSIX compatibility
|
||
- Saltzer & Schroeder (1975), "The Protection of Information in Computer Systems" — foundational principles (economy of mechanism, fail-safe defaults, complete mediation, least privilege, separation of privilege, least common mechanism, open design, psychological acceptability); DistOS security design should explicitly cite which principles each mechanism satisfies
|
||
|
||
**Open questions for research phase:** Can DistOS provide capability attenuation (creating a weaker capability from a stronger one) efficiently across the RDMA fabric? How should capability revocation be implemented in a system with thousands of agents holding capabilities to the same resource?
|
||
|
||
### 2.2 Hardware-Rooted Trust
|
||
|
||
In a multi-tenant GPU cluster, tenants cannot trust that the hypervisor or OS they are running on is uncompromised. Hardware attestation provides a root of trust that is independent of software state.
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- Trusted Computing Group (2019), TPM 2.0 Library Specification — TPM 2.0 provides measured boot, remote attestation, and sealed storage; DistOS nodes should use TPM 2.0 to attest to their firmware and software state before joining the cluster; each node's TPM endorsement key provides the hardware root of identity
|
||
- Coker et al. (2011), "Principles of Remote Attestation" — formal model for what attestation proves and what it does not; DistOS's attestation design must be grounded in this model to avoid the common mistake of conflating freshness with integrity
|
||
- Keylime (Schecter et al. 2019 / MIT Lincoln Laboratory) — open-source TPM-based remote attestation for cloud and cluster environments; DistOS should evaluate Keylime as the attestation infrastructure, particularly its continuous attestation (not just boot-time) capability that detects runtime tampering
|
||
- NVIDIA Confidential Computing (Hopper H100, Blackwell B100/B200) — NVIDIA's Confidential Computing architecture provides hardware-isolated GPU execution environments (Confidential VMs) where GPU memory is encrypted and inaccessible to the host hypervisor or other tenants; this is directly relevant to DistOS's multi-tenant workload isolation; the attestation protocol for GPU TEEs is documented in NVIDIA's Hopper Architecture White Paper (2022)
|
||
- NVIDIA Hopper Architecture Technical Overview (2022) — describes the Hardware Security Module (HSM) embedded in H100, the device attestation certificate chain rooted in NVIDIA's Certificate Authority, and the RIM (Reference Integrity Manifest) comparison mechanism; DistOS must integrate with this attestation flow to verify GPU integrity before scheduling confidential workloads
|
||
- Intel TDX (Trust Domain Extensions) and AMD SEV-SNP — CPU-level confidential computing; relevant if Grace CPUs in the Grace-Hopper Superchip include AMD or Intel TEE support (Grace is ARM-based with NVIDIA's own security architecture); the council should verify the actual TEE architecture of the Grace CPU in the target cluster
|
||
- Parno et al. (2011), "Bootstrapping Trust in Commodity Computers" — formal treatment of what a trusted boot sequence proves; DistOS should cite this when justifying its boot attestation design
|
||
|
||
**Open questions for research phase:** Does the Grace-Hopper Superchip's ARM-based Grace CPU support TrustZone or an equivalent TEE? How does NVIDIA's GPU attestation integrate with the TPM on the same node? Is there a unified attestation token that covers both CPU and GPU integrity?
|
||
|
||
### 2.3 Cryptographic Identity
|
||
|
||
Every entity in DistOS — nodes, agents, workloads, council members — needs a stable cryptographic identity that can be verified without a central authority.
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- W3C Decentralised Identifiers (DID) 1.0 (Sporny et al. 2022) — DIDs provide globally unique, cryptographically verifiable identifiers that do not require a centralised registry; DistOS should assign DIDs to nodes (anchored to their TPM endorsement key) and agents (anchored to their signing key); the DID document contains the public key and service endpoints
|
||
- Bernstein et al. (2011), "High-Speed High-Security Signatures" — Ed25519 is the recommended signing algorithm for DistOS: 128-bit security, 64-byte signatures, fast verification; all inter-agent messages, audit log entries, and artifact publications should be signed with Ed25519
|
||
- Boneh et al. (2001), "Short Signatures from the Weil Pairing" (BLS signatures) — BLS signatures support aggregation, making them suitable for the consensus layer where 1000 agents may need to sign a decision; a single aggregated signature is as compact as a single BLS signature regardless of signer count; HotStuff's use of BLS aggregation is directly relevant if council-fault selects HotStuff
|
||
- SPIFFE (Secure Production Identity Framework For Everyone) and SPIRE (SPIFFE Runtime Environment) — SPIFFE provides a standard for workload identity in multi-cloud and cluster environments; SPIRE implements SPIFFE with attestation-based identity issuance (TPM, Kubernetes, cloud provider attestation); DistOS should adopt SPIFFE SVIDs (X.509 or JWT) as the workload identity standard for inter-service mTLS; this aligns with the Google BeyondCorp model
|
||
|
||
**Open questions for research phase:** Should DistOS use a DID method that anchors identity to the local cluster registry (private, no external dependency) or to a public blockchain (tamper-evident but introduces external dependency)? How frequently should agent signing keys rotate, and what is the key rotation protocol during an active task?
|
||
|
||
### 2.4 Zero-Trust Networking
|
||
|
||
The zero-trust model eliminates implicit trust based on network position: a process on the internal network is not trusted more than one on the public internet. Every connection is authenticated, authorised, and encrypted.
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- Kindervag (2010), "No More Chewy Centers: Introducing the Zero Trust Model of Information Security" (Forrester Research) — original zero-trust concept; the core principle for DistOS is that network location grants no authority
|
||
- Ward & Beyer (2014), "BeyondCorp: A New Approach to Enterprise Security" — Google's production zero-trust implementation; device certificates, user certificates, and context-aware access proxy are the model for DistOS's control plane access
|
||
- National Institute of Standards and Technology Special Publication 800-207 (Rose et al. 2020), "Zero Trust Architecture" — formal NIST definition and deployment guidance; DistOS should achieve NIST ZTA Deployment Model 3 (micro-segmented network with resource-level enforcement)
|
||
- SPIFFE/SPIRE (see Section 2.3) — the workload identity infrastructure that makes zero-trust practical at the intra-cluster level; SPIRE's node attestor uses TPM evidence to issue SPIFFE SVIDs without requiring pre-provisioned secrets
|
||
|
||
### 2.5 Multi-Tenant GPU Isolation
|
||
|
||
The most technically novel aspect of DistOS security is providing strong isolation between tenants sharing the same physical GPU, or between a tenant's workload and the DistOS system processes.
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- NVIDIA Multi-Instance GPU (MIG) Technology (NVIDIA, 2020) — hardware partitioning of A100/H100/B100 GPUs into up to 7 isolated instances with separate compute engines, memory, and PCIe bandwidth; MIG provides hardware-enforced isolation between tenants but is coarse-grained (whole-MIG-instance allocation); DistOS must decide when to use MIG versus software-only isolation
|
||
- Jain et al. (2019), "Characterizing and Taming Model Instability Across Edge Devices" — raises GPU memory isolation as a practical problem; co-located inference workloads can observe timing side channels via shared L2 cache; DistOS must evaluate whether MIG eliminates all cache side channels or whether software countermeasures are additionally required
|
||
- NVIDIA Confidential Computing (Hopper/Blackwell) — Confidential VMs with GPU TEE; the GPU encrypts all memory and DMA transfers to a specific VM; no other VM or the host hypervisor can read that memory even with physical access; the trade-off is reduced performance (encryption overhead) and no multi-tenant MIG sharing within a Confidential Computing session
|
||
- Volos et al. (2018), "Graviton: Trusted Execution Environments on GPUs" (research prototype) — proposes a GPU TEE architecture predating NVIDIA's official Confidential Computing; useful background for understanding the design space
|
||
- Dettmers et al. (2022), "LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale" — demonstrates that quantisation affects memory layout in ways that may complicate memory isolation guarantees; DistOS's memory isolation must be quantisation-agnostic
|
||
|
||
**Open questions for research phase:** When a tenant uses MIG, is MIG-level isolation sufficient for DistOS's security model, or must Confidential Computing be mandated? What is the performance overhead of always-on Confidential Computing on H100/B100? Can two tenants share a single MIG instance using software isolation, or does that violate the security model?
|
||
|
||
### 2.6 Access Control: RBAC and ABAC
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- Sandhu et al. (1996), "Role-Based Access Control Models" — four RBAC models (RBAC0 through RBAC3); DistOS should implement RBAC3 (hierarchical roles with constraints) for human operators; role hierarchy maps to organisational structure (admin, operator, researcher, auditor)
|
||
- Hu et al. (2014), NIST Special Publication 800-162, "Guide to Attribute Based Access Control (ABAC)" — ABAC enables fine-grained, context-sensitive policies that RBAC alone cannot express; for example, "a researcher may read checkpoints only for workloads they own and only from nodes in their allocated partition"; DistOS should implement ABAC for programmatic access (agent-to-resource) and RBAC for human-to-system access
|
||
- Open Policy Agent (OPA, Toews et al. 2018) — Rego policy language for ABAC; production deployments in Kubernetes, Istio, and Terraform; DistOS should evaluate OPA as the policy engine for runtime access decisions
|
||
- XACML (eXtensible Access Control Markup Language, OASIS 2013) — formal ABAC policy language with request/response model; heavier than OPA but formally specified; relevant if DistOS requires formal policy verification
|
||
|
||
### 2.7 Secrets Management
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- HashiCorp Vault — production secrets management with dynamic secrets (short-lived credentials issued on demand), secret leasing and renewal, and multiple authentication backends including TPM and Kubernetes; DistOS should adopt Vault or an equivalent for all secret distribution; the key architectural principle is that agents never hold long-lived credentials
|
||
- Bellovin & Cheswick (1994), "Network Firewalls" — historical context for why perimeter-based secrets management (shared network secrets) is insufficient; motivates per-workload dynamic credentials
|
||
- Saltzer (1974), "Protection and the Control of Information Sharing in Multics" — the original minimal TCB argument; DistOS's secrets management TCB should be as small as possible and formally specifiable
|
||
|
||
### 2.8 Supply Chain Security
|
||
|
||
**Key Papers and Systems:**
|
||
|
||
- SLSA (Supply-chain Levels for Software Artifacts) Framework (Google, 2021) — graduated supply chain security model (Levels 1-4); DistOS should require SLSA Level 3 or higher for all CHORUS agent containers: reproducible builds, signed provenance, and verified build platform
|
||
- Sigstore (Carpenter et al. 2021) — keyless signing infrastructure for container images and artifacts using OIDC identity and certificate transparency; DistOS container registry should enforce Sigstore signatures on all agent images
|
||
- in-toto (Torres-Arias et al. 2019), "in-toto: Providing farm-to-table guarantees for bits and bytes" — end-to-end supply chain verification linking source code to deployed artifact through a verifiable chain of custody; DistOS should use in-toto layouts to verify that a running agent binary was built from the expected source commit
|
||
|
||
---
|
||
|
||
## 3. Agent Roles
|
||
|
||
| Role | Count | Responsibilities |
|
||
|------|-------|-----------------|
|
||
| `lead-architect` | 2 | Security model coherence, final ratification authority, liaison to all other councils |
|
||
| `capability-system-designer` | 6 | Object capability taxonomy, attenuation protocol, revocation mechanism, POLA enforcement |
|
||
| `attestation-engineer` | 8 | TPM integration, GPU attestation, Keylime deployment design, attestation protocol specification |
|
||
| `cryptographic-identity-specialist` | 6 | DID architecture, Ed25519 key management, BLS aggregation for consensus, SPIFFE/SPIRE integration |
|
||
| `network-security-engineer` | 6 | mTLS everywhere, SPIFFE SVID issuance, zero-trust policy design, council-net liaison |
|
||
| `gpu-isolation-specialist` | 6 | MIG policy design, Confidential Computing integration, side-channel analysis, multi-tenant boundary verification |
|
||
| `access-control-engineer` | 6 | RBAC/ABAC policy design, OPA integration, policy language selection, role hierarchy |
|
||
| `secrets-manager` | 4 | Vault architecture, dynamic secret design, credential rotation, agent secret distribution |
|
||
| `audit-engineer` | 4 | Tamper-evident audit log design, log retention policy, UCXL-addressed audit query interface |
|
||
| `supply-chain-security-specialist` | 4 | SLSA compliance, Sigstore integration, in-toto layout authoring, container image signing |
|
||
| `threat-model-analyst` | 4 | Formal threat model, adversary capability classification, attack tree analysis, red team scenario authoring |
|
||
| `formal-spec-author` | 4 | TLA+ specifications for capability and attestation state machines, interface with council-verify |
|
||
|
||
**Total: 60 agents**
|
||
|
||
---
|
||
|
||
## 4. Key Deliverables
|
||
|
||
### Phase 1: Research (Days 1-3)
|
||
|
||
| Deliverable | UCXL Address | Description |
|
||
|-------------|-------------|-------------|
|
||
| Capability system survey | `ucxl://council-sec:researcher@DistOS:security/*^/research/capability-system-survey.md` | Analysis of EROS, Capsicum, CHERI, and E-language approaches; applicability to DistOS |
|
||
| GPU attestation survey | `ucxl://council-sec:researcher@DistOS:security/*^/research/gpu-attestation-survey.md` | NVIDIA Confidential Computing, Hopper attestation protocol, MIG isolation boundaries |
|
||
| Zero-trust network survey | `ucxl://council-sec:researcher@DistOS:security/*^/research/zero-trust-network-survey.md` | SPIFFE/SPIRE, BeyondCorp, NIST ZTA model applicability |
|
||
| Multi-tenant isolation survey | `ucxl://council-sec:researcher@DistOS:security/*^/research/multi-tenant-isolation-survey.md` | MIG vs Confidential Computing trade-off analysis, side-channel taxonomy |
|
||
| Threat model draft v0 | `ucxl://council-sec:threat-model-analyst@DistOS:security/*^/research/threat-model-v0.md` | Initial adversary classification and attack surface enumeration |
|
||
|
||
### Phase 2: Architecture (Days 3-6)
|
||
|
||
| Deliverable | UCXL Address | Description |
|
||
|-------------|-------------|-------------|
|
||
| Capability system design | `ucxl://council-sec:capability-system-designer@DistOS:security/*^/architecture/capability-system-design.md` | Capability representation, attenuation protocol, revocation model |
|
||
| Trust chain architecture | `ucxl://council-sec:attestation-engineer@DistOS:security/*^/architecture/trust-chain-architecture.md` | Boot-to-runtime trust chain, TPM+GPU attestation integration |
|
||
| Cryptographic identity design | `ucxl://council-sec:cryptographic-identity-specialist@DistOS:security/*^/architecture/cryptographic-identity-design.md` | DID method, SPIFFE integration, key lifecycle |
|
||
| Zero-trust network design | `ucxl://council-sec:network-security-engineer@DistOS:security/*^/architecture/zero-trust-network-design.md` | mTLS architecture, SVID issuance policy, network microsegmentation |
|
||
| GPU isolation policy | `ucxl://council-sec:gpu-isolation-specialist@DistOS:security/*^/architecture/gpu-isolation-policy.md` | MIG allocation rules, Confidential Computing mandatory use cases |
|
||
| RBAC/ABAC policy design | `ucxl://council-sec:access-control-engineer@DistOS:security/*^/architecture/rbac-abac-policy-design.md` | Role hierarchy, OPA policy language, attribute schema |
|
||
| Threat model v1 (ratified) | `ucxl://council-sec:threat-model-analyst@DistOS:security/*^/architecture/threat-model-v1.md` | Ratified threat model; distributed to all councils as a constraint document |
|
||
| DR-SEC-001: Capability system selection | `ucxl://council-sec:lead-architect@DistOS:security/*^/decisions/DR-SEC-001-capability-system.md` | Decision record: ocap vs RBAC as primary model |
|
||
| DR-SEC-002: GPU isolation boundary | `ucxl://council-sec:lead-architect@DistOS:security/*^/decisions/DR-SEC-002-gpu-isolation-boundary.md` | Decision record: MIG vs Confidential Computing as the primary isolation mechanism |
|
||
| DR-SEC-003: Identity architecture | `ucxl://council-sec:lead-architect@DistOS:security/*^/decisions/DR-SEC-003-identity-architecture.md` | Decision record: DID method selection and SPIFFE integration |
|
||
|
||
### Phase 3: Formal Specification (Days 6-10)
|
||
|
||
| Deliverable | UCXL Address | Description |
|
||
|-------------|-------------|-------------|
|
||
| Capability system TLA+ specification | `ucxl://council-sec:formal-spec-author@DistOS:security/*^/specs/capability-system.tla` | Invariant: no capability grants access beyond its attenuation bound; revocation terminates all derived capabilities |
|
||
| Attestation protocol TLA+ specification | `ucxl://council-sec:formal-spec-author@DistOS:security/*^/specs/attestation-protocol.tla` | Invariant: no unattested node joins the cluster; attestation freshness property |
|
||
| Access control TLA+ specification | `ucxl://council-sec:formal-spec-author@DistOS:security/*^/specs/access-control.tla` | RBAC/ABAC safety: no agent accesses a resource without a valid policy authorisation |
|
||
| Audit log integrity specification | `ucxl://council-sec:formal-spec-author@DistOS:security/*^/specs/audit-log-integrity.tla` | Append-only invariant; tamper detection property |
|
||
| Security requirements matrix | `ucxl://council-sec:lead-architect@DistOS:security/*^/specs/security-requirements-matrix.md` | Table of security requirements per subsystem, distributed to all councils |
|
||
|
||
### Phase 4: Integration (Days 10-12)
|
||
|
||
| Deliverable | UCXL Address | Description |
|
||
|-------------|-------------|-------------|
|
||
| Security constraints for council-net | `ucxl://council-sec:network-security-engineer@DistOS:security/*^/integration/constraints-for-council-net.md` | mTLS requirements, SVID validation, traffic encryption standards |
|
||
| Security constraints for council-mem | `ucxl://council-sec:gpu-isolation-specialist@DistOS:security/*^/integration/constraints-for-council-mem.md` | Memory isolation requirements, confidential memory handling |
|
||
| Security constraints for council-fault | `ucxl://council-sec:lead-architect@DistOS:security/*^/integration/constraints-for-council-fault.md` | Audit requirements for failover, key management for hot standbys |
|
||
| Security constraints for council-sched | `ucxl://council-sec:access-control-engineer@DistOS:security/*^/integration/constraints-for-council-sched.md` | Authorisation for workload placement decisions |
|
||
| Security constraints for council-telemetry | `ucxl://council-sec:audit-engineer@DistOS:security/*^/integration/constraints-for-council-telemetry.md` | Audit log integration, telemetry data classification |
|
||
| KACHING security integration | `ucxl://council-sec:lead-architect@DistOS:security/*^/integration/kaching-security-integration.md` | Security model for multi-tenant billing and licensing via KACHING |
|
||
|
||
### Phase 5: Documentation (Days 12-14)
|
||
|
||
| Deliverable | UCXL Address | Description |
|
||
|-------------|-------------|-------------|
|
||
| Security model reference specification | `ucxl://council-sec:lead-architect@DistOS:security/*^/docs/security-model-reference-spec.md` | Complete security model, normative for all other councils |
|
||
| Operator security guide | `ucxl://council-sec:access-control-engineer@DistOS:security/*^/docs/operator-security-guide.md` | Role provisioning, key rotation procedures, incident response |
|
||
| Tenant security guide | `ucxl://council-sec:gpu-isolation-specialist@DistOS:security/*^/docs/tenant-security-guide.md` | What tenants can and cannot rely on from DistOS's isolation guarantees |
|
||
| Decision archaeology summary | `ucxl://council-arch:narrator@DistOS:decision-history/*^/narratives/council-sec-design-narrative.md` | Human-readable narrative of security architecture decision history |
|
||
|
||
---
|
||
|
||
## 5. Decision Points
|
||
|
||
**DP-SEC-01: Primary Access Control Paradigm**
|
||
Should DistOS adopt object capabilities as the primary access control model, or rely on RBAC/ABAC as the foundation? Object capabilities are more powerful (they enable the principle of least authority at fine granularity and eliminate ambient authority) but require a fundamentally different programming model that may not compose with POSIX APIs. RBAC/ABAC is familiar to operators and integrates with existing tooling but cannot prevent confused deputy attacks without additional mechanism. A hybrid model (ocap for inter-agent communication, RBAC for human operator access, ABAC for resource access policies) may be the practical resolution.
|
||
|
||
**DP-SEC-02: GPU Isolation Mandate**
|
||
Should Confidential Computing be mandatory for all tenant workloads on DistOS, or only for workloads that explicitly request it? Mandatory Confidential Computing eliminates the side-channel risk class entirely and simplifies the isolation proof, but incurs encryption overhead (estimated 5–10% performance penalty on H100 per NVIDIA's published benchmarks) on all workloads including those where confidentiality is not required. Optional Confidential Computing preserves performance for non-sensitive workloads but requires the security model to specify precisely what isolation guarantees apply to non-confidential workloads sharing a GPU.
|
||
|
||
**DP-SEC-03: Identity Architecture Anchoring**
|
||
Should node and agent identity DIDs be anchored to the local cluster registry, a private distributed ledger, or a public blockchain? Local registry is simple and does not introduce external dependencies, but identity certificates cannot be verified by external parties without access to the registry. A private distributed ledger (e.g., Hyperledger Fabric) provides tamper evidence without external dependencies. A public blockchain (e.g., Ethereum ENS) enables cross-cluster identity verification but introduces gas costs and external censorship risk.
|
||
|
||
**DP-SEC-04: Attestation Freshness Policy**
|
||
TPM attestation at boot time proves the initial state is correct. How frequently should DistOS re-attest nodes to detect runtime tampering? Continuous attestation (Keylime's model) provides the strongest guarantee but consumes PCR quote generation resources. Periodic re-attestation (e.g., hourly) is a common compromise. Attestation on workload scheduling (attest before placing any new workload on a node) ensures each placement decision is based on a fresh integrity check. The council must define the re-attestation frequency and the consequences of failed re-attestation.
|
||
|
||
**DP-SEC-05: Capability Revocation Mechanism**
|
||
Capability revocation in a distributed system is hard: there is no central authority that can atomically revoke a capability held by 1000 agents simultaneously. Approaches include: revocable forwarding capabilities (an intermediate object that can be zeroed out), certificate revocation lists with short validity windows, OCSP stapling, and content-addressed capabilities where the capability hash is checked against a revocation set on each use. Each approach makes different trade-offs between revocation latency, check overhead, and system complexity.
|
||
|
||
**DP-SEC-06: Supply Chain Trust Anchor**
|
||
What is the root of trust for CHORUS agent code supply chain? Options: (a) DistOS operator signs all deployed agent images (centralised, strong, single point of trust); (b) each agent image must carry an in-toto layout signed by the agent's developer plus a separate counter-signature by the DistOS operator (two-party authorisation); (c) Sigstore keyless signing using OIDC from the GITEA CI/CD pipeline (ties identity to GITEA account, reduces signing key management burden). The council must define what constitutes a valid supply chain proof and what happens when an agent cannot produce one.
|
||
|
||
**DP-SEC-07: Audit Log Architecture**
|
||
Should DistOS maintain a single authoritative audit log (simple, single point of failure and compromise) or a distributed append-only audit log replicated across multiple nodes (resilient, complex)? The audit log is itself a security-critical component: if an attacker can modify the audit log, they can erase evidence of their actions. The log must be tamper-evident (e.g., using a Merkle tree or hash chain) and durable (writes must survive the failure scenarios defined by council-fault). The council must also define who can query the audit log and with what access controls.
|
||
|
||
**DP-SEC-08: Secrets Distribution Model**
|
||
How are secrets distributed to agents at runtime? Options: (a) push model with Vault agent sidecars injecting secrets at workload startup; (b) pull model where agents authenticate to Vault using their SPIFFE SVID and fetch secrets on demand; (c) sealed secrets baked into attestation-decryptable blobs that can only be read after successful TPM+GPU attestation. The push model is simple but requires the orchestrator to hold secrets momentarily; the pull model is cleaner but adds latency to every secret access; the sealed secret model is the most secure but requires TEE integration for every workload that handles secrets.
|
||
|
||
---
|
||
|
||
## 6. Dependencies on Other Councils
|
||
|
||
Council-sec has a unique dependency relationship: it is primarily a constraint provider to other councils. Every other council must consult council-sec before finalising any architectural decision with security implications.
|
||
|
||
### council-net (Network Stack) — Bidirectional
|
||
Council-sec specifies the security requirements for inter-node and inter-agent communication. Council-net must implement mTLS between all DistOS components, validate SPIFFE SVIDs on every connection, and expose link-level failure notifications without revealing information that could aid traffic analysis. Council-net must not make performance optimisations that bypass encryption or authentication without explicit council-sec approval and a corresponding Decision Record.
|
||
|
||
**Dependency artifact:** `ucxl://council-sec:network-security-engineer@DistOS:security/*^/integration/constraints-for-council-net.md`
|
||
|
||
### council-mem (Distributed Memory) — Constraint Direction
|
||
Council-mem must implement the memory isolation primitives that council-sec's GPU isolation policy requires. Specifically: tenant memory pages must not be readable by other tenants at any point in their lifecycle, including after deallocation; CUDA unified memory must track ownership and enforce capability checks before cross-tenant pointer dereferences; Weka FS namespaces must be enforced at the kernel level so that one tenant's checkpoint data cannot be addressed by another tenant's process.
|
||
|
||
**Dependency artifact:** `ucxl://council-sec:gpu-isolation-specialist@DistOS:security/*^/integration/constraints-for-council-mem.md`
|
||
|
||
### council-fault (Fault Tolerance) — Constraint Direction
|
||
Fault recovery paths are historically a rich source of security vulnerabilities: standby nodes that bypass normal authentication, recovery operations that grant temporary elevated privileges, or checkpoints that carry credentials that expire during recovery. Council-fault must incorporate council-sec's requirements before finalising the recovery state machine. Specifically: standby node promotion must perform full attestation before accepting traffic; recovery operations must not grant capabilities beyond what the original node held; credential rotation must be coordinated with recovery so that recovering nodes receive fresh credentials.
|
||
|
||
**Dependency artifact:** `ucxl://council-sec:lead-architect@DistOS:security/*^/integration/constraints-for-council-fault.md`
|
||
|
||
### council-sched (Process Scheduling) — Constraint Direction
|
||
Scheduling decisions have security implications: placing a tenant's workload on a node with a failed attestation, co-locating competing tenants on the same MIG instance, or allowing a workload to acquire scheduling priority beyond its authorised quota. Council-sched must validate that every placement decision is authorised by council-sec's ABAC policy before executing it.
|
||
|
||
**Dependency artifact:** `ucxl://council-sec:access-control-engineer@DistOS:security/*^/integration/constraints-for-council-sched.md`
|
||
|
||
### council-telemetry (Resource Accounting) — Bidirectional
|
||
Council-telemetry collects resource usage data that includes sensitive information (which tenants are running what workloads, when, and at what cost). This telemetry data must be classified and protected by council-sec's data classification policy. Conversely, council-telemetry's audit log integration depends on council-sec defining the audit log architecture and the access control policy for log queries.
|
||
|
||
**Dependency artifact:** `ucxl://council-sec:audit-engineer@DistOS:security/*^/integration/constraints-for-council-telemetry.md`
|
||
|
||
### council-synth (Inter-Council Synthesis)
|
||
Any conflict between council-sec's security requirements and another council's performance or functionality requirements is escalated to council-synth for formal resolution. Council-sec has a strong voice in these resolutions: a performance optimisation that compromises a security invariant is not an acceptable trade-off without council-sec and council-synth joint sign-off and a corresponding Decision Record documenting the risk acceptance.
|
||
|
||
---
|
||
|
||
## 7. WHOOSH Configuration
|
||
|
||
### Team Formation
|
||
|
||
```json
|
||
{
|
||
"council_id": "council-sec",
|
||
"team_topic": "whoosh.team.distos-council-sec",
|
||
"composition": {
|
||
"lead-architect": 2,
|
||
"capability-system-designer": 6,
|
||
"attestation-engineer": 8,
|
||
"cryptographic-identity-specialist": 6,
|
||
"network-security-engineer": 6,
|
||
"gpu-isolation-specialist": 6,
|
||
"access-control-engineer": 6,
|
||
"secrets-manager": 4,
|
||
"audit-engineer": 4,
|
||
"supply-chain-security-specialist": 4,
|
||
"threat-model-analyst": 4,
|
||
"formal-spec-author": 4
|
||
},
|
||
"total_agents": 60,
|
||
"quorum_policy": {
|
||
"artifact_publication": "simple_majority",
|
||
"security_requirement_issuance": "two_thirds_supermajority",
|
||
"threat_model_ratification": "all_threat_model_analysts_plus_both_lead_architects",
|
||
"constraint_document_publication": "lead_architect_plus_two_thirds",
|
||
"formal_spec_ratification": "formal_spec_authors_plus_one_lead"
|
||
},
|
||
"join_timeout_minutes": 30,
|
||
"inactivity_eviction_minutes": 120,
|
||
"special_policy": "constraint_documents_require_all_councils_acknowledgement_before_phase_3_begins"
|
||
}
|
||
```
|
||
|
||
### Subchannels
|
||
|
||
| Subchannel | Topic Suffix | Purpose |
|
||
|-----------|-------------|---------|
|
||
| Control | `.control` | Role assignments, join/leave events, phase transitions |
|
||
| Research | `.research` | Literature survey coordination, threat modelling, design space exploration |
|
||
| Threat-Model | `.threat-model` | Adversary definition, attack tree construction, risk classification |
|
||
| Architecture | `.architecture` | Security design proposals, capability taxonomy, isolation boundary debate |
|
||
| Formal-Spec | `.formal-spec` | TLA+ review, invariant discussion, council-verify liaison |
|
||
| Inter-Council | `.inter-council` | Constraint document drafting, cross-council security negotiation |
|
||
| Voting | `.voting` | Quorum votes on decision records, security requirement issuance, threat model ratification |
|
||
| Artifacts | `.artifacts` | UCXL artifact announcement references |
|
||
|
||
### Quorum Configuration
|
||
|
||
Security decisions require a higher voting threshold than most councils because a security invariant, once broken in the specification, is difficult to retrofit. The two-thirds supermajority quorum for security requirement issuance ensures that no small subset of agents can weaken the security model. Threat model ratification requires unanimous agreement from all threat model analysts and both lead architects because the threat model is the foundation on which all other security decisions rest; a disagreement here must be resolved, not overridden.
|
||
|
||
---
|
||
|
||
## 8. Success Criteria
|
||
|
||
1. **Threat model completeness:** A ratified threat model document that classifies all DistOS adversary types (including Byzantine CHORUS agents), their capabilities, and their goals; distributed to all councils and formally acknowledged
|
||
2. **Capability system specification:** A TLA+ specification for the capability system with the following invariants model-checked: (a) no agent holds a capability more powerful than it was granted; (b) capability revocation terminates all derived capabilities within a bounded number of rounds
|
||
3. **Attestation coverage:** A design that ensures every node in the cluster is attested before it receives workloads, with a formal specification of the attestation freshness property
|
||
4. **GPU isolation proof:** A written argument (ideally backed by the GPU isolation TLA+ specification) that two tenants co-located on the same physical GPU cannot read each other's memory under the selected isolation mechanism
|
||
5. **Constraint documents delivered:** All constraint documents for dependent councils published to UCXL and formally acknowledged by those councils before Phase 3 begins
|
||
6. **Zero-trust coverage:** Formal verification that there are no network paths in the DistOS design where unauthenticated or unencrypted communication is possible
|
||
7. **Supply chain chain-of-custody:** An in-toto layout that covers the full chain from agent source code commit to running container, with Sigstore signatures at each stage
|
||
8. **Decision record completeness:** All 8 decision points have corresponding Decision Records with alternatives considered and rationale documented, including explicit risk acceptance statements where a security trade-off was made
|
||
|
||
---
|
||
|
||
## 9. Timeline
|
||
|
||
### Phase 1: Research (Days 1-3)
|
||
- Day 1: Activate council, assign roles; threat model analysts begin adversary classification; research surveyors distribute literature assignments across all specialisations
|
||
- Day 2: GPU attestation survey and zero-trust network survey completed; threat model v0 circulated; capability system and multi-tenant isolation surveys in progress
|
||
- Day 3: All research artifacts published; threat model v0 reviewed by full council; research phase closes with threat model v0 and prioritised list of open questions; threat model is immediately shared with all other councils as an informational document
|
||
|
||
### Phase 2: Architecture (Days 3-6)
|
||
- Day 3-4: Capability system design, trust chain architecture, and cryptographic identity design drafted; initial decision records for DP-SEC-01 through DP-SEC-03 circulated
|
||
- Day 4-5: GPU isolation policy, RBAC/ABAC design, and zero-trust network design drafted; inter-council constraint documents begin drafting; threat model v1 drafted for ratification
|
||
- Day 5-6: Threat model v1 ratification vote (requires all threat model analysts and both leads); all DR-SEC-* voted on; constraint documents for all councils published; architecture phase closes
|
||
|
||
### Phase 3: Formal Specification (Days 6-10)
|
||
- Day 6-7: Capability system TLA+ and attestation protocol TLA+ drafted; security requirements matrix published in final form; distributed to all councils as normative constraints
|
||
- Day 7-8: Access control TLA+ and audit log integrity specification drafted; council-verify liaison reviews specifications
|
||
- Day 8-9: All TLA+ specifications submitted to council-verify for model checking; council-sec reviews model checking results and refines specifications as needed
|
||
- Day 9-10: Formal spec phase closes; all TLA+ specifications in final state with model-checking status documented
|
||
|
||
### Phase 4: Integration (Days 10-12)
|
||
- Day 10-11: Cross-council integration review; council-sec reviews each other council's Phase 3 specs for security invariant compliance; issues objections where needed
|
||
- Day 11-12: council-synth handles any conflicts; KACHING security integration document published; final ratification of all constraint documents by both council-sec and recipient councils
|
||
|
||
### Phase 5: Documentation (Days 12-14)
|
||
- Day 12-13: Security model reference specification assembled; operator and tenant security guides drafted
|
||
- Day 13-14: council-arch produces security decision archaeology narrative; final UCXL navigability audit; any outstanding security objections escalated to council-meta for resolution
|