DistOS/councils/01-process-scheduling.md

Council Design Brief: Process Scheduling

Council ID: council-sched Mission: Design the heterogeneous process scheduling subsystem for DistOS, covering GPU kernel dispatch, workload placement across Hopper/Grace/Blackwell accelerators, fair multi-tenant queuing, gang scheduling for distributed training, and energy-aware execution across a 1024-node cluster. UCXL Base Address: ucxl://council-sched:*@DistOS:scheduling/* Agent Count: 80 Status: Constitution Phase — awaiting WHOOSH formation trigger Created: 2026-02-24

---

1. Scope and Responsibilities

council-sched owns the complete specification of the DistOS process and kernel scheduling subsystem. Scope boundaries are defined as follows.

In scope:

• GPU kernel scheduling: dispatch queue management, kernel concurrency, SM partitioning on Hopper (MIG) and Blackwell, and MPS (Multi-Process Service) lifetime management
• CPU-GPU co-scheduling on Grace Superchip (NVLink-C2C coherent interconnect), including unified virtual address space scheduling implications
• Workload placement policy across heterogeneous accelerator types (H100, GH200, B200), including topology-aware affinity scoring
• Fair multi-tenant queuing: priority classes, weighted fair queuing, and quota enforcement
• Gang scheduling for distributed training workloads: all-or-nothing allocation, partial-allotment strategies, and backfill
• Preemption strategies: checkpoint-based preemption, time-sliced preemption, and priority inversion avoidance
• NUMA-aware placement across CPU sockets and GPU memory domains
• GPU memory oversubscription scheduling: eviction policy, swap-to-host, and coordinated demand management with council-mem
• Energy-aware scheduling: frequency/voltage scaling directives, power capping, and thermal headroom management
• Formal specification of the scheduling API surface exposed to user workloads and to other DistOS subsystems

Out of scope (delegated):

• Physical memory allocation and coherence protocol (delegated to council-mem)
• Network topology discovery and network-aware placement data (delegated to council-net; consumed as inputs)
• Metering, cost attribution, and SLO enforcement (delegated to council-telemetry; consumed as outputs)
• Security isolation between tenants at the hardware level (delegated to council-sec; policies consumed as constraints)

---

2. Research Domains

2.1 GPU Kernel Scheduling and SM Partitioning

Survey the CUDA Concurrent Kernel Execution model, the NVIDIA Multi-Instance GPU (MIG) architecture on Hopper (H100), and the NVIDIA Multi-Process Service (MPS). Understand how MIG partitions a GPU into isolated GPC slices with dedicated HBM3 memory, and how MPS enables concurrent kernel execution within a single GPU context without full MIG isolation overhead.

Key materials:

• NVIDIA H100 Architecture Technical Overview (2022) — SM partitioning, GPC layout, NVLink 4.0 bandwidth
• NVIDIA MIG User Guide (CUDA 12.x) — partition profiles (1g.10gb through 7g.80gb), instance isolation, compute and memory capacity tables
• NVIDIA MPS documentation — shared context model, client limit (48 for Hopper), error containment limitations
• AMD ROCm Hardware Abstraction Layer (HAL) source — amdkfd KFD driver, compute queue management, HWS (Hardware Scheduler) in GFX12
• Jain et al., "Fractional GPUs: Software-Based Compute and Memory Bandwidth Reservation for GPUs" (RTAS 2019) — software-defined partitioning as a baseline comparison
• Xiao et al., "AntMan: Dynamic Scaling on GPU Clusters for Deep Learning" (OSDI 2020) — GPU memory and compute sharing for co-located workloads

2.2 Heterogeneous Workload Placement

Develop a placement model that accounts for the distinct performance characteristics of H100 (PCIe/SXM5), GH200 Grace Superchip (NVLink-C2C), and B200 Blackwell (NVLink 5.0 SXM). Each accelerator type has different compute density, memory bandwidth, and interconnect topology.

Key materials:

• Jouppi et al., "In-Datacenter Performance Analysis of a Tensor Processing Unit" (ISCA 2017) — heterogeneous accelerator placement rationale
• Google Borg paper: Verma et al., "Large-scale cluster management at Google with Borg" (EuroSys 2015) — machine heterogeneity handling, alloc sets, resource estimation
• Microsoft Singularity: Qiao et al., "Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning" (OSDI 2021) — goodput-aware placement for training workloads
• Weng et al., "MLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU Clusters" (NSDI 2022) — real-world heterogeneous GPU cluster scheduling from Alibaba

2.3 Fair Multi-Tenant Queuing

Design a multi-level queueing architecture with Dominant Resource Fairness (DRF) semantics extended for GPU resources (SM fraction, HBM bandwidth, NVLink bandwidth as co-dominant resources).

Key materials:

• Ghodsi et al., "Dominant Resource Fairness: Fair Allocation of Multiple Resource Types" (NSDI 2011) — foundational DRF theory
• Apache YARN CapacityScheduler and FairScheduler documentation — hierarchical queues, preemption policy, label-based node targeting
• Apache Mesos DRF implementation — DominantShareAllocator, offer model, role weights
• Kubernetes device plugin framework (k8s.io/device-plugins) — GPU resource advertisement, extended resource scheduling
• Tiresias: Gu et al., "Tiresias: A GPU Cluster Manager for Distributed Deep Learning" (NSDI 2019) — LAS (Least Attained Service) scheduling for DL jobs, 2DAS (2-dimensional attained service)
• Gandiva: Xiao et al., "Gandiva: Introspective Cluster Scheduling for Deep Learning" (OSDI 2018) — time-sliced GPU sharing, job packing, migration

2.4 Gang Scheduling for Distributed Training

Gang scheduling ensures all processes in a distributed training job (e.g., a 512-GPU allreduce ring) are co-scheduled simultaneously. This is critical for preventing head-of-line blocking and deadlock in collective communication.

Key materials:

• Feitelson and Rudolph, "Gang Scheduling Performance Benefits for Fine-Grain Synchronization" (J. Parallel Distrib. Comput., 1992) — foundational gang scheduling analysis
• Rajachandrasekar et al., "A Closer Look at All-Reduce for Deep Learning" (Workshop at SC'19) — collective communication scheduling sensitivity
• Hwang et al., "AFS: Annotation-Free Automatic Sharding for Large Language Models" (ICLR 2023) — pipeline and tensor parallel placement co-scheduling
• Slurm gang scheduling documentation — Oversubscribe=FORCE, GraceTime, slurmctld gang plugin
• Jeon et al., "Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads" (USENIX ATC 2019) — Microsoft Philly cluster trace, gang failure modes

2.5 Preemption Strategies

Survey checkpoint-based, reactive, and time-sliced preemption. Understand the cost of GPU checkpoint (saving SM register file, shared memory, and in-flight DMA) and design preemption policies that bound maximum latency impact.

Key materials:

• Park et al., "Preemptive, Low Latency Datacenter Scheduling via Lightweight Virtualization" (USENIX ATC 2020) — container-level preemption
• Lin et al., "SHEPHERD: Serving DNNs in the Wild" (NSDI 2023) — latency SLO-aware preemption for inference workloads
• NVIDIA CUDA checkpoint/restore (CRIU for GPU) — experimental support status as of CUDA 12.x
• Wang et al., "Achieving Microsecond-Scale Tail Latency Efficiently with Approximate Optimal Scheduling" (SOSP 2017) — preemption-aware scheduling theory

2.6 NUMA-Aware and Topology-Aware Placement

Model the NUMA topology of a Grace Superchip node: 72-core Arm Neoverse V2 CPU, 480 GB LPDDR5X at ~512 GB/s, connected to H100 GPU via NVLink-C2C at 900 GB/s. Compare this with standard SXM5 nodes where CPU-GPU bandwidth is limited to PCIe Gen5 (~128 GB/s bidirectional).

Key materials:

• Linux NUMA scheduling documentation — libnuma, numactl, CFS NUMA balancing (task_numa_migrate)
• NVIDIA GH200 Grace Hopper Superchip Architecture Whitepaper (2023)
• Lepers et al., "Thread and Memory Placement on NUMA Systems: Asymmetry Matters" (USENIX ATC 2015)
• Blagodurov et al., "A Case for NUMA-Aware Contention Management on Multicore Systems" (USENIX ATC 2011)

2.7 GPU Memory Oversubscription

When aggregate GPU memory demand exceeds physical HBM3 capacity, the scheduler must coordinate with council-mem's eviction and tiering policies to decide which kernels to throttle, checkpoint, or migrate.

Key materials:

• Rhu et al., "vDNN: Virtualized Deep Neural Networks for Scalable, Memory-Efficient Neural Network Design" (MICRO 2016) — layer-by-layer activation offload
• Huang et al., "Efficient Large Scale Language Modeling with Mixtures of Experts" (EMNLP 2021) — memory-efficient model parallelism
• NVIDIA Unified Memory documentation (CUDA 12.x) — page migration engine, oversubscription, prefetch advisory API
• Kumar et al., "SwapAdvisor: Push Deep Learning Beyond the GPU Memory Limit via Smart Swapping" (ASPLOS 2020)

2.8 Energy-Aware Scheduling

Design scheduling policies that honour per-node power caps enforced by Baseboard Management Controllers (BMCs) and exploit DVFS (Dynamic Voltage and Frequency Scaling) headroom reported by council-telemetry.

Key materials:

• Patel et al., "Clite: Efficient and QoS Aware Co-location of Multiple Latency-Critical Jobs for Warehouse Scale Computers" (ISCA 2020)
• Lim et al., "Adaptive Power Management for Heterogeneous Multiprocessor SoCs" (ICCAD 2009)
• NVIDIA NVML power management API — nvmlDeviceSetPowerManagementLimit, nvmlDeviceGetCurrentClocksThrottleReasons
• AMD ROCm SMI library — power cap interface (rsmi_dev_power_cap_set)
• RAPL (Running Average Power Limit) interface for Grace CPU power management

---

3. Agent Roles

Total agents: 80

Role	Count	Responsibilities
Lead Architect	2	Overall scheduling architecture decisions, cross-subsystem interface design, DR authorship for major decisions
Kernel Scheduling Researchers	8	CUDA/ROCm scheduler internals, MIG/MPS analysis, SM partitioning survey
Placement Researchers	6	Heterogeneous accelerator placement, topology modelling, workload profiling
Queueing Theory Specialists	5	DRF extensions for multi-resource GPU scheduling, fairness proof sketches
Gang Scheduling Specialists	5	Collective communication scheduling, all-or-nothing allocation protocols
Preemption Specialists	4	Checkpoint protocol design, preemption cost modelling
NUMA/Topology Analysts	4	NUMA topology modelling for Grace Superchip and SXM5 nodes
Energy Efficiency Researchers	4	Power capping, DVFS scheduling integration
Formal Specification Authors	8	TLA+ specification of scheduler state machine, safety and liveness invariants
Architects (sub-component)	10	Propose concrete scheduling algorithm designs for each domain
Internal Reviewers	8	Review research summaries and architecture proposals; cast green/yellow/red votes
Integration Liaisons	6	Interface with council-mem, council-net, council-telemetry, council-sec
Decision Record Authors	5	Author DRs for each resolved decision point; maintain UCXL provenance chain
Adversarial Critics	5	Challenge proposed designs; surface failure modes, starvation scenarios, livelock risks

Role distribution rationale: The large researcher cohort (37 agents covering 8 research domains) reflects the breadth of scheduling literature. The formal specification group (8 agents) is sized to produce parallel TLA+ modules. Internal reviewers and adversarial critics (13 agents combined) ensure no architecture proposal passes without rigorous challenge.

---

4. Key Deliverables

All artifacts are published to the DHT and addressable via UCXL.

4.1 Research Summaries

ucxl://council-sched:researcher@DistOS:scheduling/*^/research/gpu-kernel-scheduling.md
ucxl://council-sched:researcher@DistOS:scheduling/*^/research/heterogeneous-placement.md
ucxl://council-sched:researcher@DistOS:scheduling/*^/research/fair-queuing-drf.md
ucxl://council-sched:researcher@DistOS:scheduling/*^/research/gang-scheduling.md
ucxl://council-sched:researcher@DistOS:scheduling/*^/research/preemption-strategies.md
ucxl://council-sched:researcher@DistOS:scheduling/*^/research/numa-topology.md
ucxl://council-sched:researcher@DistOS:scheduling/*^/research/memory-oversubscription.md
ucxl://council-sched:researcher@DistOS:scheduling/*^/research/energy-aware-scheduling.md

4.2 Architecture Proposals

ucxl://council-sched:architect@DistOS:scheduling/*^/architecture/scheduler-overview.md
ucxl://council-sched:architect@DistOS:scheduling/*^/architecture/mig-mps-partitioning-model.md
ucxl://council-sched:architect@DistOS:scheduling/*^/architecture/placement-scoring-algorithm.md
ucxl://council-sched:architect@DistOS:scheduling/*^/architecture/drf-gpu-extension.md
ucxl://council-sched:architect@DistOS:scheduling/*^/architecture/gang-scheduler-protocol.md
ucxl://council-sched:architect@DistOS:scheduling/*^/architecture/preemption-protocol.md
ucxl://council-sched:architect@DistOS:scheduling/*^/architecture/energy-policy-interface.md

4.3 Decision Records

ucxl://council-sched:architect@DistOS:scheduling/*^/decisions/DR-SCHED-001-partition-model.md
ucxl://council-sched:architect@DistOS:scheduling/*^/decisions/DR-SCHED-002-placement-algorithm.md
ucxl://council-sched:architect@DistOS:scheduling/*^/decisions/DR-SCHED-003-queuing-policy.md
ucxl://council-sched:architect@DistOS:scheduling/*^/decisions/DR-SCHED-004-gang-protocol.md
ucxl://council-sched:architect@DistOS:scheduling/*^/decisions/DR-SCHED-005-preemption-model.md
ucxl://council-sched:architect@DistOS:scheduling/*^/decisions/DR-SCHED-006-energy-interface.md

4.4 Formal Specifications

ucxl://council-sched:verifier@DistOS:scheduling/*^/specs/SchedulerStateMachine.tla
ucxl://council-sched:verifier@DistOS:scheduling/*^/specs/GangScheduler.tla
ucxl://council-sched:verifier@DistOS:scheduling/*^/specs/PreemptionProtocol.tla
ucxl://council-sched:verifier@DistOS:scheduling/*^/specs/FairnessInvariants.tla

4.5 Interface Contracts (for other councils)

ucxl://council-sched:architect@DistOS:scheduling/*^/interfaces/sched-to-mem-contract.md
ucxl://council-sched:architect@DistOS:scheduling/*^/interfaces/sched-to-net-contract.md
ucxl://council-sched:architect@DistOS:scheduling/*^/interfaces/sched-to-telemetry-contract.md
ucxl://council-sched:architect@DistOS:scheduling/*^/interfaces/sched-to-sec-contract.md

---

5. Decision Points

The following are the major architectural questions council-sched must resolve. Each decision must produce a Decision Record with alternatives considered, evidence from research, and rationale for the chosen option.

DP-SCHED-001: Primary Partition Model

Question: Should the scheduler use MIG (hardware-enforced partition isolation), MPS (shared context with software-enforced limits), or a hybrid model where MIG is used for multi-tenant isolation and MPS for co-located jobs within a single tenant's partition?

Factors: isolation strength, context switch overhead, minimum allocation granularity, Blackwell support parity, operational complexity.

Dependency: Decision informs the security isolation model that council-sec will specify.

DP-SCHED-002: Placement Scoring Architecture

Question: Should placement be computed by a centralised scoring service (Borg-style) or a distributed, bid-based negotiation (Mesos offer model)? How does topology affinity (NVLink domain proximity) weight against fairness constraints?

Factors: convergence time at 1024-node scale, fault tolerance of the placement service itself, stale topology information handling, NVLink bandwidth utilisation efficiency.

DP-SCHED-003: Multi-Resource Fairness Extension

Question: DRF was designed for CPU/memory. GPU workloads introduce additional dominant resources: SM fraction, HBM3 bandwidth, NVLink bandwidth, and NVSwitch port occupancy. How many resource dimensions does the fairness model track, and what is the computational complexity of DRF at 80+ resource types?

Factors: implementation tractability, approximation error bounds, interaction with per-tenant quota enforcement.

DP-SCHED-004: Gang Scheduling Protocol

Question: Should gang scheduling use a two-phase reservation (reserve then commit) protocol, speculative allocation with rollback, or a backfill-with-hold strategy? What is the maximum tolerable scheduling delay for a 1024-GPU gang job?

Factors: cluster utilisation impact, deadlock risk in two-phase protocols, interaction with preemption.

DP-SCHED-005: Preemption Granularity

Question: Should preemption operate at kernel-granularity (CUDA stream checkpointing), job-granularity (full process checkpoint via CRIU-for-GPU), or a hybrid that allows kernel preemption for short jobs and process-level preemption for long jobs?

Factors: checkpoint latency (kernel-level: microseconds; process-level: seconds to tens of seconds for large model weights), HBM3 save/restore bandwidth cost, preemption frequency requirements.

DP-SCHED-006: Grace Superchip Scheduling Model

Question: For GH200 nodes with NVLink-C2C, the CPU and GPU share a unified virtual address space. Should the scheduler treat CPU and GPU on a Grace node as a single scheduling unit, or as two separate but affinity-linked resources? How does this interact with the memory model specified by council-mem?

Factors: utilisation efficiency, memory model consistency requirements, API ergonomics for user workloads, migration cost if job is split across a C2C boundary.

DP-SCHED-007: Energy Policy Interface

Question: Should energy-aware scheduling be a first-class scheduling objective (a weight in the placement score function), a hard constraint (power cap as a resource limit), or an advisory mechanism (scheduler receives power headroom hints and applies them at its discretion)?

Factors: SLO compatibility, predictability of power-capped execution, interaction with thermal management, coordination protocol with council-telemetry.

---

6. Dependencies

6.1 What council-sched Needs from Other Councils

Dependency	Source Council	Artifact	Purpose
Memory pressure signals and eviction cost model	council-mem	ucxl://council-mem:architect@DistOS:memory/*^/interfaces/mem-to-sched-contract.md	GPU memory oversubscription scheduling requires eviction cost estimates to avoid scheduling kernels that will immediately thrash
HBM3/DDR5 bandwidth allocation model	council-mem	ucxl://council-mem:architect@DistOS:memory/*^/architecture/bandwidth-allocation.md	Bandwidth is a co-dominant scheduling resource; model must be consistent
Network topology map (NVLink domains, IB fabric)	council-net	ucxl://council-net:architect@DistOS:networking/*^/architecture/topology-model.md	Topology-aware placement requires accurate NVLink domain membership and IB bisection bandwidth per rack
Network bandwidth reservation API	council-net	ucxl://council-net:architect@DistOS:networking/*^/interfaces/net-to-sched-contract.md	Gang scheduling for allreduce jobs requires co-reserving network bandwidth
Resource metering API contract	council-telemetry	ucxl://council-telemetry:architect@DistOS:telemetry/*^/interfaces/telemetry-to-sched-contract.md	Scheduler must feed placement and execution events to telemetry; power headroom signals flow back
Security isolation constraints	council-sec	ucxl://council-sec:architect@DistOS:security/*^/interfaces/sec-to-sched-contract.md	Which tenants may share a GPU partition, MPS context constraints, minimum isolation requirements per tenant class

6.2 What Other Councils Need from council-sched

Consumer Council	Artifact Required	Purpose
council-mem	Placement decisions and GPU assignment map	Memory subsystem needs to know which GPU a job is placed on to configure HBM3 address space and NVLink fabric attachment
council-mem	Preemption events and checkpoint triggers	Memory tiering must snapshot GPU memory on preemption
council-net	Job placement map with NVLink domain assignments	Network subsystem configures NCCL topology files and RDMA QP mappings based on scheduler placement
council-telemetry	Scheduling events stream (enqueue, dequeue, preempt, complete)	Metering and cost attribution require per-job lifecycle events
council-sec	Partition assignment per tenant	Security subsystem enforces isolation based on scheduler-assigned partition IDs
council-verify	TLA+ scheduler state machine spec	Formal verification council model-checks scheduler invariants

---

7. WHOOSH Configuration

# WHOOSH council formation configuration for council-sched
council_id: council-sched
project: DistOS
subsystem: scheduling
gitea_label: chorus-entrypoint
gitea_repo: distos/scheduling

formation:
  target_agents: 80
  min_agents: 60
  wave:
    max_per_wave: 12
    min_per_wave: 6
    period_sec: 30
  placement:
    max_replicas_per_node: 2
  join_stagger_ms: 2000
  bootstrap_peers_min: 5

roles:
  - role: lead-architect
    count: 2
    model: claude-opus-4-6
    priority: high
  - role: researcher
    count: 37
    model: qwen2.5-coder:32b
    priority: normal
    subgroups:
      - tag: kernel-scheduling
        count: 8
      - tag: placement
        count: 6
      - tag: queuing-theory
        count: 5
      - tag: gang-scheduling
        count: 5
      - tag: preemption
        count: 4
      - tag: numa-topology
        count: 4
      - tag: energy
        count: 5
  - role: architect
    count: 10
    model: claude-opus-4-6
    priority: normal
  - role: verifier
    count: 8
    model: deepseek-coder-v2
    priority: normal
  - role: reviewer
    count: 8
    model: claude-opus-4-6
    priority: normal
  - role: integration-liaison
    count: 6
    model: qwen2.5-coder:32b
    priority: normal
  - role: decision-record-author
    count: 5
    model: claude-opus-4-6
    priority: normal
  - role: adversarial-critic
    count: 5
    model: claude-opus-4-6
    priority: normal

subchannels:
  - name: sched-research
    description: "Research discussion and literature synthesis"
    participants: [researcher, lead-architect]
    pubsub: true
  - name: sched-architecture
    description: "Architecture proposal discussion and voting"
    participants: [architect, lead-architect, reviewer, adversarial-critic]
    pubsub: true
  - name: sched-formal-spec
    description: "TLA+ specification authoring and review"
    participants: [verifier, lead-architect, reviewer]
    pubsub: false
  - name: sched-integration
    description: "Cross-council interface negotiation"
    participants: [integration-liaison, lead-architect]
    pubsub: false
  - name: sched-decisions
    description: "Decision record authoring and consensus"
    participants: [decision-record-author, lead-architect, reviewer]
    pubsub: true

quorum:
  # Architecture decisions require supermajority
  architecture_changes:
    policy: supermajority
    threshold: 0.667
    require_domain_role: true
    require_quality_role: true
    beat_minutes: 20
    timeout_beats: 6
  # Research summaries require simple majority
  research_summaries:
    policy: simple_majority
    threshold: 0.5
    require_domain_role: true
    require_quality_role: false
    beat_minutes: 15
    timeout_beats: 4
  # Formal specifications require supermajority with verifier sign-off
  formal_specs:
    policy: supermajority
    threshold: 0.667
    require_domain_role: true
    require_quality_role: true
    require_verifier: true
    beat_minutes: 25
    timeout_beats: 8
  # Cross-council interface contracts require unanimous lead-architect approval
  interface_contracts:
    policy: unanimous
    roles: [lead-architect, integration-liaison]
    beat_minutes: 30
    timeout_beats: 4

gates:
  kaching:
    p95_latency_ms: 250
    max_error_rate: 0.01
  backbeat:
    max_stream_lag: 200
  bootstrap:
    min_healthy_peers: 5
  join:
    min_success_rate: 0.80

review:
  beat_minutes: 20
  quorum:
    total_min: 3
    require_domain_role: true
    require_quality_role: true
  timeout_beats: 6
  no_self_approval: true

---

8. Success Criteria

1. Research completeness: All 8 research domain summaries published to DHT with at least 5 primary references each, reviewed and approved by at least 3 council agents.

2. Architecture coverage: Architectural proposals exist for all 7 major scheduling components (kernel dispatch, placement, queuing, gang, preemption, NUMA, energy). Each proposal addresses the 1024-node scale constraint explicitly.

3. Decision records resolved: All 7 decision points (DP-SCHED-001 through DP-SCHED-007) have a corresponding Decision Record with at least 3 alternatives considered, evidence citations, and a chosen option ratified by council supermajority.

4. Formal specifications: TLA+ specifications for the scheduler state machine, gang scheduling protocol, and preemption protocol. At least 2 of these must have model-checked safety invariants (no starvation for highest-priority jobs, gang deadlock freedom) verified by council-verify.

5. Interface contracts ratified: All 4 interface contracts (to council-mem, council-net, council-telemetry, council-sec) are co-signed by integration liaisons from both councils.

6. UCXL navigability: A human unfamiliar with the project should be able to navigate from any Decision Record to the research summary that motivated it using only UCXL temporal navigation, within 5 hops.

7. Adversarial review pass: Each major architecture proposal has at minimum one adversarial critique documented and a resolution recorded. No proposal advances to formal specification with an unresolved red-vote critique.

---

9. Timeline

Phase 1: Research and Survey (Days 1-3)

Day 1:

• WHOOSH forms council; all 80 agents join via wave deployment
• Researchers self-assign to domain subgroups via sched-research subchannel
• Literature survey begins: GPU kernel scheduling, placement, and fair queuing domains prioritised first (these are on the critical path for Decision Points DP-SCHED-001 through DP-SCHED-003)
• Integration liaisons make initial contact with council-mem and council-net to understand their timelines

Day 2:

• Remaining research domains surveyed: gang scheduling, preemption, NUMA, energy
• Research summaries drafted in parallel across subgroups
• First internal review cycle: reviewers read summaries and post green/yellow/red votes with rationale
• Lead architects synthesise research findings into a preliminary scheduling design space map

Day 3:

• Research summaries revised based on review feedback; final versions published to DHT
• Adversarial critics challenge assumptions in each summary
• Research phase gate: all 8 summaries must achieve simple majority approval before Phase 2 begins
• Preliminary interface contract outlines shared with dependency councils

Phase 2: Architecture and Trade-offs (Days 3-6)

Day 3-4:

• Architects propose concrete options for DP-SCHED-001 (partition model) and DP-SCHED-002 (placement scoring) — these are the highest-dependency decisions
• Adversarial critics engage immediately; all alternatives documented
• DP-SCHED-001 decision record drafted; council votes; DR published

Day 4-5:

• Queuing model (DP-SCHED-003), gang scheduling (DP-SCHED-004), and preemption (DP-SCHED-005) design proposals concurrently authored
• Inter-council synthesis session with council-mem to align on oversubscription and eviction signal interfaces
• Inter-council synthesis session with council-net to align on topology model input format

Day 5-6:

• Grace Superchip model (DP-SCHED-006) and energy interface (DP-SCHED-007) decisions resolved
• All 7 Decision Records drafted; supermajority vote on each
• Architecture overview document assembled from approved DRs
• Architecture phase gate: all DPs resolved before Phase 3 begins

Phase 3: Formal Specification (Days 6-10)

Day 6-7:

• Verifiers begin TLA+ specification of the scheduler state machine based on approved architecture
• Architects continue refining component-level designs to resolve any ambiguities surfaced during spec authoring
• council-verify engaged: share spec drafts for early model-checking feedback

Day 7-8:

• Gang scheduler TLA+ module authored; parallel with preemption protocol spec
• Fairness invariants formally stated: no starvation under bounded load, gang deadlock freedom, DRF monotonicity

Day 8-10:

• Model checking runs submitted to council-verify
• Counterexample analysis: any liveness or safety violations trigger architecture revision and updated DRs
• Formal spec versions pinned in DHT; UCXL addresses published to dependent councils

Phase 4: Integration and Review (Days 10-12)

Day 10-11:

• Interface contracts with council-mem, council-net, council-telemetry, council-sec finalised and submitted for co-signature
• Cross-council integration session: scheduling placement decisions validated against network topology model
• council-synth engaged for any unresolved conflicts with other councils' specifications

Day 11-12:

• Final council review of complete scheduling specification
• Adversarial critics run end-to-end failure scenario analysis
• Any remaining yellow votes addressed with documented mitigations
• Integration review gate: all interface contracts co-signed

Phase 5: Documentation and Narrative (Days 12-14)

Day 12-13:

• Decision record authors produce narrative summaries of the scheduling architecture journey
• council-docs receives the complete scheduling specification package for standardised formatting
• UCXL navigability audit: spot-check 10 random decision paths for completeness

Day 14:

• Final specification published
• council-arch decision archaeology agents generate human-readable narrative of scheduling design evolution
• Council formally dissolved; agents released back to WHOOSH pool