DistOS/councils/11-documentation.md

# Council Design Brief: Documentation & Specification Formatting

---

## Council Identification

| Field | Value |
|-------|-------|
| **Council ID** | `council-docs` |
| **Mission** | Transform the raw technical outputs of all DistOS councils into a coherent, professionally formatted, cross-referenced, and accessible specification suite. Council-docs does not originate technical content — it is the editorial and production team responsible for ensuring that the collective intelligence of every council is rendered legible, navigable, and unambiguous to human readers. |
| **UCXL Base Address** | `ucxl://council-docs:*@DistOS:docs/*^/` |
| **Agent Count** | ~40 agents |
| **Operates From** | Day 3 (continuous intake) through Day 14 (final publication) |

---

## Scope and Responsibilities

Council-docs is the final production stage of the DistOS specification. Its scope is editorial, structural, and representational — not technical. Specifically:

- Consuming outputs from all other councils via UCXL addressing and transforming them into polished specification documents
- Enforcing consistent terminology, formatting, style, and cross-reference standards across all documents
- Generating and maintaining the master glossary and terminology register for the DistOS project
- Producing architecture decision records (ADRs) that capture the rationale for every major architectural choice, sourced from council decision artifacts
- Creating visual representations (architecture diagrams, sequence diagrams, state machine diagrams, dependency maps) that complement written specifications
- Managing document versioning as specifications evolve across the 14-day timeline
- Maintaining bidirectional cross-references between all specification documents, ensuring that a change in one council's output is reflected in all dependent documents
- Producing the final DistOS specification document suite as a coherent, publishable artifact

Council-docs explicitly does **not**:
- Make architectural or technical decisions
- Resolve technical conflicts between councils (that is council-synth's domain)
- Generate original technical analysis

When a document review reveals technical ambiguity, inconsistency, or an apparent gap, council-docs raises a formal editorial query to the originating council and council-synth — it does not resolve the issue itself.

---

## Research Domains

### 1. Standards Document Formatting (IEEE/ISO Style)

The DistOS specification must be formatted to a standard that would be recognisable to engineers familiar with IEEE or ISO technical standards. This means:

**IEEE 1016 (Software Design Descriptions)** — structure and content requirements for software design documentation. Council-docs will use IEEE 1016 as a structural template for the DistOS design specification.

**ISO/IEC 26514 (Systems and software engineering — Requirements for designers and developers of user documentation)** — best practices for technical documentation clarity, completeness, and navigability.

**RFC formatting conventions** — for protocol specifications, the IETF RFC format (with its formal requirement language: MUST, SHOULD, MAY, MUST NOT, SHOULD NOT as per RFC 2119) provides a well-understood convention for precise specification language.

Key references:
- IEEE Std 1016-2009. IEEE Standard for Information Technology — Systems Design — Software Design Descriptions.
- ISO/IEC 26514:2008. Systems and software engineering — Requirements for designers and developers of user documentation.
- Bradner, S. (1997). RFC 2119: Key words for use in RFCs to Indicate Requirement Levels. IETF.
- Nygard, M. T. (2017). "Documenting Architecture Decisions." https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions

### 2. Architecture Decision Records

An ADR captures: the context that required a decision, the options considered, the decision made, and the consequences. For DistOS, every major architectural decision made by any council must be captured as an ADR.

Council-docs is responsible for:
- Extracting decision artifacts from UCXL addresses across all councils
- Reformatting them as coherent, numbered ADRs (ADR-001, ADR-002, etc.)
- Linking each ADR to the council-arch narratives that explain the decision chain
- Maintaining an ADR status register (Proposed / Accepted / Deprecated / Superseded)

**ADR format for DistOS:**

```
# ADR-NNN: [Title]

## Status
[Proposed | Accepted | Deprecated | Superseded by ADR-NNN]

## Context
[What situation required a decision? What forces were at play?]

## Decision
[What was decided?]

## Consequences
[What are the results of this decision? What becomes easier or harder?]

## Source
[UCXL addresses of the artifacts from which this ADR was derived]

## Councils Involved
[Which councils contributed to or were affected by this decision?]
```

Key references:
- Nygard, M. T. (2011). "Documenting Architecture Decisions." Cognitect Blog.
- Tyree, J., & Akerman, A. (2005). "Architecture Decisions: Demystifying Architecture." IEEE Software 22(2).
- Richards, M., & Ford, N. (2020). Fundamentals of Software Architecture. O'Reilly Media. Chapter 19: Making Architecture Decisions.

### 3. API Reference Documentation Generation

For all APIs produced by council-api, council-docs will produce structured reference documentation following OpenAPI 3.x conventions where applicable, and custom structured formats for non-REST interfaces (GPU kernel APIs, consensus protocol messages, filesystem interfaces).

Each API reference entry must include:
- Endpoint or function signature with all parameters
- Parameter types, constraints, and defaults
- Pre-conditions and post-conditions (sourced from council-verify formal specs)
- Error codes and their meaning
- Example requests and responses
- Security requirements
- Rate limits and performance characteristics where specified

**OpenAPI 3.1** — the current standard for REST API documentation. Council-docs will produce OpenAPI specs for any HTTP-based DistOS management interfaces.

**Swagger/Redoc** — tooling for rendering OpenAPI specs into navigable HTML documentation.

Key references:
- OpenAPI Initiative. (2021). OpenAPI Specification v3.1.0. https://spec.openapis.org/oas/v3.1.0
- Jacobson, D., Brail, G., & Woods, D. (2011). APIs: A Strategy Guide. O'Reilly Media.

### 4. Diagram Generation Standards

Technical diagrams must convey system structure and behaviour precisely and consistently. Council-docs Diagram Specialists will use:

**C4 Model (Context, Container, Component, Code)** — for architecture overview diagrams at four levels of abstraction. Simon Brown's C4 model provides a structured hierarchy that prevents the common failure of a single "big ball of mud" architecture diagram.

**UML Sequence Diagrams** — for inter-council and inter-component communication flows. Especially important for documenting GPU job submission flows, consensus protocol message exchanges, and checkpoint/recovery sequences.

**UML State Machine Diagrams** — for component lifecycle documentation (GPU context states, scheduler job states, filesystem handle states).

**Graphviz / Mermaid** — as the diagramming toolchain. Mermaid's text-based syntax integrates with Markdown documentation; Graphviz provides more precise layout control for complex dependency graphs.

**Decision tree diagrams** — for documenting fault recovery decision logic, sourced from council-qa chaos playbooks and council-fault specifications.

Key references:
- Brown, S. (2018). The C4 Model for Visualising Software Architecture. Leanpub.
- Fowler, M. (2003). UML Distilled: A Brief Guide to the Standard Object Modeling Language. Addison-Wesley.
- OMG. (2017). Unified Modeling Language Specification Version 2.5.1.

### 5. Cross-Reference Management

In a specification suite produced by 10+ councils, cross-references between documents are as important as the documents themselves. A memory manager specification that references a scheduler guarantee must point to the specific version of that guarantee that was current when the memory spec was written.

Council-docs Cross-Referencers will:
- Maintain a master cross-reference registry mapping logical references to UCXL-versioned artifacts
- Flag dangling references (references to artifacts that have been superseded or deleted)
- Ensure that every inter-council dependency named in design briefs has a corresponding cross-reference in the final specification
- Produce a dependency map showing which specification sections depend on which council outputs

**UCXL temporal versioning** will be used to pin cross-references: when document A references a claim from document B, the reference includes the UCXL temporal address of the specific version of B that was current at the time of reference.

Example:
```
The memory isolation guarantee stated in [MEM-ISOLATION-V3]
(ucxl://council-mem:allocator@DistOS:mem/*~3/spec/isolation-guarantee)
depends on the scheduler preemption guarantee in [SCHED-PREEMPT-V2]
(ucxl://council-sched:scheduler@DistOS:sched/*~2/spec/preemption).
```

### 6. Terminology Standardisation

Technical specifications fail when different sections use different words for the same concept, or the same word for different concepts. For a 1024-node GPU cluster OS, the terminology surface is large.

Council-docs Terminology Guardians will maintain a canonical glossary that:
- Defines every technical term used across all council outputs
- Flags and resolves synonyms (is it "job", "task", "workload", or "kernel"?)
- Maintains provenance for each definition (which council introduced it, when, and in what context)
- Enforces consistent usage through editorial review

The glossary is a living document; it is updated continuously as new terms emerge from council outputs.

### 7. Accessibility and Readability of Technical Content

The DistOS specification should be readable by:
- GPU cluster architects who have not read the detailed formal specifications
- Systems engineers implementing DistOS components
- Future maintainers who need to understand why specific decisions were made
- Auditors reviewing security and compliance properties

Council-docs applies the following readability standards:
- Flesch-Kincaid readability targets for narrative sections (technical precision does not require unreadable prose)
- Consistent use of active voice in procedural sections
- Progressive disclosure: overview sections before detail sections in every major document
- Every document has a clearly stated purpose, audience, and scope in its opening section

---

## Agent Roles

| Role | Count | Responsibilities |
|------|-------|-----------------|
| **Editors** | 15 | Review all council outputs for clarity, completeness, and internal consistency; rewrite for precision without altering technical content; raise editorial queries to originating councils |
| **Formatters** | 10 | Enforce consistent document structure, heading hierarchy, numbering schemes, table formatting, and code block conventions across all outputs; apply IEEE/ISO and ADR templates |
| **Cross-Referencers** | 5 | Maintain master cross-reference registry; validate all UCXL-pinned references; flag dangling or outdated references; produce dependency maps |
| **Diagram Specialists** | 5 | Generate architecture diagrams (C4), sequence diagrams (UML), state machine diagrams, and decision trees from council outputs; maintain diagram source files in version control |
| **Terminology Guardians** | 5 | Maintain master glossary; audit all council outputs for terminology consistency; propose canonical definitions; resolve synonym conflicts with originating councils |

**Total: 40 agents**

---

## Key Deliverables

| Deliverable | UCXL Address | Due Phase |
|-------------|--------------|-----------|
| Documentation Standards Guide | `ucxl://council-docs:formatter@DistOS:docs/*^/spec/documentation-standards` | Phase 1 |
| Master Glossary (v0, seed terms) | `ucxl://council-docs:terminology-guardian@DistOS:docs/*^/glossary/master-v0` | Phase 2 |
| ADR Template and Register | `ucxl://council-docs:formatter@DistOS:docs/*^/template/adr-template` | Phase 2 |
| Cross-Reference Registry (live) | `ucxl://council-docs:cross-referencer@DistOS:docs/*^/registry/cross-references` | Phase 2 (continuous) |
| Architecture Overview Document | `ucxl://council-docs:editor@DistOS:docs/*^/spec/architecture-overview` | Phase 3 |
| C4 Architecture Diagrams (all levels) | `ucxl://council-docs:diagram-specialist@DistOS:docs/*^/diagram/c4-architecture` | Phase 3 |
| API Reference Documentation | `ucxl://council-docs:editor@DistOS:docs/*^/reference/api-reference` | Phase 3 |
| Subsystem Sequence Diagrams | `ucxl://council-docs:diagram-specialist@DistOS:docs/*^/diagram/sequence-diagrams` | Phase 3 |
| ADR Compilation (all councils) | `ucxl://council-docs:formatter@DistOS:docs/*^/adr/complete-register` | Phase 4 |
| Master Glossary (final) | `ucxl://council-docs:terminology-guardian@DistOS:docs/*^/glossary/master-final` | Phase 4 |
| Formal Specification Suite (assembled) | `ucxl://council-docs:editor@DistOS:docs/*^/spec/formal-spec-suite` | Phase 5 |
| State Machine Diagram Set | `ucxl://council-docs:diagram-specialist@DistOS:docs/*^/diagram/state-machines` | Phase 4 |
| QA Test Report Formatted Edition | `ucxl://council-docs:formatter@DistOS:docs/*^/report/qa-formatted` | Phase 5 |
| Final DistOS Specification Document | `ucxl://council-docs:editor@DistOS:docs/*^/spec/distos-specification-v1` | Phase 5 |

---

## Decision Points

### DD-01: Primary Document Format Standard
**Question:** What is the canonical format for the DistOS specification suite?

**Options:**
- A. Markdown with Mermaid diagrams — maximum portability, version-control friendly, renders on Gitea
- B. LaTeX — maximum typographic control, produces publication-quality PDFs, standard for academic specifications
- C. AsciiDoc — richer than Markdown, less complex than LaTeX, with strong tooling (Antora, Asciidoctor)
- D. IEEE-formatted Word/ODT — directly compatible with standards submission processes

**Implications:** Option A is most compatible with the CHORUS/UCXL toolchain and Gitea workflow. Option B produces the highest-quality PDF output but requires LaTeX toolchain expertise. The choice affects how cross-references and diagrams are managed throughout the project.

### DD-02: ADR Ownership Model
**Question:** Who owns the content of each ADR — the council that made the decision, or council-docs?

**Options:**
- A. Council-docs owns all ADRs; originating councils provide raw decision artifacts only
- B. Originating councils own ADR content; council-docs applies formatting only
- C. Co-ownership model: originating council writes ADR draft, council-docs editors revise for clarity, council-arch validates narrative accuracy

**Implications:** Option C is highest quality but requires the most coordination overhead. The council-arch dependency is critical here: council-arch's UCXL traversal output is the primary source of decision chain data.

### DD-03: Glossary Conflict Resolution Process
**Question:** When two councils use the same term to mean different things, who resolves the conflict?

**Options:**
- A. Council-docs Terminology Guardians have final authority on terminology
- B. The council that first defined the term retains its definition; later councils must use different terms
- C. Terminology conflicts escalate to council-synth for resolution; council-docs documents the resolution

**Implications:** Option C correctly positions council-synth as the cross-council arbitrator, but adds coordination overhead. Option A risks terminology decisions made by editors rather than domain experts.

### DD-04: Diagram Fidelity vs. Speed Trade-off
**Question:** For architecture diagrams, should council-docs produce high-fidelity manually-reviewed diagrams or automated/generated diagrams from machine-readable sources?

**Options:**
- A. All diagrams hand-crafted by Diagram Specialists for maximum clarity and accuracy
- B. All diagrams auto-generated from structured UCXL metadata with manual review
- C. Hybrid: C4 overview diagrams are hand-crafted; detailed sequence and state diagrams are auto-generated

**Implications:** Option B produces diagrams that stay automatically consistent with the underlying specification as it changes. Option A produces higher-quality diagrams that may drift from the specification if not carefully maintained.

### DD-05: Document Versioning Strategy
**Question:** How are document versions managed as specifications evolve across the 14-day timeline?

**Options:**
- A. Semantic versioning (v0.1, v0.2 ... v1.0) with UCXL temporal addressing for history
- B. Date-stamped snapshots only, with UCXL providing all history navigation
- C. Git-based versioning with tags at each phase boundary, UCXL addresses pointing to git refs

---

## Dependencies on Other Councils

| Council | Dependency Type | What Council-Docs Needs |
|---------|----------------|------------------------|
| **council-arch** | Critical upstream | Decision narratives and human-readable summaries to source ADR content; UCXL traversal paths for cross-reference validation |
| **council-synth** | Critical upstream | Synthesis decisions (especially conflict resolutions) which form the spine of the architecture document |
| **council-verify** | Upstream | Formal specification documents to edit and format; invariant definitions for the glossary |
| **council-api** | Upstream | API definitions to transform into formatted API reference documentation |
| **council-sched** | Upstream | Scheduler specification documents; sequence diagrams source data |
| **council-mem** | Upstream | Memory manager specification documents; state machine source data |
| **council-sec** | Upstream | Security model documents; threat model formatted edition |
| **council-net** | Upstream | Network layer specification documents |
| **council-fs** | Upstream | Filesystem integration specification documents |
| **council-qa** | Upstream | QA test reports; defect register; chaos playbooks for inclusion in operational documentation |
| **ALL councils** | Continuous | Terminology submissions for the master glossary; editorial query responses |

**Special relationship with council-arch:** Council-docs depends on council-arch not just for content but for interpretive context. When editors encounter a decision that seems unmotivated or a specification section that lacks clear rationale, they query council-arch's narrative archive before raising a formal editorial query to the technical council. This prevents unnecessary interruptions to technical councils with questions that the archaeology record already answers.

---

## WHOOSH Configuration

```yaml
council: council-docs
whoosh:
  formation:
    strategy: workflow-pipeline
    # Documentation follows a pipeline: intake -> edit -> format -> cross-ref -> publish
    stages:
      - name: intake
        roles: [editor]
        trigger: new-artifact-available
        description: Editors monitor UCXL for new artifacts from all councils
      - name: edit
        roles: [editor, terminology-guardian]
        trigger: intake-complete
        description: Editors revise for clarity; Terminology Guardians check glossary compliance
      - name: format
        roles: [formatter]
        trigger: edit-complete
        description: Formatters apply templates and structural standards
      - name: cross-reference
        roles: [cross-referencer]
        trigger: format-complete
        description: Cross-Referencers validate and register all inter-document references
      - name: diagram
        roles: [diagram-specialist]
        trigger: parallel-with-format
        description: Diagram Specialists generate/update visual representations
      - name: publish
        roles: [formatter, editor]
        trigger: cross-reference-complete
        description: Final document published to UCXL

  quorum:
    editorial-query: 2/5-editors    # Two editors must agree a query is warranted before sending
    terminology-decision: 3/5-guardians-or-escalate  # Three guardians or escalate to council-synth
    document-acceptance: editor + formatter + cross-referencer  # All three roles sign off per document

  subchannels:
    - id: artifact-intake
      description: New artifacts arriving from all councils
      subscribers: [council-docs-editors]
      retention: full-history
    - id: editorial-queries
      description: Formal editorial queries sent to originating councils
      subscribers: [all-councils, council-arch]
      retention: full-history
    - id: glossary-updates
      description: New and revised glossary entries broadcast to all councils
      subscribers: [all-councils]
      retention: full-history
    - id: document-published
      description: Notification when a document section is published
      subscribers: [all-councils, council-arch, council-synth]
    - id: cross-reference-warnings
      description: Dangling or stale cross-references requiring attention
      subscribers: [council-synth, council-arch, originating-council]

  continuous_monitoring:
    # council-docs monitors all councils' UCXL streams for new artifacts
    monitor_pattern: "ucxl://*:*@DistOS:*/*^/*"
    trigger: on-new-artifact
    action: queue-for-intake
```

---

## Success Criteria

Council-docs execution is successful when:

1. **Complete coverage:** Every formal technical output from every other council has a corresponding edited, formatted, and cross-referenced document in the DistOS specification suite.

2. **Terminology consistency:** The master glossary contains definitions for every distinct technical term used across all specification documents. Zero instances of unresolved synonym conflicts in published documents.

3. **Cross-reference integrity:** All inter-document cross-references resolve to valid UCXL-addressed artifacts. Zero dangling references in the final published suite.

4. **ADR completeness:** An ADR exists for every major architectural decision identified by council-arch's decision traversal. Every ADR links to its UCXL source artifacts.

5. **Diagram coverage:** C4 context, container, and component diagrams exist for all major DistOS subsystems. Sequence diagrams exist for all major inter-component workflows. State machine diagrams exist for all major component lifecycles.

6. **Readability standard:** A systems engineer with GPU cluster experience but no prior DistOS exposure can read the architecture overview document and correctly answer questions about the high-level design without consulting supplementary materials.

7. **Format compliance:** All documents conform to the agreed formatting standard (IEEE-aligned Markdown or equivalent) without structural inconsistencies.

8. **Version integrity:** Document versions are correctly pinned to UCXL temporal addresses. A reader can navigate to any historical version of any document by following the UCXL version chain.

9. **Editorial queries resolved:** All formal editorial queries raised by council-docs have been responded to and resolved by the relevant technical councils.

10. **Final specification published:** The complete DistOS Specification v1.0 document suite is published as a unified, navigable artifact by Day 14.

---

## Timeline Mapping

| Phase | Days | Council-Docs Activities |
|-------|------|------------------------|
| **Phase 1: Research** | 1–3 | Establish documentation standards guide; define ADR template; seed master glossary with terms from DistOS design briefs; configure UCXL monitoring for artifact intake; coordinate with council-arch on narrative format integration |
| **Phase 2: Architecture** | 3–6 | Begin editing first architecture outputs as they arrive from council-synth and subsystem councils; build cross-reference registry; produce ADRs from early architecture decisions; create initial C4 context diagram; first glossary broadcast to all councils |
| **Phase 3: Formal Specification** | 6–10 | High-throughput editing and formatting as formal specs arrive from council-verify and all subsystem councils; API reference documentation production; sequence and state machine diagram generation; continuous cross-reference validation and glossary updates; ADR compilation accelerates |
| **Phase 4: Integration** | 10–12 | Assemble subsystem documents into coherent specification suite; resolve all cross-reference issues; finalise ADR register; incorporate QA test report formatted edition; complete all diagrams; master glossary final review |
| **Phase 5: Documentation** | 12–14 | Final editorial pass on all documents; publish DistOS Specification v1.0; produce publication-ready version of the complete specification suite; validate all UCXL addresses in published documents; confirm archaeology readability test with council-arch |

---

*Council Design Brief v1.0 — DistOS Project — council-docs*
*Generated: 2026-02-24*