WHOOSH/docs/API_SPECIFICATION.md

# WHOOSH API Specification

## Production-Ready Council Formation API

**Status**: Production Ready ✅  
**Version**: v1.0  
**Base URL**: `https://whoosh.chorus.services`

### Overview

WHOOSH provides a production-ready API for autonomous project kickoff through intelligent Council Formation. The API enables automated detection of Design Brief issues, intelligent council composition, and comprehensive project artifact management through the CHORUS ecosystem.

## 🌐 Base Configuration

```yaml
Base URL: https://whoosh.chorus.services
API Version: v1
Authentication: Bearer JWT tokens + Service tokens
Content-Type: application/json
Rate Limiting: 100 requests/minute per IP (configurable)
Security: OWASP Top 10 compliant with comprehensive input validation
```

## 🔐 Authentication & Security

### Production Security Features

- **JWT Authentication**: Role-based access control with configurable expiration
- **Service Tokens**: API automation with scoped permissions  
- **Rate Limiting**: Per-IP rate limiting (100 req/min default, configurable)
- **Input Validation**: Comprehensive sanitization with regex patterns
- **Security Headers**: CSP, X-Frame-Options, X-Content-Type-Options
- **CORS**: Configurable domain restrictions (no wildcards in production)
- **HMAC Webhook Validation**: Enhanced Gitea webhook signature verification

### Authentication Endpoints

**Note**: WHOOSH integrates with existing authentication systems. Direct user authentication handled via Gitea OIDC or external identity providers.

```http
GET  /health                    # No authentication required
GET  /admin/health/details      # JWT required
POST /webhooks/gitea           # HMAC signature validation
```

### Authorization Model

| Role | Permissions | Access |
|------|-------------|--------|
| **Admin** | Full system access | All endpoints, repository management, system configuration |
| **Developer** | Council and artifact management | Council APIs, artifact creation, project analysis |
| **Service** | Automated integration | Task ingestion, webhook processing, status updates |
| **Viewer** | Read-only access | Health endpoints, council status (read-only) |

## 🏛️ Council Management API

### Council Lifecycle

#### List All Councils
```http
GET /api/v1/councils
Authorization: Bearer {jwt_token}

Response 200:
{
  "councils": [
    {
      "id": "council-12345-abcde",
      "project_name": "E-commerce Platform MVP",
      "repository": "https://gitea.chorus.services/startup/marketplace", 
      "status": "active",
      "created_at": "2025-01-12T10:30:00Z",
      "agent_count": 5,
      "artifacts_count": 12
    }
  ],
  "total": 23,
  "page": 1,
  "per_page": 20
}
```

#### Get Council Details
```http
GET /api/v1/councils/{council_id}
Authorization: Bearer {jwt_token}

Response 200:
{
  "id": "council-12345-abcde",
  "project_name": "E-commerce Platform MVP",
  "repository": "https://gitea.chorus.services/startup/marketplace",
  "project_brief": "Build scalable multi-vendor marketplace...",
  "constraints": "Launch within 6 months, Support 1000+ vendors...",
  "tech_limits": "Prefer React/Node.js stack, AWS infrastructure...",
  "targets": "MVP ready in 3 months, Beta launch with 50 vendors...",
  "status": "active",
  "created_at": "2025-01-12T10:30:00Z",
  "updated_at": "2025-01-12T14:45:00Z",
  "task_id": "task-67890-fghij",
  "agents": [
    {
      "id": "agent-001",
      "role_name": "lead-design-director",
      "agent_name": "Design Leadership Agent Alpha",
      "deployed": true,
      "status": "active",
      "service_id": "council-design-director-svc",
      "deployed_at": "2025-01-12T10:45:00Z"
    },
    {
      "id": "agent-002", 
      "role_name": "senior-software-architect",
      "agent_name": "Architecture Agent Beta",
      "deployed": true,
      "status": "active",
      "service_id": "council-architect-svc",
      "deployed_at": "2025-01-12T10:47:00Z"
    }
  ]
}
```

### Council Artifacts Management

#### List Council Artifacts
```http
GET /api/v1/councils/{council_id}/artifacts
Authorization: Bearer {jwt_token}

Response 200:
{
  "council_id": "council-12345-abcde",
  "artifacts": [
    {
      "id": "artifact-001",
      "artifact_type": "kickoff_manifest",
      "artifact_name": "Project Kickoff Manifest",
      "status": "approved",
      "produced_by": "lead-design-director",
      "produced_at": "2025-01-12T12:30:00Z",
      "content_preview": "## Project Overview\nE-commerce platform with multi-vendor support..."
    },
    {
      "id": "artifact-002",
      "artifact_type": "seminal_dr", 
      "artifact_name": "Technical Design Requirements",
      "status": "draft",
      "produced_by": "senior-software-architect",
      "produced_at": "2025-01-12T14:15:00Z",
      "content_json": {
        "architecture_type": "microservices",
        "technology_stack": ["React", "Node.js", "PostgreSQL"],
        "scaling_requirements": "1000+ concurrent users"
      }
    }
  ],
  "total": 12,
  "by_type": {
    "kickoff_manifest": 1,
    "seminal_dr": 1, 
    "scaffold_plan": 1,
    "gate_tests": 2,
    "hmmm_thread": 4,
    "shhh_policy": 1,
    "ucxl_root": 2
  }
}
```

#### Create Council Artifact
```http
POST /api/v1/councils/{council_id}/artifacts
Authorization: Bearer {jwt_token}
Content-Type: application/json

{
  "artifact_type": "scaffold_plan",
  "artifact_name": "Development Scaffold Plan", 
  "content": "## Development Phases\n1. Foundation Setup\n2. Core Features...",
  "content_json": {
    "phases": [
      {"name": "Foundation", "duration_weeks": 2, "dependencies": []},
      {"name": "Core Features", "duration_weeks": 4, "dependencies": ["Foundation"]}
    ],
    "milestones": ["MVP", "Beta", "Production"]
  },
  "produced_by": "senior-software-architect"
}

Response 201:
{
  "id": "artifact-003",
  "council_id": "council-12345-abcde",
  "artifact_type": "scaffold_plan",
  "artifact_name": "Development Scaffold Plan",
  "status": "draft",
  "produced_by": "senior-software-architect",
  "produced_at": "2025-01-12T15:30:00Z",
  "ucxl_address": "ucxl://council@marketplace:kickoff#scaffold_plan/",
  "validation_status": "passed"
}
```

## 📋 Repository Management API

### Repository Monitoring

#### List Monitored Repositories
```http
GET /api/repositories
Authorization: Bearer {jwt_token}

Response 200:
{
  "repositories": [
    {
      "id": "repo-001",
      "full_name": "startup/marketplace",
      "gitea_id": 123,
      "sync_status": "active",
      "last_issue_sync": "2025-01-12T09:00:00Z",
      "created_at": "2025-01-10T14:20:00Z",
      "issues_count": 15,
      "councils_triggered": 1
    }
  ],
  "total": 8,
  "sync_summary": {
    "active": 6,
    "pending": 1, 
    "initial_scan": 1,
    "failed": 0
  }
}
```

#### Add Repository to Monitoring
```http
POST /api/repositories
Authorization: Bearer {jwt_token}
Content-Type: application/json

{
  "full_name": "startup/new-project",
  "gitea_id": 456,
  "enable_monitoring": true,
  "auto_create_labels": true
}

Response 201:
{
  "id": "repo-002",
  "full_name": "startup/new-project", 
  "gitea_id": 456,
  "sync_status": "pending",
  "created_at": "2025-01-12T16:00:00Z",
  "labels_created": ["chorus-entrypoint", "bzzz-task"],
  "next_sync": "2025-01-12T16:05:00Z"
}
```

#### Manual Repository Synchronization
```http
POST /api/repositories/{id}/sync
Authorization: Bearer {jwt_token}
Content-Type: application/json

{
  "full_rescan": false,
  "force_sync": true
}

Response 202:
{
  "sync_job_id": "sync-repo-002-20250112",
  "status": "started",
  "estimated_duration_seconds": 30,
  "full_rescan": false,
  "tracking_url": "/api/repositories/repo-002/sync-status"
}
```

## 📊 Task Management API

### Task Lifecycle

#### List Tasks
```http
GET /api/tasks
Authorization: Bearer {jwt_token}
Query Parameters:
  - status: pending,assigned,completed
  - source_type: gitea_issue,manual
  - limit: 50
  - offset: 0

Response 200:
{
  "tasks": [
    {
      "id": "task-67890-fghij",
      "title": "Design Brief: E-commerce Marketplace Platform",
      "description": "Build a scalable multi-vendor marketplace...",
      "status": "assigned", 
      "priority": "high",
      "source_type": "gitea_issue",
      "external_id": "123",
      "council_id": "council-12345-abcde",
      "created_at": "2025-01-12T10:00:00Z",
      "assigned_at": "2025-01-12T10:30:00Z"
    }
  ],
  "total": 45,
  "status_summary": {
    "pending": 5,
    "assigned": 12,
    "completed": 28
  }
}
```

#### Task Ingestion (Webhook Integration)
```http
POST /api/tasks/ingest
Authorization: Service-Token {service_token}
Content-Type: application/json

{
  "source": "gitea_webhook",
  "repository_id": "repo-001",
  "issue_data": {
    "id": 123,
    "title": "Design Brief: Mobile Banking App",
    "body": "Create a secure mobile banking application...",
    "labels": ["chorus-entrypoint", "priority:high"],
    "created_at": "2025-01-12T16:30:00Z",
    "url": "https://gitea.chorus.services/fintech/mobile-bank/issues/123"
  }
}

Response 201:
{
  "task_id": "task-98765-klmno", 
  "ingestion_status": "processed",
  "council_triggered": true,
  "council_id": "council-98765-zzzzz",
  "processing_time_ms": 245
}
```

### Project Analysis Integration

#### Analyze Project via N8N
```http
POST /api/projects/analyze
Authorization: Bearer {jwt_token}
Content-Type: application/json

{
  "project_name": "Mobile Banking App",
  "repository_url": "https://gitea.chorus.services/fintech/mobile-bank",
  "analysis_type": "comprehensive"
}

Response 202:
{
  "analysis_id": "analysis-789-xyz",
  "status": "processing",
  "n8n_workflow_triggered": true,
  "estimated_completion_minutes": 5,
  "tracking_url": "/api/projects/analyze/analysis-789-xyz/status"
}
```

## 🔍 System Health & Monitoring API

### Health Endpoints

#### Basic Health Check
```http
GET /health

Response 200:
{
  "status": "healthy",
  "timestamp": "2025-01-12T16:45:00Z",
  "version": "1.0.0",
  "uptime_seconds": 86400
}
```

#### Comprehensive System Health
```http
GET /admin/health/details
Authorization: Bearer {jwt_token}

Response 200:
{
  "status": "healthy",
  "timestamp": "2025-01-12T16:45:00Z",
  "components": {
    "database": {
      "status": "healthy",
      "response_time_ms": 12,
      "connection_pool": {
        "active": 8,
        "idle": 2,
        "max": 25
      },
      "last_migration": "006_add_performance_indexes"
    },
    "gitea": {
      "status": "healthy", 
      "response_time_ms": 45,
      "api_version": "1.21.0",
      "repositories_monitored": 8,
      "last_sync": "2025-01-12T16:40:00Z"
    },
    "backbeat": {
      "status": "healthy",
      "response_time_ms": 23,
      "current_beat": "beat-789-active",
      "pulse_id": "pulse-456-running",
      "last_heartbeat": "2025-01-12T16:44:30Z"
    },
    "docker_swarm": {
      "status": "healthy",
      "active_services": 12,
      "running_services": 12,
      "node_count": 3,
      "council_agents_deployed": 15
    }
  },
  "performance_metrics": {
    "request_rate_per_minute": 45,
    "average_response_time_ms": 125,
    "error_rate_percent": 0.02,
    "memory_usage_mb": 512,
    "cpu_usage_percent": 23.5
  }
}
```

### Prometheus Metrics
```http
GET /metrics

Response 200:
# HELP whoosh_councils_total Total number of councils created
# TYPE whoosh_councils_total counter
whoosh_councils_total{status="active"} 15
whoosh_councils_total{status="completed"} 8

# HELP whoosh_agents_deployed Total number of council agents deployed
# TYPE whoosh_agents_deployed gauge
whoosh_agents_deployed 45

# HELP whoosh_http_requests_total Total HTTP requests processed
# TYPE whoosh_http_requests_total counter
whoosh_http_requests_total{method="GET",status="200"} 1234
whoosh_http_requests_total{method="POST",status="201"} 89

# HELP whoosh_database_connections Current database connections
# TYPE whoosh_database_connections gauge
whoosh_database_connections{state="active"} 8
whoosh_database_connections{state="idle"} 2
```

## 🔗 Webhook Integration API

### Gitea Webhook Handler

#### Design Brief Issue Detection
```http
POST /webhooks/gitea
Content-Type: application/json
X-Gitea-Signature: sha256={hmac_signature}
X-Gitea-Event: issues

{
  "action": "opened",
  "repository": {
    "id": 123,
    "full_name": "startup/marketplace",
    "html_url": "https://gitea.chorus.services/startup/marketplace"
  },
  "issue": {
    "id": 456,
    "number": 15,
    "title": "Design Brief: Multi-vendor Marketplace",
    "body": "Build a scalable marketplace platform...",
    "labels": [
      {"name": "chorus-entrypoint"},
      {"name": "priority:high"}
    ],
    "created_at": "2025-01-12T17:00:00Z",
    "html_url": "https://gitea.chorus.services/startup/marketplace/issues/15"
  }
}

Response 200:
{
  "webhook_processed": true,
  "issue_id": 456,
  "council_triggered": true,
  "council_id": "council-new-789-def",
  "task_created": true,
  "task_id": "task-new-456-abc",
  "processing_time_ms": 187
}
```

## 📊 Error Handling & Status Codes

### HTTP Status Codes

| Code | Description | Usage |
|------|-------------|-------|
| `200` | Success | Successful GET requests, updates |
| `201` | Created | Resource creation (councils, artifacts, tasks) |
| `202` | Accepted | Asynchronous operations (analysis, sync) |
| `400` | Bad Request | Invalid request format, validation errors |
| `401` | Unauthorized | Missing or invalid authentication |
| `403` | Forbidden | Valid auth but insufficient permissions |
| `404` | Not Found | Resource does not exist |
| `409` | Conflict | Resource already exists, state conflict |
| `429` | Too Many Requests | Rate limit exceeded |
| `500` | Internal Server Error | Server-side errors |

### Error Response Format

```json
{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid council artifact type",
    "details": {
      "field": "artifact_type", 
      "provided": "invalid_type",
      "allowed": ["kickoff_manifest", "seminal_dr", "scaffold_plan", "gate_tests", "hmmm_thread", "shhh_policy", "ucxl_root"]
    },
    "request_id": "req-12345-abcde",
    "timestamp": "2025-01-12T17:30:00Z"
  }
}
```

### Rate Limiting Headers

```http
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 85
X-RateLimit-Reset: 1641998400
Retry-After: 60
```

## 🔧 Configuration Options

### Environment Variables for API Behavior

```bash
# Rate Limiting
WHOOSH_RATE_LIMIT_REQUESTS=100        # Requests per window
WHOOSH_RATE_LIMIT_WINDOW=60s          # Rate limit window

# Request Validation
WHOOSH_MAX_REQUEST_SIZE=1048576       # 1MB default
WHOOSH_MAX_WEBHOOK_SIZE=10485760      # 10MB for webhooks

# Security  
WHOOSH_CORS_ALLOWED_ORIGINS=https://app.example.com,https://admin.example.com
WHOOSH_JWT_EXPIRATION=24h             # JWT token expiration

# Feature Flags
WHOOSH_FEATURE_LLM_CLASSIFICATION=false    # Enable AI analysis
WHOOSH_FEATURE_ENABLE_ANALYSIS_LOGGING=true # Debug logging
```

## 📚 Integration Examples

### cURL Examples

**Create a new council artifact:**
```bash
curl -X POST "https://whoosh.chorus.services/api/v1/councils/council-123/artifacts" \
  -H "Authorization: Bearer ${JWT_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "artifact_type": "kickoff_manifest",
    "artifact_name": "Project Kickoff Manifest",
    "content": "## Project Overview\nE-commerce platform development..."
  }'
```

**Check system health:**
```bash
curl -X GET "https://whoosh.chorus.services/admin/health/details" \
  -H "Authorization: Bearer ${JWT_TOKEN}"
```

**Monitor council status:**
```bash
curl -X GET "https://whoosh.chorus.services/api/v1/councils" \
  -H "Authorization: Bearer ${JWT_TOKEN}"
```

### JavaScript SDK Example

```javascript
class WHOOSHClient {
  constructor(baseURL, token) {
    this.baseURL = baseURL;
    this.token = token;
  }

  async getCouncils() {
    const response = await fetch(`${this.baseURL}/api/v1/councils`, {
      headers: {
        'Authorization': `Bearer ${this.token}`,
        'Content-Type': 'application/json'
      }
    });
    return response.json();
  }

  async createArtifact(councilId, artifact) {
    const response = await fetch(`${this.baseURL}/api/v1/councils/${councilId}/artifacts`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.token}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify(artifact)
    });
    return response.json();
  }
}

// Usage
const client = new WHOOSHClient('https://whoosh.chorus.services', 'your-jwt-token');
const councils = await client.getCouncils();
```

## 🚀 Future API Extensions (Roadmap)

### Phase 2: Enhanced Collaboration APIs
- **Advanced Consensus**: Voting and decision-making endpoints
- **HMMM Integration**: Structured reasoning chain APIs  
- **Cross-Council Coordination**: Multi-council project management
- **Real-time Communication**: WebSocket events for live updates

### Phase 3: Autonomous Team APIs
- **Dynamic Team Formation**: Ongoing development team creation
- **Agent Self-Organization**: Agent application and matching endpoints
- **Performance Analytics**: Team effectiveness and optimization APIs
- **BZZZ Integration**: Task handoff and continuation management

---

## 📖 Related Documentation

- **[Configuration Guide](CONFIGURATION.md)** - Complete environment variable reference
- **[Database Schema](DATABASE_SCHEMA.md)** - Database structure and relationships
- **[Security Audit](../SECURITY_AUDIT_REPORT.md)** - Security implementation details
- **[Deployment Guide](DEPLOYMENT.md)** - Production deployment procedures

**API Status**: Production Ready ✅  
**Security**: OWASP Top 10 Compliant ✅  
**Documentation**: Complete API Reference ✅