feat: Production readiness improvements for WHOOSH council formation

Major security, observability, and configuration improvements:

## Security Hardening
- Implemented configurable CORS (no more wildcards)
- Added comprehensive auth middleware for admin endpoints
- Enhanced webhook HMAC validation
- Added input validation and rate limiting
- Security headers and CSP policies

## Configuration Management
- Made N8N webhook URL configurable (WHOOSH_N8N_BASE_URL)
- Replaced all hardcoded endpoints with environment variables
- Added feature flags for LLM vs heuristic composition
- Gitea fetch hardening with EAGER_FILTER and FULL_RESCAN options

## API Completeness
- Implemented GetCouncilComposition function
- Added GET /api/v1/councils/{id} endpoint
- Council artifacts API (POST/GET /api/v1/councils/{id}/artifacts)
- /admin/health/details endpoint with component status
- Database lookup for repository URLs (no hardcoded fallbacks)

## Observability & Performance
- Added OpenTelemetry distributed tracing with goal/pulse correlation
- Performance optimization database indexes
- Comprehensive health monitoring
- Enhanced logging and error handling

## Infrastructure
- Production-ready P2P discovery (replaces mock implementation)
- Removed unused Redis configuration
- Enhanced Docker Swarm integration
- Added migration files for performance indexes

## Code Quality
- Comprehensive input validation
- Graceful error handling and failsafe fallbacks
- Backwards compatibility maintained
- Following security best practices

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude Code
2025-09-12 20:34:17 +10:00
parent 56ea52b743
commit 131868bdca
1740 changed files with 575904 additions and 171 deletions

View File

@@ -0,0 +1,23 @@
-- Drop performance optimization indexes for WHOOSH
-- Drop agents table performance indexes
DROP INDEX IF EXISTS idx_agents_status_last_seen;
-- Drop repositories table performance indexes
DROP INDEX IF EXISTS idx_repositories_full_name_lookup;
DROP INDEX IF EXISTS idx_repositories_last_issue_sync;
-- Drop tasks table performance indexes
DROP INDEX IF EXISTS idx_tasks_external_source_lookup;
-- Drop council_agents table performance indexes
DROP INDEX IF EXISTS idx_council_agents_council_lookup;
-- Drop additional performance indexes
DROP INDEX IF EXISTS idx_teams_status_task;
DROP INDEX IF EXISTS idx_repository_webhooks_active_repo;
DROP INDEX IF EXISTS idx_repository_sync_logs_recent;
DROP INDEX IF EXISTS idx_task_assignments_active;
DROP INDEX IF EXISTS idx_council_agents_deployment_status;
DROP INDEX IF EXISTS idx_tasks_completion_analysis;
DROP INDEX IF EXISTS idx_agents_performance_monitoring;

View File

@@ -0,0 +1,50 @@
-- Performance optimization indexes for WHOOSH
-- These indexes improve query performance for common access patterns
-- Agents table performance indexes
-- Composite index for status and last_seen filtering
CREATE INDEX IF NOT EXISTS idx_agents_status_last_seen ON agents(status, last_seen);
-- Repositories table performance indexes
-- Index on full_name for repository lookups
CREATE INDEX IF NOT EXISTS idx_repositories_full_name_lookup ON repositories(full_name);
-- Index on last_issue_sync for monitoring sync operations
CREATE INDEX IF NOT EXISTS idx_repositories_last_issue_sync ON repositories(last_issue_sync);
-- Tasks table performance indexes
-- Composite index for external_id and source_type lookups
CREATE INDEX IF NOT EXISTS idx_tasks_external_source_lookup ON tasks(external_id, source_type);
-- Councils table performance indexes
-- Index on councils.id for faster council lookups (covering existing primary key)
-- Note: Primary key already provides this, but adding explicit index for clarity
-- CREATE INDEX IF NOT EXISTS idx_councils_id ON councils(id); -- Redundant with PRIMARY KEY
-- Council_agents table performance indexes
-- Index on council_id for agent-to-council lookups
CREATE INDEX IF NOT EXISTS idx_council_agents_council_lookup ON council_agents(council_id);
-- Additional performance indexes based on common query patterns
-- Teams table - index on status and task relationships
CREATE INDEX IF NOT EXISTS idx_teams_status_task ON teams(status, current_task_id);
-- Repository webhooks - index for active webhook lookups
CREATE INDEX IF NOT EXISTS idx_repository_webhooks_active_repo ON repository_webhooks(is_active, repository_id);
-- Repository sync logs - index for recent sync monitoring
CREATE INDEX IF NOT EXISTS idx_repository_sync_logs_recent ON repository_sync_logs(repository_id, created_at DESC);
-- Task assignments - index for active assignments
CREATE INDEX IF NOT EXISTS idx_task_assignments_active ON team_assignments(status, team_id, agent_id) WHERE status = 'active';
-- Council agents - index for deployment status monitoring
CREATE INDEX IF NOT EXISTS idx_council_agents_deployment_status ON council_agents(deployed, status, council_id);
-- Performance statistics collection support
-- Index for task completion analysis
CREATE INDEX IF NOT EXISTS idx_tasks_completion_analysis ON tasks(status, completed_at, assigned_team_id) WHERE completed_at IS NOT NULL;
-- Index for agent performance monitoring
CREATE INDEX IF NOT EXISTS idx_agents_performance_monitoring ON agents(status, last_seen, updated_at) WHERE status IN ('available', 'busy', 'error');