Fix critical in-memory task storage with database persistence

Major architectural improvement to replace in-memory task storage with
database-backed persistence while maintaining backward compatibility.

Changes:
- Created Task SQLAlchemy model matching database schema
- Added Workflow and Execution SQLAlchemy models
- Created TaskService for database CRUD operations
- Updated UnifiedCoordinator to use database persistence
- Modified task APIs to leverage database storage
- Added task loading from database on coordinator initialization
- Implemented status change persistence during task execution
- Enhanced task cleanup with database support
- Added comprehensive task statistics from database

Benefits:
- Tasks persist across application restarts
- Better scalability and reliability
- Historical task data retention
- Comprehensive task filtering and querying
- Maintains in-memory cache for performance

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
anthonyrawlins
2025-07-11 08:52:44 +10:00
parent 4de45bf450
commit 59a59f8869
9 changed files with 594 additions and 59 deletions

View File

@@ -0,0 +1,63 @@
"""
SQLAlchemy models for workflows and executions
"""
from sqlalchemy import Column, String, Text, Integer, Boolean, DateTime, ForeignKey, UUID as SqlUUID
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.sql import func
from sqlalchemy.orm import relationship
from ..core.database import Base
import uuid
class Workflow(Base):
__tablename__ = "workflows"
# Primary identification
id = Column(SqlUUID(as_uuid=True), primary_key=True, index=True, default=uuid.uuid4)
# Workflow details
name = Column(String(255), nullable=False)
description = Column(Text)
n8n_data = Column(JSONB, nullable=False)
mcp_tools = Column(JSONB)
# Relationships
created_by = Column(SqlUUID(as_uuid=True), ForeignKey("users.id"))
# Metadata
version = Column(Integer, default=1)
active = Column(Boolean, default=True)
# Timestamps
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Relationships
creator = relationship("User", back_populates="workflows")
executions = relationship("Execution", back_populates="workflow")
tasks = relationship("Task", back_populates="workflow")
class Execution(Base):
__tablename__ = "executions"
# Primary identification
id = Column(SqlUUID(as_uuid=True), primary_key=True, index=True, default=uuid.uuid4)
# Execution details
workflow_id = Column(SqlUUID(as_uuid=True), ForeignKey("workflows.id"), nullable=True)
status = Column(String(50), default='pending')
input_data = Column(JSONB)
output_data = Column(JSONB)
error_message = Column(Text)
progress = Column(Integer, default=0)
# Timestamps
started_at = Column(DateTime(timezone=True), nullable=True)
completed_at = Column(DateTime(timezone=True), nullable=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
# Relationships
workflow = relationship("Workflow", back_populates="executions")
tasks = relationship("Task", back_populates="execution")