WHOOSH/tests/README.md

# WHOOSH Council Artifact Tests

## Overview

This directory contains integration tests for verifying that WHOOSH councils are properly generating project artifacts through the CHORUS agent collaboration system.

## Test Coverage

The `test_council_artifacts.py` script performs end-to-end testing of:

1. **WHOOSH Health Check** - Verifies WHOOSH API is accessible
2. **Project Creation** - Creates a test project with council formation
3. **Council Formation** - Verifies council was created with correct structure
4. **Role Claiming** - Waits for CHORUS agents to claim council roles
5. **Artifact Fetching** - Retrieves artifacts produced by the council
6. **Content Validation** - Verifies artifact content is complete and valid
7. **Cleanup** - Removes test data (optional)

## Requirements

```bash
pip install requests
```

Or install from requirements file:
```bash
pip install -r requirements.txt
```

## Usage

### Basic Test Run

```bash
python test_council_artifacts.py
```

### With Verbose Output

```bash
python test_council_artifacts.py --verbose
```

### Custom WHOOSH URL

```bash
python test_council_artifacts.py --whoosh-url http://whoosh.example.com:8080
```

### Extended Wait Time for Role Claims

```bash
python test_council_artifacts.py --wait-time 60
```

### Skip Cleanup (Keep Test Project)

```bash
python test_council_artifacts.py --skip-cleanup
```

### Full Example

```bash
python test_council_artifacts.py \
  --whoosh-url http://localhost:8800 \
  --verbose \
  --wait-time 45 \
  --skip-cleanup
```

## Command-Line Options

| Option | Description | Default |
|--------|-------------|---------|
| `--whoosh-url URL` | WHOOSH base URL | `http://localhost:8800` |
| `--verbose`, `-v` | Enable detailed output | `False` |
| `--skip-cleanup` | Don't delete test project | `False` |
| `--wait-time SECONDS` | Max wait for role claims | `30` |

## Expected Output

### Successful Test Run

```
======================================================================
COUNCIL ARTIFACT GENERATION TEST SUITE
======================================================================

[14:23:45] HEADER: TEST 1: Checking WHOOSH health...
[14:23:45] SUCCESS: ✓ WHOOSH is healthy and accessible

[14:23:45] HEADER: TEST 2: Creating test project...
[14:23:46] SUCCESS: ✓ Project created successfully: abc-123-def
[14:23:46] INFO:   Council ID: abc-123-def

[14:23:46] HEADER: TEST 3: Verifying council formation...
[14:23:46] SUCCESS: ✓ Council found: abc-123-def
[14:23:46] INFO:   Status: forming

[14:23:46] HEADER: TEST 4: Waiting for agent role claims (max 30s)...
[14:24:15] SUCCESS: ✓ Council activated! All roles claimed

[14:24:15] HEADER: TEST 5: Fetching council artifacts...
[14:24:15] SUCCESS: ✓ Found 3 artifact(s)

  Artifact 1:
    ID: art-001
    Type: architecture_document
    Name: System Architecture Design
    Status: approved
    Produced by: chorus-agent-002
    Produced at: 2025-10-06T14:24:10Z

[14:24:15] HEADER: TEST 6: Verifying artifact content...
[14:24:15] SUCCESS: ✓ All 3 artifact(s) are valid

[14:24:15] HEADER: TEST 7: Cleaning up test project...
[14:24:16] SUCCESS: ✓ Project deleted successfully: abc-123-def

======================================================================
TEST SUMMARY
======================================================================

Total Tests: 7
  Passed:  7 ✓✓✓✓✓✓✓

Success Rate: 100.0%
```

### Test Failure Example

```
[14:23:46] HEADER: TEST 5: Fetching council artifacts...
[14:23:46] WARNING: ⚠ No artifacts found yet
[14:23:46] INFO:   This is normal - councils need time to produce artifacts

======================================================================
TEST SUMMARY
======================================================================

Total Tests: 7
  Passed:  6 ✓✓✓✓✓✓
  Failed:  1 ✗

Success Rate: 85.7%
```

## Test Scenarios

### Scenario 1: Fresh Deployment Test

Tests a newly deployed WHOOSH/CHORUS system:

```bash
python test_council_artifacts.py --wait-time 60 --verbose
```

**Expected**: Role claiming may take longer on first run as agents initialize.

### Scenario 2: Production Readiness Test

Quick validation that production system is working:

```bash
python test_council_artifacts.py --whoosh-url https://whoosh.production.com
```

**Expected**: All tests should pass in < 1 minute.

### Scenario 3: Development/Debug Test

Keep test project for manual inspection:

```bash
python test_council_artifacts.py --skip-cleanup --verbose
```

**Expected**: Project remains in database for debugging.

## Troubleshooting

### Test 1 Fails: WHOOSH Not Accessible

**Problem**: Cannot connect to WHOOSH API

**Solutions**:
- Verify WHOOSH is running: `docker service ps CHORUS_whoosh`
- Check URL is correct: `--whoosh-url http://localhost:8800`
- Check firewall/network settings

### Test 4 Fails: Role Claims Timeout

**Problem**: CHORUS agents not claiming roles

**Solutions**:
- Increase wait time: `--wait-time 60`
- Check CHORUS agents are running: `docker service ps CHORUS_chorus`
- Check agent logs: `docker service logs CHORUS_chorus`
- Verify P2P discovery is working

### Test 5 Fails: No Artifacts Found

**Problem**: Council formed but no artifacts produced

**Solutions**:
- This is expected initially - councils need time to collaborate
- Check council status in UI or database
- Verify CHORUS agents have proper capabilities configured
- Check agent logs for artifact production errors

## Integration with CI/CD

### GitHub Actions Example

```yaml
name: Test Council Artifacts

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Start WHOOSH
        run: docker-compose up -d
      - name: Wait for services
        run: sleep 30
      - name: Run tests
        run: |
          cd tests
          python test_council_artifacts.py --verbose
```

### Jenkins Example

```groovy
stage('Test Council Artifacts') {
    steps {
        sh '''
            cd tests
            python test_council_artifacts.py \
                --whoosh-url http://whoosh-test:8080 \
                --wait-time 60 \
                --verbose
        '''
    }
}
```

## Test Data

The test creates a temporary project using:
- **Repository**: `https://gitea.chorus.services/tony/test-council-project`
- **Project Name**: Auto-generated from repository
- **Council**: Automatically formed with 8 core roles

All test data is cleaned up unless `--skip-cleanup` is specified.

## Exit Codes

- `0` - All tests passed
- `1` - One or more tests failed
- Non-zero - System error occurred

## Logging

Test logs include:
- Timestamp for each action
- Color-coded output (INFO/SUCCESS/WARNING/ERROR)
- Request/response details in verbose mode
- Complete artifact metadata

## Future Enhancements

- [ ] Test multiple concurrent project creations
- [ ] Verify artifact versioning
- [ ] Test artifact approval workflow
- [ ] Performance benchmarking
- [ ] Load testing with many councils
- [ ] WebSocket event stream validation
- [ ] Agent collaboration pattern verification

## Support

For issues or questions:
- Check logs: `docker service logs CHORUS_whoosh`
- Review integration status: `COUNCIL_AGENT_INTEGRATION_STATUS.md`
- Open issue on project repository