Fix Docker Swarm discovery network name mismatch
- Changed NetworkName from 'chorus_default' to 'chorus_net' - This matches the actual network 'CHORUS_chorus_net' (service prefix added automatically) - Fixes discovered_count:0 issue - now successfully discovering all 25 agents - Updated IMPLEMENTATION-SUMMARY with deployment status Result: All 25 CHORUS agents now discovered successfully via Docker Swarm API 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude Code
290
tests/README.md
Normal file
290
tests/README.md
Normal file
@@ -0,0 +1,290 @@
|
||||
# WHOOSH Council Artifact Tests
|
||||
|
||||
## Overview
|
||||
|
||||
This directory contains integration tests for verifying that WHOOSH councils are properly generating project artifacts through the CHORUS agent collaboration system.
|
||||
|
||||
## Test Coverage
|
||||
|
||||
The `test_council_artifacts.py` script performs end-to-end testing of:
|
||||
|
||||
1. **WHOOSH Health Check** - Verifies WHOOSH API is accessible
|
||||
2. **Project Creation** - Creates a test project with council formation
|
||||
3. **Council Formation** - Verifies council was created with correct structure
|
||||
4. **Role Claiming** - Waits for CHORUS agents to claim council roles
|
||||
5. **Artifact Fetching** - Retrieves artifacts produced by the council
|
||||
6. **Content Validation** - Verifies artifact content is complete and valid
|
||||
7. **Cleanup** - Removes test data (optional)
|
||||
|
||||
## Requirements
|
||||
|
||||
```bash
|
||||
pip install requests
|
||||
```
|
||||
|
||||
Or install from requirements file:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Test Run
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py
|
||||
```
|
||||
|
||||
### With Verbose Output
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py --verbose
|
||||
```
|
||||
|
||||
### Custom WHOOSH URL
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py --whoosh-url http://whoosh.example.com:8080
|
||||
```
|
||||
|
||||
### Extended Wait Time for Role Claims
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py --wait-time 60
|
||||
```
|
||||
|
||||
### Skip Cleanup (Keep Test Project)
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py --skip-cleanup
|
||||
```
|
||||
|
||||
### Full Example
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py \
|
||||
--whoosh-url http://localhost:8800 \
|
||||
--verbose \
|
||||
--wait-time 45 \
|
||||
--skip-cleanup
|
||||
```
|
||||
|
||||
## Command-Line Options
|
||||
|
||||
| Option | Description | Default |
|
||||
|--------|-------------|---------|
|
||||
| `--whoosh-url URL` | WHOOSH base URL | `http://localhost:8800` |
|
||||
| `--verbose`, `-v` | Enable detailed output | `False` |
|
||||
| `--skip-cleanup` | Don't delete test project | `False` |
|
||||
| `--wait-time SECONDS` | Max wait for role claims | `30` |
|
||||
|
||||
## Expected Output
|
||||
|
||||
### Successful Test Run
|
||||
|
||||
```
|
||||
======================================================================
|
||||
COUNCIL ARTIFACT GENERATION TEST SUITE
|
||||
======================================================================
|
||||
|
||||
[14:23:45] HEADER: TEST 1: Checking WHOOSH health...
|
||||
[14:23:45] SUCCESS: ✓ WHOOSH is healthy and accessible
|
||||
|
||||
[14:23:45] HEADER: TEST 2: Creating test project...
|
||||
[14:23:46] SUCCESS: ✓ Project created successfully: abc-123-def
|
||||
[14:23:46] INFO: Council ID: abc-123-def
|
||||
|
||||
[14:23:46] HEADER: TEST 3: Verifying council formation...
|
||||
[14:23:46] SUCCESS: ✓ Council found: abc-123-def
|
||||
[14:23:46] INFO: Status: forming
|
||||
|
||||
[14:23:46] HEADER: TEST 4: Waiting for agent role claims (max 30s)...
|
||||
[14:24:15] SUCCESS: ✓ Council activated! All roles claimed
|
||||
|
||||
[14:24:15] HEADER: TEST 5: Fetching council artifacts...
|
||||
[14:24:15] SUCCESS: ✓ Found 3 artifact(s)
|
||||
|
||||
Artifact 1:
|
||||
ID: art-001
|
||||
Type: architecture_document
|
||||
Name: System Architecture Design
|
||||
Status: approved
|
||||
Produced by: chorus-agent-002
|
||||
Produced at: 2025-10-06T14:24:10Z
|
||||
|
||||
[14:24:15] HEADER: TEST 6: Verifying artifact content...
|
||||
[14:24:15] SUCCESS: ✓ All 3 artifact(s) are valid
|
||||
|
||||
[14:24:15] HEADER: TEST 7: Cleaning up test project...
|
||||
[14:24:16] SUCCESS: ✓ Project deleted successfully: abc-123-def
|
||||
|
||||
======================================================================
|
||||
TEST SUMMARY
|
||||
======================================================================
|
||||
|
||||
Total Tests: 7
|
||||
Passed: 7 ✓✓✓✓✓✓✓
|
||||
|
||||
Success Rate: 100.0%
|
||||
```
|
||||
|
||||
### Test Failure Example
|
||||
|
||||
```
|
||||
[14:23:46] HEADER: TEST 5: Fetching council artifacts...
|
||||
[14:23:46] WARNING: ⚠ No artifacts found yet
|
||||
[14:23:46] INFO: This is normal - councils need time to produce artifacts
|
||||
|
||||
======================================================================
|
||||
TEST SUMMARY
|
||||
======================================================================
|
||||
|
||||
Total Tests: 7
|
||||
Passed: 6 ✓✓✓✓✓✓
|
||||
Failed: 1 ✗
|
||||
|
||||
Success Rate: 85.7%
|
||||
```
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### Scenario 1: Fresh Deployment Test
|
||||
|
||||
Tests a newly deployed WHOOSH/CHORUS system:
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py --wait-time 60 --verbose
|
||||
```
|
||||
|
||||
**Expected**: Role claiming may take longer on first run as agents initialize.
|
||||
|
||||
### Scenario 2: Production Readiness Test
|
||||
|
||||
Quick validation that production system is working:
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py --whoosh-url https://whoosh.production.com
|
||||
```
|
||||
|
||||
**Expected**: All tests should pass in < 1 minute.
|
||||
|
||||
### Scenario 3: Development/Debug Test
|
||||
|
||||
Keep test project for manual inspection:
|
||||
|
||||
```bash
|
||||
python test_council_artifacts.py --skip-cleanup --verbose
|
||||
```
|
||||
|
||||
**Expected**: Project remains in database for debugging.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Test 1 Fails: WHOOSH Not Accessible
|
||||
|
||||
**Problem**: Cannot connect to WHOOSH API
|
||||
|
||||
**Solutions**:
|
||||
- Verify WHOOSH is running: `docker service ps CHORUS_whoosh`
|
||||
- Check URL is correct: `--whoosh-url http://localhost:8800`
|
||||
- Check firewall/network settings
|
||||
|
||||
### Test 4 Fails: Role Claims Timeout
|
||||
|
||||
**Problem**: CHORUS agents not claiming roles
|
||||
|
||||
**Solutions**:
|
||||
- Increase wait time: `--wait-time 60`
|
||||
- Check CHORUS agents are running: `docker service ps CHORUS_chorus`
|
||||
- Check agent logs: `docker service logs CHORUS_chorus`
|
||||
- Verify P2P discovery is working
|
||||
|
||||
### Test 5 Fails: No Artifacts Found
|
||||
|
||||
**Problem**: Council formed but no artifacts produced
|
||||
|
||||
**Solutions**:
|
||||
- This is expected initially - councils need time to collaborate
|
||||
- Check council status in UI or database
|
||||
- Verify CHORUS agents have proper capabilities configured
|
||||
- Check agent logs for artifact production errors
|
||||
|
||||
## Integration with CI/CD
|
||||
|
||||
### GitHub Actions Example
|
||||
|
||||
```yaml
|
||||
name: Test Council Artifacts
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
- name: Start WHOOSH
|
||||
run: docker-compose up -d
|
||||
- name: Wait for services
|
||||
run: sleep 30
|
||||
- name: Run tests
|
||||
run: |
|
||||
cd tests
|
||||
python test_council_artifacts.py --verbose
|
||||
```
|
||||
|
||||
### Jenkins Example
|
||||
|
||||
```groovy
|
||||
stage('Test Council Artifacts') {
|
||||
steps {
|
||||
sh '''
|
||||
cd tests
|
||||
python test_council_artifacts.py \
|
||||
--whoosh-url http://whoosh-test:8080 \
|
||||
--wait-time 60 \
|
||||
--verbose
|
||||
'''
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Test Data
|
||||
|
||||
The test creates a temporary project using:
|
||||
- **Repository**: `https://gitea.chorus.services/tony/test-council-project`
|
||||
- **Project Name**: Auto-generated from repository
|
||||
- **Council**: Automatically formed with 8 core roles
|
||||
|
||||
All test data is cleaned up unless `--skip-cleanup` is specified.
|
||||
|
||||
## Exit Codes
|
||||
|
||||
- `0` - All tests passed
|
||||
- `1` - One or more tests failed
|
||||
- Non-zero - System error occurred
|
||||
|
||||
## Logging
|
||||
|
||||
Test logs include:
|
||||
- Timestamp for each action
|
||||
- Color-coded output (INFO/SUCCESS/WARNING/ERROR)
|
||||
- Request/response details in verbose mode
|
||||
- Complete artifact metadata
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] Test multiple concurrent project creations
|
||||
- [ ] Verify artifact versioning
|
||||
- [ ] Test artifact approval workflow
|
||||
- [ ] Performance benchmarking
|
||||
- [ ] Load testing with many councils
|
||||
- [ ] WebSocket event stream validation
|
||||
- [ ] Agent collaboration pattern verification
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
- Check logs: `docker service logs CHORUS_whoosh`
|
||||
- Review integration status: `COUNCIL_AGENT_INTEGRATION_STATUS.md`
|
||||
- Open issue on project repository
|
||||
Reference in New Issue
Block a user