Initial release of CHORUS Development Images
Some checks failed
Build and Push Development Images / build-base (push) Has been cancelled
Build and Push Development Images / build-rust (push) Has been cancelled
Build and Push Development Images / build-go (push) Has been cancelled
Build and Push Development Images / build-python (push) Has been cancelled
Build and Push Development Images / build-node (push) Has been cancelled
Build and Push Development Images / build-java (push) Has been cancelled
Build and Push Development Images / build-cpp (push) Has been cancelled
Build and Push Development Images / test-images (base) (push) Has been cancelled
Build and Push Development Images / test-images (cpp-dev) (push) Has been cancelled
Build and Push Development Images / test-images (go-dev) (push) Has been cancelled
Build and Push Development Images / test-images (java-dev) (push) Has been cancelled
Build and Push Development Images / test-images (node-dev) (push) Has been cancelled
Build and Push Development Images / test-images (python-dev) (push) Has been cancelled
Build and Push Development Images / test-images (rust-dev) (push) Has been cancelled

This commit establishes the foundation for language-specific development
container images used by CHORUS autonomous agents.

Features:
- Multi-stage Dockerfile with 7 layered images
- Base Debian Bookworm image with common tools
- Language-specific images: Rust, Go, Python, Node.js, Java, C/C++
- Standardized /workspace/{input,data,output} structure
- Automated CI/CD pipeline for weekly security updates
- Comprehensive test suite for all images
- Full documentation (README, USAGE, MAINTENANCE)

Images available:
- chorus/base:1.0.0 (~200MB)
- chorus/rust-dev:1.0.0 (~1.2GB)
- chorus/go-dev:1.0.0 (~600MB)
- chorus/python-dev:1.0.0 (~800MB)
- chorus/node-dev:1.0.0 (~700MB)
- chorus/java-dev:1.0.0 (~1.5GB)
- chorus/cpp-dev:1.0.0 (~900MB)

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
anthonyrawlins
2025-09-30 11:09:36 +10:00
commit 90249aad89
15 changed files with 1803 additions and 0 deletions

317
docs/MAINTENANCE.md Normal file
View File

@@ -0,0 +1,317 @@
# CHORUS Development Images - Maintenance Guide
## Version Management
### Semantic Versioning
We follow [Semantic Versioning 2.0.0](https://semver.org/):
- **MAJOR** (X.0.0): Breaking changes (base image OS upgrade, removed tools)
- **MINOR** (x.Y.0): New features (new tools added, new image variants)
- **PATCH** (x.y.Z): Bug fixes, security updates, tool version updates
### Updating Version
1. Edit the `VERSION` file:
```bash
echo "1.1.0" > VERSION
```
2. Commit the change:
```bash
git add VERSION
git commit -m "Bump version to 1.1.0"
git push
```
3. Automated build will create images tagged with new version
## Updating Base Image
### Debian Version Update
When a new Debian stable release is available:
1. Update `ARG DEBIAN_VERSION` in `images/base/Dockerfile`:
```dockerfile
ARG DEBIAN_VERSION=bookworm-20240615 # Update date suffix
```
2. Test all images:
```bash
make build-all
make test-all
```
3. If tests pass, increment MAJOR version
## Updating Language Toolchains
### Go Version Update
1. Check latest Go version: https://go.dev/dl/
2. Update `ARG GO_VERSION` in Dockerfile:
```dockerfile
ARG GO_VERSION=1.23.0 # Update version
```
3. Build and test:
```bash
make build-go
make test-go
```
### Node.js Version Update
1. Check LTS versions: https://nodejs.org/
2. Update `ARG NODE_VERSION` in Dockerfile:
```dockerfile
ARG NODE_VERSION=22 # Update to new LTS
```
3. Build and test:
```bash
make build-node
make test-node
```
### Rust Toolchain Update
Rust auto-updates to stable via rustup. To pin a specific version:
1. Modify Dockerfile rust-dev stage:
```dockerfile
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
--default-toolchain 1.78.0 \ # Pin version
--profile default \
--no-modify-path
```
### Python Version Update
1. Check available versions: `apt-cache search python3.`
2. Update Dockerfile:
```dockerfile
RUN apt-get update && apt-get install -y --no-install-recommends \
python3.12 \ # Update version
python3.12-dev \ # Update version
python3-pip \
python3-venv
```
## Adding New Tools
### Adding Tool to Existing Image
1. Edit `images/base/Dockerfile` in appropriate stage
2. Add installation command
3. Update README.md with new tool
4. Add test for new tool in `tests/test-<image>.sh`
5. Increment MINOR version
Example - Adding `tokei` to Rust image:
```dockerfile
# In rust-dev stage
RUN cargo install \
cargo-edit \
cargo-audit \
tokei \ # New tool
&& rm -rf /home/chorus/.cargo/registry/cache
```
Add test:
```bash
# In tests/test-rust-dev.sh
docker run --rm "$IMAGE" which tokei > /dev/null || exit 1
```
### Creating New Language Image
1. Add new stage to `images/base/Dockerfile`:
```dockerfile
FROM base AS ruby-dev
USER root
RUN apt-get update && apt-get install -y --no-install-recommends \
ruby-full \
&& rm -rf /var/lib/apt/lists/*
USER chorus
RUN gem install bundler
LABEL org.opencontainers.image.title="CHORUS Ruby Development Image"
```
2. Create test script `tests/test-ruby-dev.sh`
3. Add build target to `Makefile`
4. Add to CI workflow `.gitea/workflows/build-and-push.yml`
5. Update README.md with new image
## Security Updates
### Weekly Automated Rebuild
Images automatically rebuild weekly (Mondays 2 AM UTC) to pull latest security updates.
### Manual Security Update
For critical CVEs:
1. Trigger manual rebuild:
- Go to: https://gitea.chorus.services/tony/chorus-dev-images/actions
- Click "Build and Push Development Images"
- Click "Run workflow"
- Check "Force rebuild all images"
2. Monitor build progress
3. Increment PATCH version after successful build
## Testing
### Local Testing
```bash
# Test all images
make test-all
# Test specific image
make test-rust
```
### Adding New Tests
Add assertions to appropriate test script:
```bash
# tests/test-rust-dev.sh
echo " ✓ Checking new feature..."
docker run --rm "$IMAGE" bash -c '
# Test commands here
rustc --version | grep -q "1.77"
' || exit 1
```
## Registry Management
### Cleaning Old Tags
Periodically remove old version tags to save space:
```bash
# List all tags for an image
curl -u username:password \
https://registry.home.deepblack.cloud/v2/chorus/rust-dev/tags/list
# Delete specific tag (requires registry API v2)
curl -X DELETE -u username:password \
https://registry.home.deepblack.cloud/v2/chorus/rust-dev/manifests/<digest>
```
### Checking Image Sizes
```bash
# Local sizes
docker images | grep chorus
# Registry sizes (requires access to registry)
docker manifest inspect registry.home.deepblack.cloud/chorus/rust-dev:latest | \
jq -r '.layers[].size' | awk '{s+=$1} END {print s/1024/1024 " MB"}'
```
## Build Optimization
### Layer Caching
Ensure frequently changing operations are at the end:
```dockerfile
# ✅ Good - static dependencies first
RUN apt-get update && apt-get install -y build-essential
RUN cargo install cargo-edit # Changes infrequently
# ❌ Bad - changing operation first
COPY requirements.txt /tmp/
RUN pip install -r /tmp/requirements.txt
RUN apt-get update # Should be before pip
```
### Multi-Stage Build Size
Check base layer is shared:
```bash
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}\t{{.ID}}" | grep chorus
```
All images should share the same base layer ID.
## Troubleshooting
### Build Fails
1. Check Docker build logs
2. Verify external dependencies (Go downloads, npm registry)
3. Test locally: `make build-<image>`
4. Check network connectivity
### Test Fails
1. Run test manually: `bash tests/test-<image>.sh`
2. Check tool availability in container
3. Verify tool versions
4. Check for breaking changes in tool updates
### CI/CD Issues
1. Check Gitea Actions logs
2. Verify registry credentials (secrets)
3. Check Docker buildx support
4. Verify network access to registry
## Release Checklist
Before releasing new version:
- [ ] All tests pass locally (`make test-all`)
- [ ] README.md updated with new features/changes
- [ ] USAGE.md updated if usage patterns changed
- [ ] VERSION file updated
- [ ] Git tag created: `git tag v1.x.x && git push --tags`
- [ ] Registry contains new version tags
- [ ] CHANGELOG.md updated (if exists)
- [ ] CHORUS engine updated to use new images (if needed)
## Monitoring
### Build Status
Check automated builds: https://gitea.chorus.services/tony/chorus-dev-images/actions
### Registry Health
```bash
# Check registry is accessible
curl -u username:password \
https://registry.home.deepblack.cloud/v2/_catalog
# Check image is pullable
docker pull registry.home.deepblack.cloud/chorus/base:latest
```
### Usage Metrics
Track which images are most pulled (requires registry logging):
```bash
# Check registry logs
docker service logs registry_registry | grep "GET /v2/chorus" | \
awk '{print $10}' | sort | uniq -c | sort -rn
```

297
docs/USAGE.md Normal file
View File

@@ -0,0 +1,297 @@
# CHORUS Development Images - Usage Guide
## Basic Usage Patterns
### Pattern 1: Interactive Development
Use when you need to explore a codebase or debug interactively:
```bash
# Start interactive shell in Rust environment
docker run -it --rm \
-v /path/to/repository:/workspace/input \
-v /path/to/output:/workspace/output \
registry.home.deepblack.cloud/chorus/rust-dev:latest
```
Inside the container:
```bash
cd /workspace/input
cargo build
cargo test
# Copy results to output
cp target/release/binary /workspace/output/
```
### Pattern 2: Single Command Execution
Use for automated agent tasks:
```bash
# Build Rust project
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v $(pwd)/output:/workspace/output \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo build --release && cp target/release/* /workspace/output/"
```
### Pattern 3: Multi-Step Workflow
Use for complex build pipelines:
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v $(pwd)/output:/workspace/output \
registry.home.deepblack.cloud/chorus/python-dev:latest \
bash -c "
cd /workspace/input
uv sync
uv run pytest
uv run ruff check
uv run mypy .
cp -r dist/* /workspace/output/
"
```
## Language-Specific Examples
### Rust
**Build and Test:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo build && cargo test"
```
**Format and Lint:**
```bash
docker run --rm \
-v $(pwd):/workspace/input \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo fmt && cargo clippy -- -D warnings"
```
### Go
**Build Binary:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v $(pwd)/bin:/workspace/output \
registry.home.deepblack.cloud/chorus/go-dev:latest \
bash -c "cd /workspace/input && go build -o /workspace/output/app"
```
**Run Tests:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/go-dev:latest \
bash -c "cd /workspace/input && go test ./..."
```
### Python
**Install Dependencies and Test:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/python-dev:latest \
bash -c "cd /workspace/input && uv sync && uv run pytest"
```
**Lint and Type Check:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/python-dev:latest \
bash -c "cd /workspace/input && ruff check . && mypy ."
```
### Node.js
**Install and Build:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v $(pwd)/dist:/workspace/output \
registry.home.deepblack.cloud/chorus/node-dev:latest \
bash -c "cd /workspace/input && npm install && npm run build && cp -r dist/* /workspace/output/"
```
**Run Tests:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/node-dev:latest \
bash -c "cd /workspace/input && npm install && npm test"
```
### Java
**Maven Build:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v $(pwd)/target:/workspace/output \
registry.home.deepblack.cloud/chorus/java-dev:latest \
bash -c "cd /workspace/input && mvn clean package && cp target/*.jar /workspace/output/"
```
**Gradle Build:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v $(pwd)/build:/workspace/output \
registry.home.deepblack.cloud/chorus/java-dev:latest \
bash -c "cd /workspace/input && gradle build && cp build/libs/*.jar /workspace/output/"
```
### C/C++
**CMake Build:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v $(pwd)/build:/workspace/output \
registry.home.deepblack.cloud/chorus/cpp-dev:latest \
bash -c "
cd /workspace/data
cmake /workspace/input
make
cp bin/* /workspace/output/
"
```
## Advanced Usage
### Preserving Build Cache
Mount a volume for package caches to speed up subsequent builds:
**Rust:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v cargo-cache:/home/chorus/.cargo/registry \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo build"
```
**Python:**
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-v pip-cache:/home/chorus/.cache/pip \
registry.home.deepblack.cloud/chorus/python-dev:latest \
bash -c "cd /workspace/input && pip install -r requirements.txt"
```
### Setting Environment Variables
```bash
docker run --rm \
-v $(pwd):/workspace/input:ro \
-e DATABASE_URL=postgres://localhost/db \
-e API_KEY=secret \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo test"
```
### Resource Limits
```bash
docker run --rm \
--memory="2g" \
--cpus="2.0" \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo build --release"
```
### Network Access Control
```bash
# No network access
docker run --rm --network=none \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo build --offline"
# Custom network
docker run --rm --network=my-network \
-v $(pwd):/workspace/input:ro \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo test"
```
## Troubleshooting
### Permission Issues
If you encounter permission errors with output files:
```bash
# Run with matching UID
docker run --rm \
--user $(id -u):$(id -g) \
-v $(pwd):/workspace/input \
-v $(pwd)/output:/workspace/output \
registry.home.deepblack.cloud/chorus/rust-dev:latest \
bash -c "cd /workspace/input && cargo build"
```
### Image Not Found
Ensure you're logged into the registry:
```bash
docker login registry.home.deepblack.cloud
```
### Out of Disk Space
Clean up old images and build cache:
```bash
docker system prune -a
```
## Best Practices
1. **Use Read-Only Input**: Mount source code as read-only (`:ro`) to prevent accidental modifications
2. **Separate Output**: Use dedicated output directory for build artifacts
3. **Cache Dependencies**: Mount package cache volumes for faster rebuilds
4. **Limit Resources**: Set memory and CPU limits for production use
5. **Version Pin**: Use specific version tags (e.g., `1.0.0`) in production, not `latest`
## Integration with CI/CD
### Example GitLab CI
```yaml
test:
image: registry.home.deepblack.cloud/chorus/rust-dev:1.0.0
script:
- cargo test
artifacts:
paths:
- target/release/
```
### Example GitHub Actions
```yaml
jobs:
test:
runs-on: ubuntu-latest
container:
image: registry.home.deepblack.cloud/chorus/rust-dev:1.0.0
steps:
- uses: actions/checkout@v4
- run: cargo test
```