chorus-dev-images/docs/MAINTENANCE.md

# CHORUS Development Images - Maintenance Guide

## Version Management

### Semantic Versioning

We follow [Semantic Versioning 2.0.0](https://semver.org/):

- **MAJOR** (X.0.0): Breaking changes (base image OS upgrade, removed tools)
- **MINOR** (x.Y.0): New features (new tools added, new image variants)
- **PATCH** (x.y.Z): Bug fixes, security updates, tool version updates

### Updating Version

1. Edit the `VERSION` file:
   ```bash
   echo "1.1.0" > VERSION
   ```

2. Commit the change:
   ```bash
   git add VERSION
   git commit -m "Bump version to 1.1.0"
   git push
   ```

3. Automated build will create images tagged with new version

## Updating Base Image

### Debian Version Update

When a new Debian stable release is available:

1. Update `ARG DEBIAN_VERSION` in `images/base/Dockerfile`:
   ```dockerfile
   ARG DEBIAN_VERSION=bookworm-20240615  # Update date suffix
   ```

2. Test all images:
   ```bash
   make build-all
   make test-all
   ```

3. If tests pass, increment MAJOR version

## Updating Language Toolchains

### Go Version Update

1. Check latest Go version: https://go.dev/dl/

2. Update `ARG GO_VERSION` in Dockerfile:
   ```dockerfile
   ARG GO_VERSION=1.23.0  # Update version
   ```

3. Build and test:
   ```bash
   make build-go
   make test-go
   ```

### Node.js Version Update

1. Check LTS versions: https://nodejs.org/

2. Update `ARG NODE_VERSION` in Dockerfile:
   ```dockerfile
   ARG NODE_VERSION=22  # Update to new LTS
   ```

3. Build and test:
   ```bash
   make build-node
   make test-node
   ```

### Rust Toolchain Update

Rust auto-updates to stable via rustup. To pin a specific version:

1. Modify Dockerfile rust-dev stage:
   ```dockerfile
   RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
       --default-toolchain 1.78.0 \  # Pin version
       --profile default \
       --no-modify-path
   ```

### Python Version Update

1. Check available versions: `apt-cache search python3.`

2. Update Dockerfile:
   ```dockerfile
   RUN apt-get update && apt-get install -y --no-install-recommends \
       python3.12 \        # Update version
       python3.12-dev \    # Update version
       python3-pip \
       python3-venv
   ```

## Adding New Tools

### Adding Tool to Existing Image

1. Edit `images/base/Dockerfile` in appropriate stage
2. Add installation command
3. Update README.md with new tool
4. Add test for new tool in `tests/test-<image>.sh`
5. Increment MINOR version

Example - Adding `tokei` to Rust image:

```dockerfile
# In rust-dev stage
RUN cargo install \
    cargo-edit \
    cargo-audit \
    tokei \  # New tool
    && rm -rf /home/chorus/.cargo/registry/cache
```

Add test:
```bash
# In tests/test-rust-dev.sh
docker run --rm "$IMAGE" which tokei > /dev/null || exit 1
```

### Creating New Language Image

1. Add new stage to `images/base/Dockerfile`:
   ```dockerfile
   FROM base AS ruby-dev

   USER root
   RUN apt-get update && apt-get install -y --no-install-recommends \
       ruby-full \
       && rm -rf /var/lib/apt/lists/*

   USER chorus
   RUN gem install bundler

   LABEL org.opencontainers.image.title="CHORUS Ruby Development Image"
   ```

2. Create test script `tests/test-ruby-dev.sh`

3. Add build target to `Makefile`

4. Add to CI workflow `.gitea/workflows/build-and-push.yml`

5. Update README.md with new image

## Security Updates

### Weekly Automated Rebuild

Images automatically rebuild weekly (Mondays 2 AM UTC) to pull latest security updates.

### Manual Security Update

For critical CVEs:

1. Trigger manual rebuild:
   - Go to: https://gitea.chorus.services/tony/chorus-dev-images/actions
   - Click "Build and Push Development Images"
   - Click "Run workflow"
   - Check "Force rebuild all images"

2. Monitor build progress

3. Increment PATCH version after successful build

## Testing

### Local Testing

```bash
# Test all images
make test-all

# Test specific image
make test-rust
```

### Adding New Tests

Add assertions to appropriate test script:

```bash
# tests/test-rust-dev.sh
echo "  ✓ Checking new feature..."
docker run --rm "$IMAGE" bash -c '
  # Test commands here
  rustc --version | grep -q "1.77"
' || exit 1
```

## Registry Management

### Cleaning Old Tags

Periodically remove old version tags to save space:

```bash
# List all tags for an image
curl -u username:password \
  https://registry.home.deepblack.cloud/v2/chorus/rust-dev/tags/list

# Delete specific tag (requires registry API v2)
curl -X DELETE -u username:password \
  https://registry.home.deepblack.cloud/v2/chorus/rust-dev/manifests/<digest>
```

### Checking Image Sizes

```bash
# Local sizes
docker images | grep chorus

# Registry sizes (requires access to registry)
docker manifest inspect registry.home.deepblack.cloud/chorus/rust-dev:latest | \
  jq -r '.layers[].size' | awk '{s+=$1} END {print s/1024/1024 " MB"}'
```

## Build Optimization

### Layer Caching

Ensure frequently changing operations are at the end:

```dockerfile
# ✅ Good - static dependencies first
RUN apt-get update && apt-get install -y build-essential
RUN cargo install cargo-edit  # Changes infrequently

# ❌ Bad - changing operation first
COPY requirements.txt /tmp/
RUN pip install -r /tmp/requirements.txt
RUN apt-get update  # Should be before pip
```

### Multi-Stage Build Size

Check base layer is shared:

```bash
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}\t{{.ID}}" | grep chorus
```

All images should share the same base layer ID.

## Troubleshooting

### Build Fails

1. Check Docker build logs
2. Verify external dependencies (Go downloads, npm registry)
3. Test locally: `make build-<image>`
4. Check network connectivity

### Test Fails

1. Run test manually: `bash tests/test-<image>.sh`
2. Check tool availability in container
3. Verify tool versions
4. Check for breaking changes in tool updates

### CI/CD Issues

1. Check Gitea Actions logs
2. Verify registry credentials (secrets)
3. Check Docker buildx support
4. Verify network access to registry

## Release Checklist

Before releasing new version:

- [ ] All tests pass locally (`make test-all`)
- [ ] README.md updated with new features/changes
- [ ] USAGE.md updated if usage patterns changed
- [ ] VERSION file updated
- [ ] Git tag created: `git tag v1.x.x && git push --tags`
- [ ] Registry contains new version tags
- [ ] CHANGELOG.md updated (if exists)
- [ ] CHORUS engine updated to use new images (if needed)

## Monitoring

### Build Status

Check automated builds: https://gitea.chorus.services/tony/chorus-dev-images/actions

### Registry Health

```bash
# Check registry is accessible
curl -u username:password \
  https://registry.home.deepblack.cloud/v2/_catalog

# Check image is pullable
docker pull registry.home.deepblack.cloud/chorus/base:latest
```

### Usage Metrics

Track which images are most pulled (requires registry logging):

```bash
# Check registry logs
docker service logs registry_registry | grep "GET /v2/chorus" | \
  awk '{print $10}' | sort | uniq -c | sort -rn
```