chorus-dev-images/docs/MAINTENANCE.md

CHORUS Development Images - Maintenance Guide

Version Management

Semantic Versioning

We follow Semantic Versioning 2.0.0:

• MAJOR (X.0.0): Breaking changes (base image OS upgrade, removed tools)
• MINOR (x.Y.0): New features (new tools added, new image variants)
• PATCH (x.y.Z): Bug fixes, security updates, tool version updates

Updating Version

1. Edit the VERSION file:

   echo "1.1.0" > VERSION
   

2. Commit the change:

   git add VERSION
   git commit -m "Bump version to 1.1.0"
   git push
   

3. Automated build will create images tagged with new version

Updating Base Image

Debian Version Update

When a new Debian stable release is available:

1. Update ARG DEBIAN_VERSION in images/base/Dockerfile:

   ARG DEBIAN_VERSION=bookworm-20240615  # Update date suffix
   

2. Test all images:

   make build-all
   make test-all
   

3. If tests pass, increment MAJOR version

Updating Language Toolchains

Go Version Update

1. Check latest Go version: https://go.dev/dl/

2. Update ARG GO_VERSION in Dockerfile:

   ARG GO_VERSION=1.23.0  # Update version
   

3. Build and test:

   make build-go
   make test-go
   

Node.js Version Update

1. Check LTS versions: https://nodejs.org/

2. Update ARG NODE_VERSION in Dockerfile:

   ARG NODE_VERSION=22  # Update to new LTS
   

3. Build and test:

   make build-node
   make test-node
   

Rust Toolchain Update

Rust auto-updates to stable via rustup. To pin a specific version:

1. Modify Dockerfile rust-dev stage:

   RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
       --default-toolchain 1.78.0 \  # Pin version
       --profile default \
       --no-modify-path
   

Python Version Update

1. Check available versions: apt-cache search python3.

2. Update Dockerfile:

   RUN apt-get update && apt-get install -y --no-install-recommends \
       python3.12 \        # Update version
       python3.12-dev \    # Update version
       python3-pip \
       python3-venv
   

Adding New Tools

Adding Tool to Existing Image

1. Edit images/base/Dockerfile in appropriate stage
2. Add installation command
3. Update README.md with new tool
4. Add test for new tool in tests/test-<image>.sh
5. Increment MINOR version

Example - Adding tokei to Rust image:

# In rust-dev stage
RUN cargo install \
    cargo-edit \
    cargo-audit \
    tokei \  # New tool
    && rm -rf /home/chorus/.cargo/registry/cache

Add test:

# In tests/test-rust-dev.sh
docker run --rm "$IMAGE" which tokei > /dev/null || exit 1

Creating New Language Image

1. Add new stage to images/base/Dockerfile:

   FROM base AS ruby-dev
   
   USER root
   RUN apt-get update && apt-get install -y --no-install-recommends \
       ruby-full \
       && rm -rf /var/lib/apt/lists/*
   
   USER chorus
   RUN gem install bundler
   
   LABEL org.opencontainers.image.title="CHORUS Ruby Development Image"
   

2. Create test script tests/test-ruby-dev.sh

3. Add build target to Makefile

4. Add to CI workflow .gitea/workflows/build-and-push.yml

5. Update README.md with new image

Security Updates

Weekly Automated Rebuild

Images automatically rebuild weekly (Mondays 2 AM UTC) to pull latest security updates.

Manual Security Update

For critical CVEs:

1. Trigger manual rebuild:

   • Go to: https://gitea.chorus.services/tony/chorus-dev-images/actions
   • Click "Build and Push Development Images"
   • Click "Run workflow"
   • Check "Force rebuild all images"

2. Monitor build progress

3. Increment PATCH version after successful build

Testing

Local Testing

# Test all images
make test-all

# Test specific image
make test-rust

Adding New Tests

Add assertions to appropriate test script:

# tests/test-rust-dev.sh
echo "  ✓ Checking new feature..."
docker run --rm "$IMAGE" bash -c '
  # Test commands here
  rustc --version | grep -q "1.77"
' || exit 1

Registry Management

Cleaning Old Tags

Periodically remove old version tags to save space:

# List all tags for an image
curl -u username:password \
  https://registry.home.deepblack.cloud/v2/chorus/rust-dev/tags/list

# Delete specific tag (requires registry API v2)
curl -X DELETE -u username:password \
  https://registry.home.deepblack.cloud/v2/chorus/rust-dev/manifests/<digest>

Checking Image Sizes

# Local sizes
docker images | grep chorus

# Registry sizes (requires access to registry)
docker manifest inspect registry.home.deepblack.cloud/chorus/rust-dev:latest | \
  jq -r '.layers[].size' | awk '{s+=$1} END {print s/1024/1024 " MB"}'

Build Optimization

Layer Caching

Ensure frequently changing operations are at the end:

# ✅ Good - static dependencies first
RUN apt-get update && apt-get install -y build-essential
RUN cargo install cargo-edit  # Changes infrequently

# ❌ Bad - changing operation first
COPY requirements.txt /tmp/
RUN pip install -r /tmp/requirements.txt
RUN apt-get update  # Should be before pip

Multi-Stage Build Size

Check base layer is shared:

docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}\t{{.ID}}" | grep chorus

All images should share the same base layer ID.

Troubleshooting

Build Fails

1. Check Docker build logs
2. Verify external dependencies (Go downloads, npm registry)
3. Test locally: make build-<image>
4. Check network connectivity

Test Fails

1. Run test manually: bash tests/test-<image>.sh
2. Check tool availability in container
3. Verify tool versions
4. Check for breaking changes in tool updates

CI/CD Issues

1. Check Gitea Actions logs
2. Verify registry credentials (secrets)
3. Check Docker buildx support
4. Verify network access to registry

Release Checklist

Before releasing new version:

• All tests pass locally (make test-all)
• README.md updated with new features/changes
• USAGE.md updated if usage patterns changed
• VERSION file updated
• Git tag created: git tag v1.x.x && git push --tags
• Registry contains new version tags
• CHANGELOG.md updated (if exists)
• CHORUS engine updated to use new images (if needed)

Monitoring

Build Status

Check automated builds: https://gitea.chorus.services/tony/chorus-dev-images/actions

Registry Health

# Check registry is accessible
curl -u username:password \
  https://registry.home.deepblack.cloud/v2/_catalog

# Check image is pullable
docker pull registry.home.deepblack.cloud/chorus/base:latest

Usage Metrics

Track which images are most pulled (requires registry logging):

# Check registry logs
docker service logs registry_registry | grep "GET /v2/chorus" | \
  awk '{print $10}' | sort | uniq -c | sort -rn