Phase 2: Implement Execution Environment Abstraction (v0.3.0)

This commit implements Phase 2 of the CHORUS Task Execution Engine development plan,
providing a comprehensive execution environment abstraction layer with Docker
container sandboxing support.

## New Features

### Core Sandbox Interface
- Comprehensive ExecutionSandbox interface with isolated task execution
- Support for command execution, file I/O, environment management
- Resource usage monitoring and sandbox lifecycle management
- Standardized error handling with SandboxError types and categories

### Docker Container Sandbox Implementation
- Full Docker API integration with secure container creation
- Transparent repository mounting with configurable read/write access
- Advanced security policies with capability dropping and privilege controls
- Comprehensive resource limits (CPU, memory, disk, processes, file handles)
- Support for tmpfs mounts, masked paths, and read-only bind mounts
- Container lifecycle management with proper cleanup and health monitoring

### Security & Resource Management
- Configurable security policies with SELinux, AppArmor, and Seccomp support
- Fine-grained capability management with secure defaults
- Network isolation options with configurable DNS and proxy settings
- Resource monitoring with real-time CPU, memory, and network usage tracking
- Comprehensive ulimits configuration for process and file handle limits

### Repository Integration
- Seamless repository mounting from local paths to container workspaces
- Git configuration support with user credentials and global settings
- File inclusion/exclusion patterns for selective repository access
- Configurable permissions and ownership for mounted repositories

### Testing Infrastructure
- Comprehensive test suite with 60+ test cases covering all functionality
- Docker integration tests with Alpine Linux containers (skipped in short mode)
- Mock sandbox implementation for unit testing without Docker dependencies
- Security policy validation tests with read-only filesystem enforcement
- Resource usage monitoring and cleanup verification tests

## Technical Details

### Dependencies Added
- github.com/docker/docker v28.4.0+incompatible - Docker API client
- github.com/docker/go-connections v0.6.0 - Docker connection utilities
- github.com/docker/go-units v0.5.0 - Docker units and formatting
- Associated Docker API dependencies for complete container management

### Architecture
- Interface-driven design enabling multiple sandbox implementations
- Comprehensive configuration structures for all sandbox aspects
- Resource usage tracking with detailed metrics collection
- Error handling with retryable error classification
- Proper cleanup and resource management throughout sandbox lifecycle

### Compatibility
- Maintains backward compatibility with existing CHORUS architecture
- Designed for future integration with Phase 3 Core Task Execution Engine
- Extensible design supporting additional sandbox implementations (VM, process)

This Phase 2 implementation provides the foundation for secure, isolated task
execution that will be integrated with the AI model providers from Phase 1
in the upcoming Phase 3 development.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
anthonyrawlins
2025-09-25 14:28:08 +10:00
parent d1252ade69
commit 8d9b62daf3
653 changed files with 88039 additions and 3766 deletions

View File

@@ -1,24 +1,14 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// SPDX-License-Identifier: Apache-2.0
package attribute // import "go.opentelemetry.io/otel/attribute"
import (
"cmp"
"encoding/json"
"reflect"
"slices"
"sort"
"sync"
)
type (
@@ -26,30 +16,33 @@ type (
// immutable set of attributes, with an internal cache for storing
// attribute encodings.
//
// This type supports the Equivalent method of comparison using values of
// type Distinct.
// This type will remain comparable for backwards compatibility. The
// equivalence of Sets across versions is not guaranteed to be stable.
// Prior versions may find two Sets to be equal or not when compared
// directly (i.e. ==), but subsequent versions may not. Users should use
// the Equals method to ensure stable equivalence checking.
//
// Users should also use the Distinct returned from Equivalent as a map key
// instead of a Set directly. In addition to that type providing guarantees
// on stable equivalence, it may also provide performance improvements.
Set struct {
equivalent Distinct
}
// Distinct wraps a variable-size array of KeyValue, constructed with keys
// in sorted order. This can be used as a map key or for equality checking
// between Sets.
// Distinct is a unique identifier of a Set.
//
// Distinct is designed to ensure equivalence stability: comparisons will
// return the same value across versions. For this reason, Distinct should
// always be used as a map key instead of a Set.
Distinct struct {
iface interface{}
iface any
}
// Filter supports removing certain attributes from attribute sets. When
// the filter returns true, the attribute will be kept in the filtered
// attribute set. When the filter returns false, the attribute is excluded
// from the filtered attribute set, and the attribute instead appears in
// the removed list of excluded attributes.
Filter func(KeyValue) bool
// Sortable implements sort.Interface, used for sorting KeyValue. This is
// an exported type to support a memory optimization. A pointer to one of
// these is needed for the call to sort.Stable(), which the caller may
// provide in order to avoid an allocation. See NewSetWithSortable().
// Sortable implements sort.Interface, used for sorting KeyValue.
//
// Deprecated: This type is no longer used. It was added as a performance
// optimization for Go < 1.21 that is no longer needed (Go < 1.21 is no
// longer supported by the module).
Sortable []KeyValue
)
@@ -63,12 +56,6 @@ var (
iface: [0]KeyValue{},
},
}
// sortables is a pool of Sortables used to create Sets with a user does
// not provide one.
sortables = sync.Pool{
New: func() interface{} { return new(Sortable) },
}
)
// EmptySet returns a reference to a Set with no elements.
@@ -83,7 +70,7 @@ func (d Distinct) reflectValue() reflect.Value {
return reflect.ValueOf(d.iface)
}
// Valid returns true if this value refers to a valid Set.
// Valid reports whether this value refers to a valid Set.
func (d Distinct) Valid() bool {
return d.iface != nil
}
@@ -133,7 +120,7 @@ func (l *Set) Value(k Key) (Value, bool) {
return Value{}, false
}
// HasValue tests whether a key is defined in this set.
// HasValue reports whether a key is defined in this set.
func (l *Set) HasValue(k Key) bool {
if l == nil {
return false
@@ -168,7 +155,7 @@ func (l *Set) Equivalent() Distinct {
return l.equivalent
}
// Equals returns true if the argument set is equivalent to this set.
// Equals reports whether the argument set is equivalent to this set.
func (l *Set) Equals(o *Set) bool {
return l.Equivalent() == o.Equivalent()
}
@@ -194,13 +181,7 @@ func empty() Set {
// Except for empty sets, this method adds an additional allocation compared
// with calls that include a Sortable.
func NewSet(kvs ...KeyValue) Set {
// Check for empty set.
if len(kvs) == 0 {
return empty()
}
srt := sortables.Get().(*Sortable)
s, _ := NewSetWithSortableFiltered(kvs, srt, nil)
sortables.Put(srt)
s, _ := NewSetWithFiltered(kvs, nil)
return s
}
@@ -208,12 +189,10 @@ func NewSet(kvs ...KeyValue) Set {
// NewSetWithSortableFiltered for more details.
//
// This call includes a Sortable option as a memory optimization.
func NewSetWithSortable(kvs []KeyValue, tmp *Sortable) Set {
// Check for empty set.
if len(kvs) == 0 {
return empty()
}
s, _ := NewSetWithSortableFiltered(kvs, tmp, nil)
//
// Deprecated: Use [NewSet] instead.
func NewSetWithSortable(kvs []KeyValue, _ *Sortable) Set {
s, _ := NewSetWithFiltered(kvs, nil)
return s
}
@@ -227,10 +206,37 @@ func NewSetWithFiltered(kvs []KeyValue, filter Filter) (Set, []KeyValue) {
if len(kvs) == 0 {
return empty(), nil
}
srt := sortables.Get().(*Sortable)
s, filtered := NewSetWithSortableFiltered(kvs, srt, filter)
sortables.Put(srt)
return s, filtered
// Stable sort so the following de-duplication can implement
// last-value-wins semantics.
slices.SortStableFunc(kvs, func(a, b KeyValue) int {
return cmp.Compare(a.Key, b.Key)
})
position := len(kvs) - 1
offset := position - 1
// The requirements stated above require that the stable
// result be placed in the end of the input slice, while
// overwritten values are swapped to the beginning.
//
// De-duplicate with last-value-wins semantics. Preserve
// duplicate values at the beginning of the input slice.
for ; offset >= 0; offset-- {
if kvs[offset].Key == kvs[position].Key {
continue
}
position--
kvs[offset], kvs[position] = kvs[position], kvs[offset]
}
kvs = kvs[position:]
if filter != nil {
if div := filteredToFront(kvs, filter); div != 0 {
return Set{equivalent: computeDistinct(kvs[div:])}, kvs[:div]
}
}
return Set{equivalent: computeDistinct(kvs)}, nil
}
// NewSetWithSortableFiltered returns a new Set.
@@ -256,82 +262,71 @@ func NewSetWithFiltered(kvs []KeyValue, filter Filter) (Set, []KeyValue) {
//
// The second []KeyValue return value is a list of attributes that were
// excluded by the Filter (if non-nil).
func NewSetWithSortableFiltered(kvs []KeyValue, tmp *Sortable, filter Filter) (Set, []KeyValue) {
// Check for empty set.
if len(kvs) == 0 {
return empty(), nil
}
*tmp = kvs
// Stable sort so the following de-duplication can implement
// last-value-wins semantics.
sort.Stable(tmp)
*tmp = nil
position := len(kvs) - 1
offset := position - 1
// The requirements stated above require that the stable
// result be placed in the end of the input slice, while
// overwritten values are swapped to the beginning.
//
// De-duplicate with last-value-wins semantics. Preserve
// duplicate values at the beginning of the input slice.
for ; offset >= 0; offset-- {
if kvs[offset].Key == kvs[position].Key {
continue
}
position--
kvs[offset], kvs[position] = kvs[position], kvs[offset]
}
if filter != nil {
return filterSet(kvs[position:], filter)
}
return Set{
equivalent: computeDistinct(kvs[position:]),
}, nil
//
// Deprecated: Use [NewSetWithFiltered] instead.
func NewSetWithSortableFiltered(kvs []KeyValue, _ *Sortable, filter Filter) (Set, []KeyValue) {
return NewSetWithFiltered(kvs, filter)
}
// filterSet reorders kvs so that included keys are contiguous at the end of
// the slice, while excluded keys precede the included keys.
func filterSet(kvs []KeyValue, filter Filter) (Set, []KeyValue) {
var excluded []KeyValue
// Move attributes that do not match the filter so they're adjacent before
// calling computeDistinct().
distinctPosition := len(kvs)
// Swap indistinct keys forward and distinct keys toward the
// end of the slice.
offset := len(kvs) - 1
for ; offset >= 0; offset-- {
if filter(kvs[offset]) {
distinctPosition--
kvs[offset], kvs[distinctPosition] = kvs[distinctPosition], kvs[offset]
continue
// filteredToFront filters slice in-place using keep function. All KeyValues that need to
// be removed are moved to the front. All KeyValues that need to be kept are
// moved (in-order) to the back. The index for the first KeyValue to be kept is
// returned.
func filteredToFront(slice []KeyValue, keep Filter) int {
n := len(slice)
j := n
for i := n - 1; i >= 0; i-- {
if keep(slice[i]) {
j--
slice[i], slice[j] = slice[j], slice[i]
}
}
excluded = kvs[:distinctPosition]
return Set{
equivalent: computeDistinct(kvs[distinctPosition:]),
}, excluded
return j
}
// Filter returns a filtered copy of this Set. See the documentation for
// NewSetWithSortableFiltered for more details.
func (l *Set) Filter(re Filter) (Set, []KeyValue) {
if re == nil {
return Set{
equivalent: l.equivalent,
}, nil
return *l, nil
}
// Note: This could be refactored to avoid the temporary slice
// allocation, if it proves to be expensive.
return filterSet(l.ToSlice(), re)
// Iterate in reverse to the first attribute that will be filtered out.
n := l.Len()
first := n - 1
for ; first >= 0; first-- {
kv, _ := l.Get(first)
if !re(kv) {
break
}
}
// No attributes will be dropped, return the immutable Set l and nil.
if first < 0 {
return *l, nil
}
// Copy now that we know we need to return a modified set.
//
// Do not do this in-place on the underlying storage of *Set l. Sets are
// immutable and filtering should not change this.
slice := l.ToSlice()
// Don't re-iterate the slice if only slice[0] is filtered.
if first == 0 {
// It is safe to assume len(slice) >= 1 given we found at least one
// attribute above that needs to be filtered out.
return Set{equivalent: computeDistinct(slice[1:])}, slice[:1]
}
// Move the filtered slice[first] to the front (preserving order).
kv := slice[first]
copy(slice[1:first+1], slice[:first])
slice[0] = kv
// Do not re-evaluate re(slice[first+1:]).
div := filteredToFront(slice[1:first+1], re) + 1
return Set{equivalent: computeDistinct(slice[div:])}, slice[:div]
}
// computeDistinct returns a Distinct using either the fixed- or
@@ -349,48 +344,28 @@ func computeDistinct(kvs []KeyValue) Distinct {
// computeDistinctFixed computes a Distinct for small slices. It returns nil
// if the input is too large for this code path.
func computeDistinctFixed(kvs []KeyValue) interface{} {
func computeDistinctFixed(kvs []KeyValue) any {
switch len(kvs) {
case 1:
ptr := new([1]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [1]KeyValue(kvs)
case 2:
ptr := new([2]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [2]KeyValue(kvs)
case 3:
ptr := new([3]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [3]KeyValue(kvs)
case 4:
ptr := new([4]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [4]KeyValue(kvs)
case 5:
ptr := new([5]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [5]KeyValue(kvs)
case 6:
ptr := new([6]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [6]KeyValue(kvs)
case 7:
ptr := new([7]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [7]KeyValue(kvs)
case 8:
ptr := new([8]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [8]KeyValue(kvs)
case 9:
ptr := new([9]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [9]KeyValue(kvs)
case 10:
ptr := new([10]KeyValue)
copy((*ptr)[:], kvs)
return *ptr
return [10]KeyValue(kvs)
default:
return nil
}
@@ -398,7 +373,7 @@ func computeDistinctFixed(kvs []KeyValue) interface{} {
// computeDistinctReflect computes a Distinct using reflection, works for any
// size input.
func computeDistinctReflect(kvs []KeyValue) interface{} {
func computeDistinctReflect(kvs []KeyValue) any {
at := reflect.New(reflect.ArrayOf(len(kvs), keyValueType)).Elem()
for i, keyValue := range kvs {
*(at.Index(i).Addr().Interface().(*KeyValue)) = keyValue
@@ -411,8 +386,8 @@ func (l *Set) MarshalJSON() ([]byte, error) {
return json.Marshal(l.equivalent.iface)
}
// MarshalLog is the marshaling function used by the logging system to represent this exporter.
func (l Set) MarshalLog() interface{} {
// MarshalLog is the marshaling function used by the logging system to represent this Set.
func (l Set) MarshalLog() any {
kvs := make(map[string]string)
for _, kv := range l.ToSlice() {
kvs[string(kv.Key)] = kv.Value.Emit()