 8d9b62daf3
			
		
	
	8d9b62daf3
	
	
	
		
			
			This commit implements Phase 2 of the CHORUS Task Execution Engine development plan, providing a comprehensive execution environment abstraction layer with Docker container sandboxing support. ## New Features ### Core Sandbox Interface - Comprehensive ExecutionSandbox interface with isolated task execution - Support for command execution, file I/O, environment management - Resource usage monitoring and sandbox lifecycle management - Standardized error handling with SandboxError types and categories ### Docker Container Sandbox Implementation - Full Docker API integration with secure container creation - Transparent repository mounting with configurable read/write access - Advanced security policies with capability dropping and privilege controls - Comprehensive resource limits (CPU, memory, disk, processes, file handles) - Support for tmpfs mounts, masked paths, and read-only bind mounts - Container lifecycle management with proper cleanup and health monitoring ### Security & Resource Management - Configurable security policies with SELinux, AppArmor, and Seccomp support - Fine-grained capability management with secure defaults - Network isolation options with configurable DNS and proxy settings - Resource monitoring with real-time CPU, memory, and network usage tracking - Comprehensive ulimits configuration for process and file handle limits ### Repository Integration - Seamless repository mounting from local paths to container workspaces - Git configuration support with user credentials and global settings - File inclusion/exclusion patterns for selective repository access - Configurable permissions and ownership for mounted repositories ### Testing Infrastructure - Comprehensive test suite with 60+ test cases covering all functionality - Docker integration tests with Alpine Linux containers (skipped in short mode) - Mock sandbox implementation for unit testing without Docker dependencies - Security policy validation tests with read-only filesystem enforcement - Resource usage monitoring and cleanup verification tests ## Technical Details ### Dependencies Added - github.com/docker/docker v28.4.0+incompatible - Docker API client - github.com/docker/go-connections v0.6.0 - Docker connection utilities - github.com/docker/go-units v0.5.0 - Docker units and formatting - Associated Docker API dependencies for complete container management ### Architecture - Interface-driven design enabling multiple sandbox implementations - Comprehensive configuration structures for all sandbox aspects - Resource usage tracking with detailed metrics collection - Error handling with retryable error classification - Proper cleanup and resource management throughout sandbox lifecycle ### Compatibility - Maintains backward compatibility with existing CHORUS architecture - Designed for future integration with Phase 3 Core Task Execution Engine - Extensible design supporting additional sandbox implementations (VM, process) This Phase 2 implementation provides the foundation for secure, isolated task execution that will be integrated with the AI model providers from Phase 1 in the upcoming Phase 3 development. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
		
			
				
	
	
		
			408 lines
		
	
	
		
			19 KiB
		
	
	
	
		
			Markdown
		
	
	
	
	
	
			
		
		
	
	
			408 lines
		
	
	
		
			19 KiB
		
	
	
	
		
			Markdown
		
	
	
	
	
	
| # A minimal logging API for Go
 | |
| 
 | |
| [](https://pkg.go.dev/github.com/go-logr/logr)
 | |
| [](https://goreportcard.com/report/github.com/go-logr/logr)
 | |
| [](https://securityscorecards.dev/viewer/?platform=github.com&org=go-logr&repo=logr)
 | |
| 
 | |
| logr offers an(other) opinion on how Go programs and libraries can do logging
 | |
| without becoming coupled to a particular logging implementation.  This is not
 | |
| an implementation of logging - it is an API.  In fact it is two APIs with two
 | |
| different sets of users.
 | |
| 
 | |
| The `Logger` type is intended for application and library authors.  It provides
 | |
| a relatively small API which can be used everywhere you want to emit logs.  It
 | |
| defers the actual act of writing logs (to files, to stdout, or whatever) to the
 | |
| `LogSink` interface.
 | |
| 
 | |
| The `LogSink` interface is intended for logging library implementers.  It is a
 | |
| pure interface which can be implemented by logging frameworks to provide the actual logging
 | |
| functionality.
 | |
| 
 | |
| This decoupling allows application and library developers to write code in
 | |
| terms of `logr.Logger` (which has very low dependency fan-out) while the
 | |
| implementation of logging is managed "up stack" (e.g. in or near `main()`.)
 | |
| Application developers can then switch out implementations as necessary.
 | |
| 
 | |
| Many people assert that libraries should not be logging, and as such efforts
 | |
| like this are pointless.  Those people are welcome to convince the authors of
 | |
| the tens-of-thousands of libraries that *DO* write logs that they are all
 | |
| wrong.  In the meantime, logr takes a more practical approach.
 | |
| 
 | |
| ## Typical usage
 | |
| 
 | |
| Somewhere, early in an application's life, it will make a decision about which
 | |
| logging library (implementation) it actually wants to use.  Something like:
 | |
| 
 | |
| ```
 | |
|     func main() {
 | |
|         // ... other setup code ...
 | |
| 
 | |
|         // Create the "root" logger.  We have chosen the "logimpl" implementation,
 | |
|         // which takes some initial parameters and returns a logr.Logger.
 | |
|         logger := logimpl.New(param1, param2)
 | |
| 
 | |
|         // ... other setup code ...
 | |
| ```
 | |
| 
 | |
| Most apps will call into other libraries, create structures to govern the flow,
 | |
| etc.  The `logr.Logger` object can be passed to these other libraries, stored
 | |
| in structs, or even used as a package-global variable, if needed.  For example:
 | |
| 
 | |
| ```
 | |
|     app := createTheAppObject(logger)
 | |
|     app.Run()
 | |
| ```
 | |
| 
 | |
| Outside of this early setup, no other packages need to know about the choice of
 | |
| implementation.  They write logs in terms of the `logr.Logger` that they
 | |
| received:
 | |
| 
 | |
| ```
 | |
|     type appObject struct {
 | |
|         // ... other fields ...
 | |
|         logger logr.Logger
 | |
|         // ... other fields ...
 | |
|     }
 | |
| 
 | |
|     func (app *appObject) Run() {
 | |
|         app.logger.Info("starting up", "timestamp", time.Now())
 | |
| 
 | |
|         // ... app code ...
 | |
| ```
 | |
| 
 | |
| ## Background
 | |
| 
 | |
| If the Go standard library had defined an interface for logging, this project
 | |
| probably would not be needed.  Alas, here we are.
 | |
| 
 | |
| When the Go developers started developing such an interface with
 | |
| [slog](https://github.com/golang/go/issues/56345), they adopted some of the
 | |
| logr design but also left out some parts and changed others:
 | |
| 
 | |
| | Feature | logr | slog |
 | |
| |---------|------|------|
 | |
| | High-level API | `Logger` (passed by value) | `Logger` (passed by [pointer](https://github.com/golang/go/issues/59126)) |
 | |
| | Low-level API | `LogSink` | `Handler` |
 | |
| | Stack unwinding | done by `LogSink` | done by `Logger` |
 | |
| | Skipping helper functions | `WithCallDepth`, `WithCallStackHelper` | [not supported by Logger](https://github.com/golang/go/issues/59145) |
 | |
| | Generating a value for logging on demand | `Marshaler` | `LogValuer` |
 | |
| | Log levels | >= 0, higher meaning "less important" | positive and negative, with 0 for "info" and higher meaning "more important" |
 | |
| | Error log entries | always logged, don't have a verbosity level | normal log entries with level >= `LevelError` |
 | |
| | Passing logger via context | `NewContext`, `FromContext` | no API |
 | |
| | Adding a name to a logger | `WithName` | no API |
 | |
| | Modify verbosity of log entries in a call chain | `V` | no API |
 | |
| | Grouping of key/value pairs | not supported | `WithGroup`, `GroupValue` |
 | |
| | Pass context for extracting additional values | no API | API variants like `InfoCtx` |
 | |
| 
 | |
| The high-level slog API is explicitly meant to be one of many different APIs
 | |
| that can be layered on top of a shared `slog.Handler`. logr is one such
 | |
| alternative API, with [interoperability](#slog-interoperability) provided by
 | |
| some conversion functions.
 | |
| 
 | |
| ### Inspiration
 | |
| 
 | |
| Before you consider this package, please read [this blog post by the
 | |
| inimitable Dave Cheney][warning-makes-no-sense].  We really appreciate what
 | |
| he has to say, and it largely aligns with our own experiences.
 | |
| 
 | |
| ### Differences from Dave's ideas
 | |
| 
 | |
| The main differences are:
 | |
| 
 | |
| 1. Dave basically proposes doing away with the notion of a logging API in favor
 | |
| of `fmt.Printf()`.  We disagree, especially when you consider things like output
 | |
| locations, timestamps, file and line decorations, and structured logging.  This
 | |
| package restricts the logging API to just 2 types of logs: info and error.
 | |
| 
 | |
| Info logs are things you want to tell the user which are not errors.  Error
 | |
| logs are, well, errors.  If your code receives an `error` from a subordinate
 | |
| function call and is logging that `error` *and not returning it*, use error
 | |
| logs.
 | |
| 
 | |
| 2. Verbosity-levels on info logs.  This gives developers a chance to indicate
 | |
| arbitrary grades of importance for info logs, without assigning names with
 | |
| semantic meaning such as "warning", "trace", and "debug."  Superficially this
 | |
| may feel very similar, but the primary difference is the lack of semantics.
 | |
| Because verbosity is a numerical value, it's safe to assume that an app running
 | |
| with higher verbosity means more (and less important) logs will be generated.
 | |
| 
 | |
| ## Implementations (non-exhaustive)
 | |
| 
 | |
| There are implementations for the following logging libraries:
 | |
| 
 | |
| - **a function** (can bridge to non-structured libraries): [funcr](https://github.com/go-logr/logr/tree/master/funcr)
 | |
| - **a testing.T** (for use in Go tests, with JSON-like output): [testr](https://github.com/go-logr/logr/tree/master/testr)
 | |
| - **github.com/google/glog**: [glogr](https://github.com/go-logr/glogr)
 | |
| - **k8s.io/klog** (for Kubernetes): [klogr](https://git.k8s.io/klog/klogr)
 | |
| - **a testing.T** (with klog-like text output): [ktesting](https://git.k8s.io/klog/ktesting)
 | |
| - **go.uber.org/zap**: [zapr](https://github.com/go-logr/zapr)
 | |
| - **log** (the Go standard library logger): [stdr](https://github.com/go-logr/stdr)
 | |
| - **github.com/sirupsen/logrus**: [logrusr](https://github.com/bombsimon/logrusr)
 | |
| - **github.com/wojas/genericr**: [genericr](https://github.com/wojas/genericr) (makes it easy to implement your own backend)
 | |
| - **logfmt** (Heroku style [logging](https://www.brandur.org/logfmt)): [logfmtr](https://github.com/iand/logfmtr)
 | |
| - **github.com/rs/zerolog**: [zerologr](https://github.com/go-logr/zerologr)
 | |
| - **github.com/go-kit/log**: [gokitlogr](https://github.com/tonglil/gokitlogr) (also compatible with github.com/go-kit/kit/log since v0.12.0)
 | |
| - **bytes.Buffer** (writing to a buffer): [bufrlogr](https://github.com/tonglil/buflogr) (useful for ensuring values were logged, like during testing)
 | |
| 
 | |
| ## slog interoperability
 | |
| 
 | |
| Interoperability goes both ways, using the `logr.Logger` API with a `slog.Handler`
 | |
| and using the `slog.Logger` API with a `logr.LogSink`. `FromSlogHandler` and
 | |
| `ToSlogHandler` convert between a `logr.Logger` and a `slog.Handler`.
 | |
| As usual, `slog.New` can be used to wrap such a `slog.Handler` in the high-level
 | |
| slog API.
 | |
| 
 | |
| ### Using a `logr.LogSink` as backend for slog
 | |
| 
 | |
| Ideally, a logr sink implementation should support both logr and slog by
 | |
| implementing both the normal logr interface(s) and `SlogSink`.  Because
 | |
| of a conflict in the parameters of the common `Enabled` method, it is [not
 | |
| possible to implement both slog.Handler and logr.Sink in the same
 | |
| type](https://github.com/golang/go/issues/59110).
 | |
| 
 | |
| If both are supported, log calls can go from the high-level APIs to the backend
 | |
| without the need to convert parameters. `FromSlogHandler` and `ToSlogHandler` can
 | |
| convert back and forth without adding additional wrappers, with one exception:
 | |
| when `Logger.V` was used to adjust the verbosity for a `slog.Handler`, then
 | |
| `ToSlogHandler` has to use a wrapper which adjusts the verbosity for future
 | |
| log calls.
 | |
| 
 | |
| Such an implementation should also support values that implement specific
 | |
| interfaces from both packages for logging (`logr.Marshaler`, `slog.LogValuer`,
 | |
| `slog.GroupValue`). logr does not convert those.
 | |
| 
 | |
| Not supporting slog has several drawbacks:
 | |
| - Recording source code locations works correctly if the handler gets called
 | |
|   through `slog.Logger`, but may be wrong in other cases. That's because a
 | |
|   `logr.Sink` does its own stack unwinding instead of using the program counter
 | |
|   provided by the high-level API.
 | |
| - slog levels <= 0 can be mapped to logr levels by negating the level without a
 | |
|   loss of information. But all slog levels > 0 (e.g. `slog.LevelWarning` as
 | |
|   used by `slog.Logger.Warn`) must be mapped to 0 before calling the sink
 | |
|   because logr does not support "more important than info" levels.
 | |
| - The slog group concept is supported by prefixing each key in a key/value
 | |
|   pair with the group names, separated by a dot. For structured output like
 | |
|   JSON it would be better to group the key/value pairs inside an object.
 | |
| - Special slog values and interfaces don't work as expected.
 | |
| - The overhead is likely to be higher.
 | |
| 
 | |
| These drawbacks are severe enough that applications using a mixture of slog and
 | |
| logr should switch to a different backend.
 | |
| 
 | |
| ### Using a `slog.Handler` as backend for logr
 | |
| 
 | |
| Using a plain `slog.Handler` without support for logr works better than the
 | |
| other direction:
 | |
| - All logr verbosity levels can be mapped 1:1 to their corresponding slog level
 | |
|   by negating them.
 | |
| - Stack unwinding is done by the `SlogSink` and the resulting program
 | |
|   counter is passed to the `slog.Handler`.
 | |
| - Names added via `Logger.WithName` are gathered and recorded in an additional
 | |
|   attribute with `logger` as key and the names separated by slash as value.
 | |
| - `Logger.Error` is turned into a log record with `slog.LevelError` as level
 | |
|   and an additional attribute with `err` as key, if an error was provided.
 | |
| 
 | |
| The main drawback is that `logr.Marshaler` will not be supported. Types should
 | |
| ideally support both `logr.Marshaler` and `slog.Valuer`. If compatibility
 | |
| with logr implementations without slog support is not important, then
 | |
| `slog.Valuer` is sufficient.
 | |
| 
 | |
| ### Context support for slog
 | |
| 
 | |
| Storing a logger in a `context.Context` is not supported by
 | |
| slog. `NewContextWithSlogLogger` and `FromContextAsSlogLogger` can be
 | |
| used to fill this gap. They store and retrieve a `slog.Logger` pointer
 | |
| under the same context key that is also used by `NewContext` and
 | |
| `FromContext` for `logr.Logger` value.
 | |
| 
 | |
| When `NewContextWithSlogLogger` is followed by `FromContext`, the latter will
 | |
| automatically convert the `slog.Logger` to a
 | |
| `logr.Logger`. `FromContextAsSlogLogger` does the same for the other direction.
 | |
| 
 | |
| With this approach, binaries which use either slog or logr are as efficient as
 | |
| possible with no unnecessary allocations. This is also why the API stores a
 | |
| `slog.Logger` pointer: when storing a `slog.Handler`, creating a `slog.Logger`
 | |
| on retrieval would need to allocate one.
 | |
| 
 | |
| The downside is that switching back and forth needs more allocations. Because
 | |
| logr is the API that is already in use by different packages, in particular
 | |
| Kubernetes, the recommendation is to use the `logr.Logger` API in code which
 | |
| uses contextual logging.
 | |
| 
 | |
| An alternative to adding values to a logger and storing that logger in the
 | |
| context is to store the values in the context and to configure a logging
 | |
| backend to extract those values when emitting log entries. This only works when
 | |
| log calls are passed the context, which is not supported by the logr API.
 | |
| 
 | |
| With the slog API, it is possible, but not
 | |
| required. https://github.com/veqryn/slog-context is a package for slog which
 | |
| provides additional support code for this approach. It also contains wrappers
 | |
| for the context functions in logr, so developers who prefer to not use the logr
 | |
| APIs directly can use those instead and the resulting code will still be
 | |
| interoperable with logr.
 | |
| 
 | |
| ## FAQ
 | |
| 
 | |
| ### Conceptual
 | |
| 
 | |
| #### Why structured logging?
 | |
| 
 | |
| - **Structured logs are more easily queryable**: Since you've got
 | |
|   key-value pairs, it's much easier to query your structured logs for
 | |
|   particular values by filtering on the contents of a particular key --
 | |
|   think searching request logs for error codes, Kubernetes reconcilers for
 | |
|   the name and namespace of the reconciled object, etc.
 | |
| 
 | |
| - **Structured logging makes it easier to have cross-referenceable logs**:
 | |
|   Similarly to searchability, if you maintain conventions around your
 | |
|   keys, it becomes easy to gather all log lines related to a particular
 | |
|   concept.
 | |
| 
 | |
| - **Structured logs allow better dimensions of filtering**: if you have
 | |
|   structure to your logs, you've got more precise control over how much
 | |
|   information is logged -- you might choose in a particular configuration
 | |
|   to log certain keys but not others, only log lines where a certain key
 | |
|   matches a certain value, etc., instead of just having v-levels and names
 | |
|   to key off of.
 | |
| 
 | |
| - **Structured logs better represent structured data**: sometimes, the
 | |
|   data that you want to log is inherently structured (think tuple-link
 | |
|   objects.)  Structured logs allow you to preserve that structure when
 | |
|   outputting.
 | |
| 
 | |
| #### Why V-levels?
 | |
| 
 | |
| **V-levels give operators an easy way to control the chattiness of log
 | |
| operations**.  V-levels provide a way for a given package to distinguish
 | |
| the relative importance or verbosity of a given log message.  Then, if
 | |
| a particular logger or package is logging too many messages, the user
 | |
| of the package can simply change the v-levels for that library.
 | |
| 
 | |
| #### Why not named levels, like Info/Warning/Error?
 | |
| 
 | |
| Read [Dave Cheney's post][warning-makes-no-sense].  Then read [Differences
 | |
| from Dave's ideas](#differences-from-daves-ideas).
 | |
| 
 | |
| #### Why not allow format strings, too?
 | |
| 
 | |
| **Format strings negate many of the benefits of structured logs**:
 | |
| 
 | |
| - They're not easily searchable without resorting to fuzzy searching,
 | |
|   regular expressions, etc.
 | |
| 
 | |
| - They don't store structured data well, since contents are flattened into
 | |
|   a string.
 | |
| 
 | |
| - They're not cross-referenceable.
 | |
| 
 | |
| - They don't compress easily, since the message is not constant.
 | |
| 
 | |
| (Unless you turn positional parameters into key-value pairs with numerical
 | |
| keys, at which point you've gotten key-value logging with meaningless
 | |
| keys.)
 | |
| 
 | |
| ### Practical
 | |
| 
 | |
| #### Why key-value pairs, and not a map?
 | |
| 
 | |
| Key-value pairs are *much* easier to optimize, especially around
 | |
| allocations.  Zap (a structured logger that inspired logr's interface) has
 | |
| [performance measurements](https://github.com/uber-go/zap#performance)
 | |
| that show this quite nicely.
 | |
| 
 | |
| While the interface ends up being a little less obvious, you get
 | |
| potentially better performance, plus avoid making users type
 | |
| `map[string]string{}` every time they want to log.
 | |
| 
 | |
| #### What if my V-levels differ between libraries?
 | |
| 
 | |
| That's fine.  Control your V-levels on a per-logger basis, and use the
 | |
| `WithName` method to pass different loggers to different libraries.
 | |
| 
 | |
| Generally, you should take care to ensure that you have relatively
 | |
| consistent V-levels within a given logger, however, as this makes deciding
 | |
| on what verbosity of logs to request easier.
 | |
| 
 | |
| #### But I really want to use a format string!
 | |
| 
 | |
| That's not actually a question.  Assuming your question is "how do
 | |
| I convert my mental model of logging with format strings to logging with
 | |
| constant messages":
 | |
| 
 | |
| 1. Figure out what the error actually is, as you'd write in a TL;DR style,
 | |
|    and use that as a message.
 | |
| 
 | |
| 2. For every place you'd write a format specifier, look to the word before
 | |
|    it, and add that as a key value pair.
 | |
| 
 | |
| For instance, consider the following examples (all taken from spots in the
 | |
| Kubernetes codebase):
 | |
| 
 | |
| - `klog.V(4).Infof("Client is returning errors: code %v, error %v",
 | |
|   responseCode, err)` becomes `logger.Error(err, "client returned an
 | |
|   error", "code", responseCode)`
 | |
| 
 | |
| - `klog.V(4).Infof("Got a Retry-After %ds response for attempt %d to %v",
 | |
|   seconds, retries, url)` becomes `logger.V(4).Info("got a retry-after
 | |
|   response when requesting url", "attempt", retries, "after
 | |
|   seconds", seconds, "url", url)`
 | |
| 
 | |
| If you *really* must use a format string, use it in a key's value, and
 | |
| call `fmt.Sprintf` yourself.  For instance: `log.Printf("unable to
 | |
| reflect over type %T")` becomes `logger.Info("unable to reflect over
 | |
| type", "type", fmt.Sprintf("%T"))`.  In general though, the cases where
 | |
| this is necessary should be few and far between.
 | |
| 
 | |
| #### How do I choose my V-levels?
 | |
| 
 | |
| This is basically the only hard constraint: increase V-levels to denote
 | |
| more verbose or more debug-y logs.
 | |
| 
 | |
| Otherwise, you can start out with `0` as "you always want to see this",
 | |
| `1` as "common logging that you might *possibly* want to turn off", and
 | |
| `10` as "I would like to performance-test your log collection stack."
 | |
| 
 | |
| Then gradually choose levels in between as you need them, working your way
 | |
| down from 10 (for debug and trace style logs) and up from 1 (for chattier
 | |
| info-type logs). For reference, slog pre-defines -4 for debug logs
 | |
| (corresponds to 4 in logr), which matches what is
 | |
| [recommended for Kubernetes](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md#what-method-to-use).
 | |
| 
 | |
| #### How do I choose my keys?
 | |
| 
 | |
| Keys are fairly flexible, and can hold more or less any string
 | |
| value. For best compatibility with implementations and consistency
 | |
| with existing code in other projects, there are a few conventions you
 | |
| should consider.
 | |
| 
 | |
| - Make your keys human-readable.
 | |
| - Constant keys are generally a good idea.
 | |
| - Be consistent across your codebase.
 | |
| - Keys should naturally match parts of the message string.
 | |
| - Use lower case for simple keys and
 | |
|   [lowerCamelCase](https://en.wiktionary.org/wiki/lowerCamelCase) for
 | |
|   more complex ones. Kubernetes is one example of a project that has
 | |
|   [adopted that
 | |
|   convention](https://github.com/kubernetes/community/blob/HEAD/contributors/devel/sig-instrumentation/migration-to-structured-logging.md#name-arguments).
 | |
| 
 | |
| While key names are mostly unrestricted (and spaces are acceptable),
 | |
| it's generally a good idea to stick to printable ascii characters, or at
 | |
| least match the general character set of your log lines.
 | |
| 
 | |
| #### Why should keys be constant values?
 | |
| 
 | |
| The point of structured logging is to make later log processing easier.  Your
 | |
| keys are, effectively, the schema of each log message.  If you use different
 | |
| keys across instances of the same log line, you will make your structured logs
 | |
| much harder to use.  `Sprintf()` is for values, not for keys!
 | |
| 
 | |
| #### Why is this not a pure interface?
 | |
| 
 | |
| The Logger type is implemented as a struct in order to allow the Go compiler to
 | |
| optimize things like high-V `Info` logs that are not triggered.  Not all of
 | |
| these implementations are implemented yet, but this structure was suggested as
 | |
| a way to ensure they *can* be implemented.  All of the real work is behind the
 | |
| `LogSink` interface.
 | |
| 
 | |
| [warning-makes-no-sense]: http://dave.cheney.net/2015/11/05/lets-talk-about-logging
 |