Integrate BACKBEAT SDK and resolve KACHING license validation

Major integrations and fixes:
- Added BACKBEAT SDK integration for P2P operation timing
- Implemented beat-aware status tracking for distributed operations
- Added Docker secrets support for secure license management
- Resolved KACHING license validation via HTTPS/TLS
- Updated docker-compose configuration for clean stack deployment
- Disabled rollback policies to prevent deployment failures
- Added license credential storage (CHORUS-DEV-MULTI-001)

Technical improvements:
- BACKBEAT P2P operation tracking with phase management
- Enhanced configuration system with file-based secrets
- Improved error handling for license validation
- Clean separation of KACHING and CHORUS deployment stacks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
anthonyrawlins
2025-09-06 07:56:26 +10:00
parent 543ab216f9
commit 9bdcbe0447
4730 changed files with 1480093 additions and 1916 deletions

View File

@@ -0,0 +1,57 @@
Why does this package exist?
----------------------------
The `linking/cid` package bends the `github.com/ipfs/go-cid` package into conforming to the `ipld.Link` interface.
The `linking/cid` package also contains factory functions for `ipld.LinkSystem`.
These LinkSystem will be constructed with `EncoderChooser`, `DecoderChooser`, and `HasherChooser` funcs
which will use multicodec registries and multihash registries respectively.
### Why not use go-cid directly?
We need a "Link" interface in the root `ipld` package or things just aren't definable.
But we don't want the root `ipld.Link` concept to directly map to `go-cid.Cid` for several reasons:
1. We might want to revisit the go-cid library. Possibly in the "significantly breaking changes" sense.
- It's also not clear when we might do this -- and if we do, the transition period will be *long* because it's a highly-depended-upon library.
- See below for some links to a gist that discusses why.
2. We might want to extend the concept of linking to more than just plain CIDs.
- This is hypothetical at present -- but an often-discussed example is "what if CID+Path was also a Link?"
3. We might sometimes want to use IPLD libraries without using any CID implementation at all.
- e.g. it's totally believable to want to use IPLD libraries for handling JSON and CBOR, even if you don't want IPLD linking.
- if the CID packages were cheap enough, maybe this concern would fade -- but right now, they're **definitely** not; the transitive dependency tree of go-cid is *huge*.
#### If go-cid is revisited, what might that look like?
No idea. (At least, not in a committal way.)
https://gist.github.com/warpfork/e871b7fee83cb814fb1f043089983bb3#existing-implementations
gathers some reflections on the problems that would be nice to solve, though.
https://gist.github.com/warpfork/e871b7fee83cb814fb1f043089983bb3#file-cid-go
contains a draft outline of what a revisited API could look like,
but note that at the time of writing, it is not strongly ratified nor in any way committed to.
At any rate, though, the operative question for this package is:
if we do revisit go-cid, how are we going to make the transition managable?
It seems unlikely we'd be able to make the transition manageable without some interface, somewhere.
So we might as well draw that line at `ipld.Link`.
(I hypothesize that a transition story might involve two CID packages,
which could grow towards a shared interface,
doing so in a way that's purely additive in the established `go-cid` package.
We'd need two separate go modules to do this, since the aim is reducing dependency bloat for those that use the new one.
The shared interface in this story could have more info than `ipld.Link` does now,
but would nonetheless still certainly be an interface in order to support the separation of modules.)
### Why are LinkSystem factory functions here, instead of in the main IPLD package?
Same reason as why we don't use go-cid directly.
If we put these LinkSystem defaults in the root `ipld` package,
we'd bring on all the transitive dependencies of `go-cid` onto an user of `ipld` unconditionally...
and we don't want to do that.
You know that Weird Al song "It's all about the pentiums"?
Retune that in your mind to "It's all about dependencies".

View File

@@ -0,0 +1,76 @@
package cidlink
import (
"fmt"
cid "github.com/ipfs/go-cid"
"github.com/ipld/go-ipld-prime/datamodel"
multihash "github.com/multiformats/go-multihash"
)
var (
_ datamodel.Link = Link{}
_ datamodel.LinkPrototype = LinkPrototype{}
)
// Link implements the datamodel.Link interface using a CID.
// See https://github.com/ipfs/go-cid for more information about CIDs.
//
// When using this value, typically you'll use it as `Link`, and not `*Link`.
// This includes when handling the value as an `datamodel.Link` interface -- the non-pointer form is typically preferable.
// This is because the datamodel.Link inteface is often desirable to be able to use as a golang map key,
// and in that context, pointers would not result in the desired behavior.
type Link struct {
cid.Cid
}
func (lnk Link) Prototype() datamodel.LinkPrototype {
return LinkPrototype{lnk.Cid.Prefix()}
}
func (lnk Link) String() string {
return lnk.Cid.String()
}
func (lnk Link) Binary() string {
return lnk.Cid.KeyString()
}
type LinkPrototype struct {
cid.Prefix
}
func (lp LinkPrototype) BuildLink(hashsum []byte) datamodel.Link {
// Does this method body look surprisingly complex? I agree.
// We actually have to do all this work. The go-cid package doesn't expose a constructor that just lets us directly set the bytes and the prefix numbers next to each other.
// No, `cid.Prefix.Sum` is not the method you are looking for: that expects the whole data body.
// Most of the logic here is the same as the body of `cid.Prefix.Sum`; we just couldn't get at the relevant parts without copypasta.
// There is also some logic that's sort of folded in from the go-multihash module. This is really a mess.
// The go-cid package needs review. So does go-multihash. Their responsibilies are not well compartmentalized and they don't play well with other stdlib golang interfaces.
p := lp.Prefix
length := p.MhLength
if p.MhType == multihash.IDENTITY {
length = -1
}
if p.Version == 0 && (p.MhType != multihash.SHA2_256 ||
(p.MhLength != 32 && p.MhLength != -1)) {
panic(fmt.Errorf("invalid cid v0 prefix"))
}
if length != -1 {
hashsum = hashsum[:p.MhLength]
}
mh, err := multihash.Encode(hashsum, p.MhType)
if err != nil {
panic(err) // No longer possible, but multihash still returns an error for legacy reasons.
}
switch lp.Prefix.Version {
case 0:
return Link{cid.NewCidV0(mh)}
case 1:
return Link{cid.NewCidV1(p.Codec, mh)}
default:
panic(fmt.Errorf("invalid cid version"))
}
}

View File

@@ -0,0 +1,71 @@
package cidlink
import (
"fmt"
"hash"
"github.com/multiformats/go-multihash/core"
"github.com/ipld/go-ipld-prime/codec"
"github.com/ipld/go-ipld-prime/datamodel"
"github.com/ipld/go-ipld-prime/linking"
"github.com/ipld/go-ipld-prime/multicodec"
)
// DefaultLinkSystem returns a linking.LinkSystem which uses cidlink.Link for datamodel.Link.
// During selection of encoders, decoders, and hashers, it examines the multicodec indicator numbers and multihash indicator numbers from the CID,
// and uses the default global multicodec registry (see the go-ipld-prime/multicodec package) for resolving codec implementations,
// and the default global multihash registry (see the go-multihash/core package) for resolving multihash implementations.
//
// No storage functions are present in the returned LinkSystem.
// The caller can assign those themselves as desired.
func DefaultLinkSystem() linking.LinkSystem {
return LinkSystemUsingMulticodecRegistry(multicodec.DefaultRegistry)
}
// LinkSystemUsingMulticodecRegistry is similar to DefaultLinkSystem, but accepts a multicodec.Registry as a parameter.
//
// This can help create a LinkSystem which uses different multicodec implementations than the global registry.
// (Sometimes this can be desired if you want some parts of a program to support a more limited suite of codecs than other parts of the program,
// or needed to use a different multicodec registry than the global one for synchronization purposes, or etc.)
func LinkSystemUsingMulticodecRegistry(mcReg multicodec.Registry) linking.LinkSystem {
return linking.LinkSystem{
EncoderChooser: func(lp datamodel.LinkPrototype) (codec.Encoder, error) {
switch lp2 := lp.(type) {
case LinkPrototype:
fn, err := mcReg.LookupEncoder(lp2.GetCodec())
if err != nil {
return nil, err
}
return fn, nil
default:
return nil, fmt.Errorf("this encoderChooser can only handle cidlink.LinkPrototype; got %T", lp)
}
},
DecoderChooser: func(lnk datamodel.Link) (codec.Decoder, error) {
lp := lnk.Prototype()
switch lp2 := lp.(type) {
case LinkPrototype:
fn, err := mcReg.LookupDecoder(lp2.GetCodec())
if err != nil {
return nil, err
}
return fn, nil
default:
return nil, fmt.Errorf("this decoderChooser can only handle cidlink.LinkPrototype; got %T", lp)
}
},
HasherChooser: func(lp datamodel.LinkPrototype) (hash.Hash, error) {
switch lp2 := lp.(type) {
case LinkPrototype:
h, err := multihash.GetHasher(lp2.MhType)
if err != nil {
return nil, fmt.Errorf("no hasher registered for multihash indicator 0x%x: %w", lp2.MhType, err)
}
return h, nil
default:
return nil, fmt.Errorf("this hasherChooser can only handle cidlink.LinkPrototype; got %T", lp)
}
},
}
}

View File

@@ -0,0 +1,56 @@
package cidlink
import (
"bytes"
"fmt"
"io"
"os"
"github.com/ipld/go-ipld-prime/datamodel"
"github.com/ipld/go-ipld-prime/linking"
)
// Memory is a simple in-memory storage for cidlinks. It's the same as `storage.Memory`
// but uses typical multihash semantics used when reading/writing cidlinks.
//
// Using multihash as the storage key rather than the whole CID will remove the
// distinction between CIDv0 and their CIDv1 counterpart. It also removes the
// distinction between CIDs where the multihash is the same but the codec is
// different, e.g. `dag-cbor` and a `raw` version of the same data.
type Memory struct {
Bag map[string][]byte
}
func (store *Memory) beInitialized() {
if store.Bag != nil {
return
}
store.Bag = make(map[string][]byte)
}
func (store *Memory) OpenRead(lnkCtx linking.LinkContext, lnk datamodel.Link) (io.Reader, error) {
store.beInitialized()
cl, ok := lnk.(Link)
if !ok {
return nil, fmt.Errorf("incompatible link type: %T", lnk)
}
data, exists := store.Bag[string(cl.Hash())]
if !exists {
return nil, os.ErrNotExist
}
return bytes.NewReader(data), nil
}
func (store *Memory) OpenWrite(lnkCtx linking.LinkContext) (io.Writer, linking.BlockWriteCommitter, error) {
store.beInitialized()
buf := bytes.Buffer{}
return &buf, func(lnk datamodel.Link) error {
cl, ok := lnk.(Link)
if !ok {
return fmt.Errorf("incompatible link type: %T", lnk)
}
store.Bag[string(cl.Hash())] = buf.Bytes()
return nil
}, nil
}