Skip to content

Add support for OCI image manifests and indexes #584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aman-coder03 wants to merge 2 commits into
base: master
Choose a base branch
from
+151 −13
Open

Add support for OCI image manifests and indexes #584

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bcfccfe
support for OCI image manifests and indexes
aman-coder03 Apr 6, 2026
a4a1082
addressing review comments
aman-coder03 Apr 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ require (
github.com/jmoiron/sqlx v0.0.0-20190319043955-cdf62fdf55f6
github.com/mattn/go-sqlite3 v1.14.0
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.0.2
github.com/pressly/goose v2.6.0+incompatible
github.com/satori/go.uuid v1.2.0
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72
Expand Down Expand Up @@ -88,7 +89,6 @@ require (
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/opencontainers/runc v1.0.2 // indirect
github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417 // indirect
github.com/pkg/errors v0.9.1 // indirect
Expand Down
6 changes: 5 additions & 1 deletion proxy/proxyserver/preheat.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,11 @@ import (
"github.com/uber/kraken/utils/log"
)

var _manifestRegexp = regexp.MustCompile(`^application/vnd.docker.distribution.manifest.v\d\+(json|prettyjws)`)
var _manifestRegexp = regexp.MustCompile(
`^application/vnd\.docker\.distribution\.manifest\.v\d\+(json|prettyjws)` +
`|^application/vnd\.oci\.image\.manifest\.v1\+json` +
`|^application/vnd\.oci\.image\.index\.v1\+json`,
)
Comment on lines +35 to +39
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding OCI index media types to the preheat event filter means Kraken will start processing push events whose manifest.References() typically include only child manifests (not layer/config blobs). With the current process() implementation, this can result in preheating only nested manifests rather than the actual layers unless additional manifest push events are also received. Consider either keeping index/list types out of this filter, or recursively fetching referenced manifests and preheating their references to ensure layers are warmed.

Copilot uses AI. Check for mistakes.

// PreheatHandler defines the handler of preheat.
type PreheatHandler struct {
Expand Down
64 changes: 59 additions & 5 deletions utils/dockerutil/dockerutil.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@

"github.com/docker/distribution"
"github.com/docker/distribution/manifest/manifestlist"
"github.com/docker/distribution/manifest/ocischema"
"github.com/docker/distribution/manifest/schema2"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/uber/kraken/core"
)

Expand All @@ -35,13 +37,26 @@
return nil, core.Digest{}, fmt.Errorf("read: %s", err)
}

// Try Docker v2 manifest.
manifest, d, err := ParseManifestV2(b)
if err == nil {
return manifest, d, err
return manifest, d, nil
}

// Retry with v2 manifest list.
return ParseManifestV2List(b)
// Try Docker v2 manifest list.
manifest, d, err = ParseManifestV2List(b)
if err == nil {
return manifest, d, nil
}

// Try OCI image manifest.
manifest, d, err = ParseManifestOCI(b)
if err == nil {
return manifest, d, nil
}

// Try OCI image index.
return ParseManifestOCIIndex(b)
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ParseManifest falls through multiple unmarshal attempts but, if all attempts fail, it returns only the final OCI-index error. This can produce misleading error messages (e.g., reporting an OCI-index parse failure for non-OCI inputs) and makes debugging harder. Consider returning an aggregated error that includes failures from each attempted format, or detect the manifest mediaType from JSON first and unmarshal only the matching type.

Copilot uses AI. Check for mistakes.
}

// ParseManifestV2 returns a parsed v2 manifest and its digest.
Expand Down Expand Up @@ -86,6 +101,40 @@
return manifestList, d, nil
}

// ParseManifestOCI returns a parsed OCI image manifest and its digest.
func ParseManifestOCI(bytes []byte) (distribution.Manifest, core.Digest, error) {
manifest, desc, err := distribution.UnmarshalManifest(specs.MediaTypeImageManifest, bytes)
if err != nil {
return nil, core.Digest{}, fmt.Errorf("unmarshal OCI manifest: %s", err)
}
_, ok := manifest.(*ocischema.DeserializedManifest)
if !ok {
return nil, core.Digest{}, errors.New("expected ocischema.DeserializedManifest")
}
d, err := core.ParseSHA256Digest(string(desc.Digest))
if err != nil {
return nil, core.Digest{}, fmt.Errorf("parse digest: %s", err)
}
return manifest, d, nil
}

// ParseManifestOCIIndex returns a parsed OCI image index and its digest.
func ParseManifestOCIIndex(bytes []byte) (distribution.Manifest, core.Digest, error) {
manifestIndex, desc, err := distribution.UnmarshalManifest(specs.MediaTypeImageIndex, bytes)
if err != nil {
return nil, core.Digest{}, fmt.Errorf("unmarshal OCI image index: %s", err)
}
_, ok := manifestIndex.(*manifestlist.DeserializedManifestList)
if !ok {
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message here exposes the internal Docker distribution type name rather than describing the OCI concept being validated. Consider rewording to something user-facing like "expected OCI image index" (and optionally including the actual type via %T) to make troubleshooting easier.

Suggested change
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
return nil, core.Digest{}, fmt.Errorf("expected OCI image index, got %T", manifestIndex)

Copilot uses AI. Check for mistakes.
}
Comment on lines +127 to +130
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ParseManifestOCIIndex asserts the returned type but does not validate SchemaVersion (unlike ParseManifestV2List, which checks SchemaVersion==2 on the same DeserializedManifestList type). For consistency and to avoid accepting unsupported index versions, store the cast result and validate SchemaVersion before returning.

Suggested change
_, ok := manifestIndex.(*manifestlist.DeserializedManifestList)
if !ok {
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
}
deserializedManifestIndex, ok := manifestIndex.(*manifestlist.DeserializedManifestList)
if !ok {
return nil, core.Digest{}, errors.New("expected manifestlist.DeserializedManifestList for OCI index")
}
version := deserializedManifestIndex.SchemaVersion
if version != 2 {
return nil, core.Digest{}, fmt.Errorf("unsupported OCI image index version: %d", version)
}

Copilot uses AI. Check for mistakes.
d, err := core.ParseSHA256Digest(string(desc.Digest))
if err != nil {
return nil, core.Digest{}, fmt.Errorf("parse digest: %s", err)
}
return manifestIndex, d, nil
}

// GetManifestReferences returns a list of references by a V2 manifest
func GetManifestReferences(manifest distribution.Manifest) ([]core.Digest, error) {
var refs []core.Digest
Expand All @@ -100,5 +149,10 @@
}

func GetSupportedManifestTypes() string {
return fmt.Sprintf("%s,%s", _v2ManifestType, _v2ManifestListType)
}
return fmt.Sprintf("%s,%s,%s,%s",
_v2ManifestType,
_v2ManifestListType,
specs.MediaTypeImageManifest,
specs.MediaTypeImageIndex,
)
}

Check failure on line 158 in utils/dockerutil/dockerutil.go

View workflow job for this annotation

GitHub Actions / lint 

File is not properly formatted (gofmt)
66 changes: 66 additions & 0 deletions utils/dockerutil/dockerutil_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package dockerutil_test

import (
"bytes"
"testing"

"github.com/docker/distribution/manifest/manifestlist"
Expand Down Expand Up @@ -88,3 +89,68 @@ func TestParseManifestV2List(t *testing.T) {
})
}
}

var testOCIManifestBytes = []byte(`{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"size": 985,
"digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b"
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"size": 153263,
"digest": "sha256:62d8908bee94c202b2d35224a221aaa2058318bfa9879fa541efaecba272331b"
}
]
}`)

var testOCIIndexBytes = []byte(`{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 985,
"digest": "sha256:1a9ec845ee94c202b2d5da74a24f0ed2058318bfa9879fa541efaecba272e86b",
"platform": {
"architecture": "amd64",
"os": "linux"
}
}
]
}`)

func TestParseManifestOCI(t *testing.T) {
require := require.New(t)
manifest, d, err := dockerutil.ParseManifestOCI(testOCIManifestBytes)
require.NoError(err)
mediaType, _, err := manifest.Payload()
require.NoError(err)
require.Equal("application/vnd.oci.image.manifest.v1+json", mediaType)
Comment on lines +130 to +134
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test only asserts that the returned digest uses the sha256 algorithm, which would still pass if the digest value is incorrect. Consider asserting the full digest string matches the expected digest of testOCIManifestBytes (and adding a negative test case to ensure ParseManifestOCI rejects non-OCI media types).

Copilot uses AI. Check for mistakes.
require.Equal("sha256", d.Algo())
}

func TestParseManifestOCIIndex(t *testing.T) {
require := require.New(t)
manifest, d, err := dockerutil.ParseManifestOCIIndex(testOCIIndexBytes)
require.NoError(err)
mediaType, _, err := manifest.Payload()
require.NoError(err)
require.Equal("application/vnd.oci.image.index.v1+json", mediaType)
Comment on lines +147 to +151
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test only checks the digest algorithm rather than the full digest value, and it doesn't exercise error paths (e.g., passing a manifest instead of an index). Consider asserting the exact digest string for testOCIIndexBytes and adding a failure-case test to confirm ParseManifestOCIIndex rejects non-index inputs.

Copilot uses AI. Check for mistakes.
require.Equal("sha256", d.Algo())
}

func TestParseManifestOCIViaParseManifest(t *testing.T) {
require := require.New(t)

_, d, err := dockerutil.ParseManifest(bytes.NewReader(testOCIManifestBytes))
require.NoError(err)
require.Equal("sha256", d.Algo())

_, d, err = dockerutil.ParseManifest(bytes.NewReader(testOCIIndexBytes))
require.NoError(err)
require.Equal("sha256", d.Algo())
}
Loading