mirror of
https://codeberg.org/forgejo/forgejo
synced 2025-07-06 10:02:52 +02:00
bd6f3243ab
# Fix OCI artifact uploads with`oras` ## Problem ORAS (OCI Registry As Storage) artifact uploads were failing with several HTTP-related errors when pushing to Forgejo's container registry. This prevented users from storing OCI artifacts like `artifacthub-repo.yaml` in commands like `oras push [...] artifacthub-repo.yaml:application/vnd.cncf.artifacthub.repository-metadata.layer.v1.yaml`. This has been discussed previously in https://github.com/go-gitea/gitea/issues/25846 ## Root Causes and Fixes ### 1. Missing Content-Length for Empty Blobs **Issue**: Empty blobs (size 0) were not getting the required `Content-Length: 0` header, causing ORAS to fail with "unknown response Content-Length". **Fix**: Changed the condition in `setResponseHeaders` from `if h.ContentLength != 0` to `if h.ContentLength >= 0` to ensure the Content-Length header is always set for valid blob sizes. ```go // Before if h.ContentLength != 0 { resp.Header().Set("Content-Length", strconv.FormatInt(h.ContentLength, 10)) } // After if h.ContentLength >= 0 { resp.Header().Set("Content-Length", strconv.FormatInt(h.ContentLength, 10)) } ``` ### 2. Content-Length Mismatch in JSON Error Responses **Issue**: The `jsonResponse` function was calling `WriteHeader()` before writing JSON content, causing "wrote more than the declared Content-Length" errors when the HTTP stack calculated a different Content-Length than what was actually written. **Fix**: Modified `jsonResponse` to buffer JSON content first, calculate the exact Content-Length, then write the complete response. ### 3. Incomplete HTTP Responses in Error Handling **Issue**: The `apiError` function was only setting response headers without writing any response body, causing EOF errors when clients expected a complete HTTP response. **Fix**: Updated `apiError` to write proper JSON error responses following the OCI Distribution Specification format with `code` and `message` fields. ### 4. Empty Config Blob Handling for OCI Artifacts **Issue**: OCI artifacts often have empty config blobs (required by spec but contain no data). The JSON decoder was failing with EOF when trying to parse these empty configs. **Fix**: Added EOF handling in `parseOCIImageConfig` to return a valid default metadata object for empty config blobs. ```go if err := json.NewDecoder(r).Decode(&image); err != nil { // Handle empty config blobs (common in OCI artifacts) if err == io.EOF { return &Metadata{ Type: TypeOCI, Platform: DefaultPlatform, }, nil } return nil, err } ``` ## Testing Verified that ORAS artifact uploads now work correctly: ```bash oras push registry/owner/package:artifacthub.io \ --config /dev/null:application/vnd.cncf.artifacthub.config.v1+yaml \ artifacthub-repo.yaml:application/vnd.cncf.artifacthub.repository-metadata.layer.v1.yaml ``` ### Tests - I added test coverage for Go changes... - [x] in their respective `*_test.go` for unit tests. - [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server. - I added test coverage for JavaScript changes... - [ ] in `web_src/js/*.test.js` if it can be unit tested. - [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)). ### Documentation - [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change. - [x] I did not document these changes and I do not expect someone else to do it. ### Release notes - [ ] I do not want this change to show in the release notes. - [x] I want the title to show in the release notes with a link to this pull request. - [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title. Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8070 Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org> Co-authored-by: pat-s <patrick.schratz@gmail.com> Co-committed-by: pat-s <patrick.schratz@gmail.com> |
||
|---|---|---|
| .. | ||
| alpine | ||
| alt | ||
| arch | ||
| cargo | ||
| chef | ||
| composer | ||
| conan | ||
| conda | ||
| container | ||
| cran | ||
| debian | ||
| generic | ||
| goproxy | ||
| helm | ||
| helper | ||
| maven | ||
| npm | ||
| nuget | ||
| pub | ||
| pypi | ||
| rpm | ||
| rubygems | ||
| swift | ||
| vagrant | ||
| api.go | ||
| README.md | ||
Gitea Package Registry
This document gives a brief overview how the package registry is organized in code.
Structure
The package registry code is divided into multiple modules to split the functionality and make code reuse possible.
| Module | Description |
|---|---|
models/packages |
Common methods and models used by all registry types |
models/packages/<type> |
Methods used by specific registry type. There should be no need to use type specific models. |
modules/packages |
Common methods and types used by multiple registry types |
modules/packages/<type> |
Registry type specific methods and types (e.g. metadata extraction of package files) |
routers/api/packages |
Route definitions for all registry types |
routers/api/packages/<type> |
Route implementation for a specific registry type |
services/packages |
Helper methods used by registry types to handle common tasks like package creation and deletion in routers |
services/packages/<type> |
Registry type specific methods used by routers and services |
Models
Every package registry implementation uses the same underlying models:
| Model | Description |
|---|---|
Package |
The root of a package providing values fixed for every version (e.g. the package name) |
PackageVersion |
A version of a package containing metadata (e.g. the package description) |
PackageFile |
A file of a package describing its content (e.g. file name) |
PackageBlob |
The content of a file (may be shared by multiple files) |
PackageProperty |
Additional properties attached to Package, PackageVersion or PackageFile (e.g. used if metadata is needed for routing) |
The following diagram shows the relationship between the models:
Package <1---*> PackageVersion <1---*> PackageFile <*---1> PackageBlob
Adding a new package registry type
Before adding a new package registry type have a look at the existing implementation to get an impression of how it could work.
Most registry types offer endpoints to retrieve the metadata, upload and download package files.
The upload endpoint is often the heavy part because it must validate the uploaded blob, extract metadata and create the models.
The methods to validate and extract the metadata should be added in the modules/packages/<type> package.
If the upload is valid the methods in services/packages allow to store the upload and create the corresponding models.
It depends if the registry type allows multiple files per package version which method should be called:
CreatePackageAndAddFile: error if package version already existsCreatePackageOrAddFileToExisting: error if file already existsAddFileToExistingPackage: error if package version does not exist or file already exists
services/packages also contains helper methods to download a file or to remove a package version.
There are no helper methods for metadata endpoints because they are very type specific.