Add support for --from-delta for bootc switch and update - #2449
Open
alexlarsson wants to merge 6 commits into
Open
Add support for --from-delta for bootc switch and update#2449alexlarsson wants to merge 6 commits into
alexlarsson wants to merge 6 commits into
Conversation
github-actions
Bot
added
area/install
alexlarsson
force-pushed
the
from-delta
branch
from
September 10, 2026 14:52
b57b08a to
e99dc64
Compare
The diff_id appended for the generated layer was the bare hex encoding rather than the `sha256:...` form the spec requires. Our own importer does not look at it, so nothing noticed, but any other consumer of an image built by this helper trips over it. Assisted-by: AI Signed-off-by: Alexander Larsson <alexl@redhat.com>
This adds a `--from-delta PATH` flag to `bootc upgrade` and `bootc switch` that applies an oci-delta in one step. This commit add support only for the composefs backend, and uses the native delta pull support in composefs-rs. Later commits add ostree backend support. Basic parsing of the delta file is done by the oci-delta rust bindings. See https://github.com/containers/oci-delta for details of the format. Unified storage cannot apply a delta yet and will fail. Real support for this would require delta support in containers/storage. Generated-by: AI Signed-off-by: Alexander Larsson <alexl@redhat.com>
The importer has always fetched both the image metadata and the layer bytes over the same containers-image-proxy connection. To apply an OCI delta we need to split those apart: the manifest and config come out of the delta file, and each layer is either already in the repository or reconstructed locally from a binary patch. No network access at all. Introduce a `LayerSource` trait, which is just the existing `fetch_layer` signature behind a trait object, and a `ProxyLayerSource` implementing today's behaviour. `PreparedImport` now carries a LayerSource instead of a bare `OpenedImage`, and `ImageImporter::prepare_from_manifest` builds a `PreparedImport` from a manifest and config the caller already has, with the layers coming from whatever source it passes in. `ImageImporter::new_without_proxy` skips spawning skopeo entirely, so an importer that will never touch the network does not need it installed. Along the way, `prepare_internal` splits into `check_sigverify` and `diff_previous_state` so both prepare paths share the "is this already imported?" logic, and `cache_pending` moves into `create_prepared_import` where both paths reach it. `ProxyLayerSource` also fetches the proxy's layer info once and caches it, where before both `unencapsulate_base()` and `import()` asked for it separately. Assisted-by: AI Signed-off-by: Alexander Larsson <alexl@redhat.com>
We will need this later. We're reusing most of the old list_container_deployment_manifests() for this. Signed-off-by: Alexander Larsson <alexl@redhat.com>
`bootc upgrade --from-delta` and `bootc switch --from-delta` worked only with the composefs backend. Wire them up for ostree too. A tar-diff patch reads the source image by *path*, not from the original tar byte stream, so the source does not have to be a container layer: an ostree commit holding the same filesystem works just as well. `OstreeDataSource` serves file content out of a commit, and `DeltaLayerSource` plugs that into the container importer in place of the image proxy, so nothing is fetched. ostree does not store an image's root filesystem verbatim - the tar importer moves `/etc` to `/usr/etc`, and `/var` to `/usr/share/factory/var` on ostree older than v2024.3 - so a source path is tried at each of the locations that importer could have put it. Each reconstructed layer is streamed to the importer over a pipe, with the reconstruction itself running as the driver future that `join_fetch` already polls concurrently, so a multi-GB base layer never has to be spooled to disk. The source image is looked up by config digest among the images in the repository, and an absent source is fatal: there is no guarantee of a network connection at the point a delta is applied, so falling back to the registry would defeat the purpose. Byte-level progress is not reported for delta layers: the reconstructed bytes are uncompressed, so counting them against the descriptor's compressed size would overshoot. Per-layer start/completion still is. A layer's ref is written as the layer is unpacked, i.e. before the driver future has had the chance to report a diff_id mismatch, so a failed apply can leave a ref to unverified content that a retry would reuse. Prune the unreferenced layers after a failed delta apply. Also, `switch --from-delta` no longer short-circuits when the image specification is unchanged - a delta names one specific target digest, and switching to the reference you are already tracking is exactly how it is normally used. Assisted-by: AI Signed-off-by: Alexander Larsson <alexl@redhat.com>
Three tests covering the whole path from a delta on disk to an imported image, at increasing degrees of realism: `test_apply_delta_whole_layers` builds a delta whose patches are the target layers carried verbatim. The format allows that, and the layer reconstruction dispatches on media type, so this exercises parsing, validation, source lookup and the import while needing no external tooling - it runs everywhere. `test_apply_delta_chunked` and `test_apply_delta_derived` use the real `oci-delta` to build the delta, and skip themselves when it is not installed. The first covers an ostree-native image, whose layers are made of repo objects under `sysroot/ostree/` that do not exist in the commit's file tree - it passes only because oci-delta is told never to ask for one. The second covers a derived layer, an ordinary root filesystem tar, which is the case that actually drives `OstreeDataSource`; the file it patches lives at `/etc/bigconf`, i.e. at a path the importer relocates, and the test asserts both that it lands at `/usr/etc/bigconf` and that the patch is far smaller than the layer, which it can only be if the content really was read back out of the source commit. `oci-delta` goes in a new optional package list, because it is not available on every distribution we build on. Assisted-by: AI Signed-off-by: Alexander Larsson <alexl@redhat.com>
alexlarsson
force-pushed
the
from-delta
branch
from
September 10, 2026 14:53
e99dc64 to
033dc0d
Compare
Contributor
Author
|
New version that is not depending on the new composefs-rs, instead using the separated out delta parsing from the rust code in containers/oci-delta. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds native support for oci-delta files to bootc, for both the native composefs backend and the ostree backend.
The composefs backend just uses the existing support in composefs-rs, whereas the ostree backend has a bit more custom code. This code uses the generic delta code in composefs-rs to avoid having to reimplement delta stuff as we're applying deltas directly. This depends on composefs/composefs-rs#394 and means we have a WIP commit that points to this MR. The intent is to get some release with that out before merging this PR.
Note: I have not really tested the native composefs codepath, because I didn't manage to get such a setup going. I will try to do this.