Buildx always pulls and never reuses images

[mattheys]

I seem to be missing something somewhere, is it possible to get the buildx plugin to not download the base images and even the buildx image itself on every single build? It seems to download moby/buildkit:buildx-stable-1 along with the base images in my Dockerfile.

I've tried several things, caching which I don't understand and don't think I setup correctly. I tried setting up a pull through cache which didn't seem to work either, probably set that up wrong too. I've set daemon_off to true and pull_image to false, then mapped the host docker socket through (probably a bad idea) but that hasn't worked either. Enabled trusted on the repository.

90% of the build time is spent downloading images that should already exist and there is no real reason to redownload.

I feel there should be a fairly simple way of setting the config to be able keep those images around and reuse them.

[pat-s]

This has nothing to do with the plugin or WP at all, this is just how docker works. If you do a docker build you will always get the latest version of the referenced tag as you need something to start with.

To pin that for the Dockerfile images, you can use sha256 references instead of tags.

moby/buildkit is the buildkit image which is not the buildx image. The latter is the plugin image which uses a fixed version of buildx which again bundles buildkit.

Caching has nothing to do with that, this relates to caching layers you've build to reuse them in future builds.

I am not sure what you're trying to achieve but

90% of the build time is spent downloading images that should already exist and there is no real reason to redownload.

sounds strange as downloading required base images is just "part of the game". If your image just has two RUN statements which finish quickly, it might be that 90% of the time is spent on bootstrapping everything, yes.

[mattheys]

Thank you for your reply.

Maybe I am assuming Docker does something that it doesn't then.

I thought it would download a base layer image the first time, but if I rebuild my app a few minutes later and that base image hasn't been updated then Docker would download the image manifest, check the layer hashes and know it already has the layers so not redownload them. A bit like if a docker compose stack has multiple instances of the same database image, then it would only download the image once and they would all re-use that.

It takes about 20s to actually compile my app, but it takes about 4 mins to download all the images, moby/buildkit, mcr.microsoft.com/dotnet/sdk:9.0 and mcr.microsoft.com/dotnet/aspnet:9.0.

I'm practically debugging on a remote machine so I'm re-building it constantly and it's frustrating waiting for it to download. If I was on a metered connection this wouldn't even be possible.

Someone suggested running a pull through cache, but this seems to not work so well with non docker hub registries. I think my best bet is going to be spinning up a registry and having a cron job keep a local registry up to date with the images I'm going to need. I'm not sure how to go about that with the moby/buildkit part as that seem to be part of the plugin itself that pulls that.

[pat-s]

It depends on the backend. If it is a static VM and and image is still there and pruned from some external process AND the image really didn't change (tags like 9.0 are rolling and might change more often than one might think) then it should reuse them, yes.

But all of this is purely docker related and not in the hands of WP. I guess you should query more information how the server you're operating on is handling image pruning etc. to understand the behavior.

I think my best bet is going to be spinning up a registry and having a cron job keep a local registry up to date with the images I'm going to need.

That doesn't sound like the right approach.

moby/buildkit

Is 90mb and shouldn't cause any substantial delay.

but it takes about 4 mins to download all the images, moby/buildkit, mcr.microsoft.com/dotnet/sdk:9.0 and mcr.microsoft.com/dotnet/aspnet:9.0.

This sounds excessive and looks like a really bad bandwidth on your server there. Yet overall I think all components are working "fine" and the issue is more with your infrastructure.