CD on K8S building multi-architecture images
• Mark Eschbach
With the upgrade to Kubernetes 1.24 and ContainerD the
buildx --platforms setup is now failing with strange messages
when attempting to execute commands. Eventually I would like to leverage multi-architecture builds within Kubernetes
anyway so this might be a good place to take the plunge. In the past most images were built for the underlying
computational platform. This is great except the Raspberry Pi node is half as fast on a single core and many fewer
cores than all of them put together.
kaniko is a project I have been keeping an eye on for a while.
Kaniko: Building images without Docker seems to be a good place to start to get that rolling. Time to spin things up. Note: 1.8.1 is the latest version as of the time of writing.
apiVersion: v1 kind: Pod metadata: name: kaniko namespace: xp-kaniko-build spec: initContainers: - name: git-clone image: docker.workshop.meschbach.org/mee/platform/golang-builder:1.18.3 command: ["sh","-c"] args: - | git clone git.meschbach.com:mee/chorinator.git /workspace volumeMounts: - name: build-context mountPath: /workspace containers: - name: kaniko image: gcr.io/kaniko-project/executor:v1.8.1 args: - "--dockerfile=/workspace/cmd/service/Dockerfile" - "--context=/workspace" - "--destination=docker.workshop.meschbach.org/mee/chorinator:experiment" volumeMounts: - name: docker-config mountPath: /kaniko/.docker/ - name: build-context mountPath: /workspace restartPolicy: Never volumes: - name: docker-config secret: secretName: internal items: - key: .dockerconfigjson path: config.json - name: build-context emptyDir: medium: "Memory" sizeLimit: "1Gi"
So this step works. It produces a viable and runnable image for the compiling platform. I was hoping the flag
would allow for cross architecture builds however it requires the hardware still. Looks like it’s possible to build
with multiple jobs then using something like
Well, the step worked. Except
kaniko will not run on ARM64. I apparently up voted
the issue last year. Excitingly there is a possible solution
multi-arch-v1.8.1 for the tag. In the upgrade to
containerd the system is now attempting to run containers
as arm7l again. Upgrading appears non-trivial so I will have to return to this problem later.