With the upgrade to Kubernetes 1.24 and ContainerD the buildx --platforms setup is now failing with strange messages when attempting to execute commands. Eventually I would like to leverage multi-architecture builds within Kubernetes anyway so this might be a good place to take the plunge. In the past most images were built for the underlying computational platform. This is great except the Raspberry Pi node is half as fast on a single core and many fewer cores than all of them put together.

kaniko is a project I have been keeping an eye on for a while.
Kaniko: Building images without Docker seems to be a good place to start to get that rolling. Time to spin things up. Note: 1.8.1 is the latest version as of the time of writing.

apiVersion: v1
kind: Pod
metadata:
  name: kaniko
  namespace: xp-kaniko-build
spec:
  initContainers:
    - name: git-clone
      image: docker.workshop.meschbach.org/mee/platform/golang-builder:1.18.3
      command: ["sh","-c"]
      args:
        - |
          git clone git.meschbach.com:mee/chorinator.git /workspace
      volumeMounts:
        - name: build-context
          mountPath: /workspace
  containers:
    - name: kaniko
      image: gcr.io/kaniko-project/executor:v1.8.1
      args:
        - "--dockerfile=/workspace/cmd/service/Dockerfile"
        - "--context=/workspace"
        - "--destination=docker.workshop.meschbach.org/mee/chorinator:experiment"
      volumeMounts:
        - name: docker-config
          mountPath: /kaniko/.docker/
        - name: build-context
          mountPath: /workspace
  restartPolicy: Never
  volumes:
    - name: docker-config
      secret:
        secretName: internal
        items:
          - key: .dockerconfigjson
            path: config.json
    - name: build-context
      emptyDir:
        medium: "Memory"
        sizeLimit: "1Gi"

So this step works. It produces a viable and runnable image for the compiling platform. I was hoping the flag --customPlatform would allow for cross architecture builds however it requires the hardware still. Looks like it’s possible to build with multiple jobs then using something like manifest-tool

Well, the step worked. Except kaniko will not run on ARM64. I apparently up voted the issue last year. Excitingly there is a possible solution of using multi-arch-v1.8.1 for the tag. In the upgrade to containerd the system is now attempting to run containers as arm7l again. Upgrading appears non-trivial so I will have to return to this problem later.

References