k8s: Pipelines in Pipelines
• Mark Eschbach
A few searches later turns out Tekton has an experimental set of CRDs
which might be able to help! Installation is fairly straight forward:
kubectl apply --filename https://storage.googleapis.com/tekton-releases-nightly/pipelines-in-pipelines/latest/release.yaml
.
With a pipeline like:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: go-trusted-service-container
spec:
description: |
Clones, tests, and builds Golang artiactes
params:
- name: repo-url
type: string
description: The git repo URL to clone from.
- name: docker-image
type: string
description: Docker image to store product as
- name: TARGETARCH
type: string
workspaces:
- name: scm
description: |
Workspace cloned from the scource code management system
- name: git-credentials
description: My ssh credentials
tasks:
- name: fetch-source
taskRef:
name: git-clone
workspaces:
- name: output
workspace: scm
- name: ssh-directory
workspace: git-credentials
params:
- name: url
value: $(params.repo-url)
- name: build
runAfter: ["fetch-source"]
taskRef:
name: golang-build
workspaces:
- name: source
workspace: scm
params:
- name: TARGETOS
value: linux
- name: TARGETARCH
value: $(params.TARGETARCH)
- name: package
value: "./cmd/service"
- name: docker-image
value: "$(params.docker-image)"
Running with both architectures as:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: go-trusted-services-containers-all
spec:
tasks:
- name: container-amd64
taskRef:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
name: go-trusted-service-container
podTemplateSpec:
nodeSelector:
"kubernetes.io/arch": "amd64"
params:
- name: TARGETARCH
value: amd64
- name: container-amd64
taskRef:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
name: go-trusted-service-container
podTemplateSpec:
nodeSelector:
"kubernetes.io/arch": "amd64"
params:
- name: TARGETARCH
value: arm64
Well that failed spectacularly! podTemplateSpec
fails validation which makes sense when I think through. taskRef
does not define additional fields.
Maybe buildah ?
While researching how to solve this issue looks like Buildah does support multi-architecture builds which would simplify this whole mess. Many of the toolmakers seem pretty annoyed at the requests for multi-arch oddly.
Turns out buildah requires a privileged
container. I would rather avoid that.
Maybe img ?
Well that looks promising. Let’s give this another whirl this time.
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: golang-build-img
labels:
app.kubernetes.io/version: "0.1"
annotations:
tekton.dev/pipelines.minVersion: "0.12.1"
tekton.dev/categories: Build Tools
tekton.dev/tags: build-tool
tekton.dev/displayName: "golang build"
tekton.dev/platforms: "linux/amd64,linux/arm64"
spec:
description: >-
Tests then builds a Golang service
params:
- name: package
description: base package to build in
- name: docker-image
description: Docker image URL in repository:tag format
- name: TARGETOS
description: target operating system. generally linux
- name: TARGETARCH
description: target architecture. generally AMD64 or ARM64
workspaces:
- name: source
steps:
- name: build
image: docker.workshop.meschbach.org/mee/platform/golang-builder:1.18.3
workingDir: $(workspaces.source.path)
script: |
go test ./pkg/...
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags='-w -s -extldflags "-static"' -o service $(params.package)
- name: package
image: r.j3ss.co/img
workingDir: $(workspaces.source.path)
args: ["build", "--tag","$(params.docker-image)", "--file $(params.package)/Dockerfile", "$(workspaces.source.path)"]
After some experimentation this resulted in a lot of extra priveleges.