Vault on GKE, part 3
• Mark Eschbach
Picking up from yesterday the Vault instance wasn’t unsealed. The pod
time="2019-02-05T05:11:06Z" level=info msg="checking if vault is sealed..." time="2019-02-05T05:11:06Z" level=info msg="vault sealed: true" time="2019-02-05T05:11:06Z" level=error msg="error unsealing vault: unable to get key 'vault-unseal-0': error getting secret for key 'vault-unseal-0': secrets \"vault-unseal-keys\" is forbidden: User \"system:serviceaccount:default:default\" cannot get secrets in the namespace \"default\""
Hmm, I think I misunderstood something in the resource specification. Looks like I need to create the role specified in the resource definition. Makes total sense but I was going for simplicity of implementation. Now to make it happen!
Based on the error message I believe the container is running under the credential
kubectl describe pod vault-0 did not contain any references to the
kubectl get rolebinding --namespace=kube-system :-(.
Demystifying RBAC in Kubernetes defines two
types of roles: Users and Service Accounts. Service Accounts are intra-cluster resources while users are located
outside of the pod. There was a lot of great details on the principal (subject), action (verbs), and target (api
ClusterRole is cluster wide while the
Role is scoped to a namespace. A
ClusterRoleBinding is utilized to give a subject, either a group or specific role, a binding.
Okay! After a whole bunch of reading I think I finally figured it out.
system:serviceaccount is stating the named
resource is a service account. The next section
default is referring to the
default namespace. While the last
element in the
: path is the named service account under the
default namespace. According to this hypothesis the
results I should be able to to list the account with
kubectl get serviceaccounts --namespace=default.
kubectl describe serviceaccounts default provides the following:
Name: default Namespace: default Labels: <none> Annotations: <none> Image pull secrets: <none> Mountable secrets: default-token-svm5r Tokens: default-token-svm5r Events: <none>
There are many interesting fields here however the
Mountable secrets &
Tokens look most interesting. Looks like the
OpenShift project confirms my hypothesis of the URI,
so that is always exciting.
Turns out you can edit service accounts using something like
kubectl describe serviceaccounts default. This will
allow one to add the
vault-unseal-keys to the list of mountable secrets. This however was insufficient as I think
this only means it will expose an existing one and can be confirmed as missing using
kubectl describe serviceaccounts default
shows it is missing.
Tune in tomorrow for either get Vault established or shaving more yaks :-) !