Today I pick up where I left off. Vault Operator is bound to the default role in the default namespace and is unable to read or write to the secret vault-unseal-keys. My quest, as I have accepted, is to allow this.

From my explorations yesterday I believe I need to add the following snippet the role.

rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

One of the largest issues I have run into is default is a ServiceAccount and not a role. At least for now these appear to magical roles which do not show in either kubectl get rolebindings --namespace=default, kubectl get clustrolebindings or kubectl get rolebindings --namespace=kube-system.

From here I think the best path forward is to stop using the magical roles and define my own to run the Vault Operator works under.

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: vault-unsealer
rules:
- apiGroups: [""]
  resources: ["secrets/vault-unseal-key"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

A digression into privilege escalation

Applying this resulted in an interesting result, listed as follows. The error message both fortunately and unfortunately carries a lot fo information, including the user trying to execute the role. Based on the error message I would like to verify which permissions I do have.

Error from server (Forbidden): error when creating "role-vault-unsealer.yml": roles.rbac.authorization.k8s.io "vault-unsealer" is forbidden: attempt to grant extra privileges: [{[get] [] [secrets/vault-unseal-key] [] []} {[list] [] [secrets/vault-unseal-key] [] []} {[watch] [] [secrets/vault-unseal-key] [] []} {[create] [] [secrets/vault-unseal-key] [] []} {[update] [] [secrets/vault-unseal-key] [] []} {[patch] [] [secrets/vault-unseal-key] [] []} {[delete] [] [secrets/vault-unseal-key] [] []}] user=&{user@example.com  [system:authenticated] map[user-assertion.cloud.google.com:[redacted]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]

First is just the check to create a new role within a given namespace. Interesting enough I continue to get errors regarding users additional privilege grants. Turns out I need to grant myself a ClusterRole, despite being able to create service accounts in kube-system :-/. Once I ran kubectl create clusterrolebinding myname-cluster-admin-binding --clusterrole=cluster-admin --user=myname@example.org I could create the role wherever I would like. This of course was documented in the Banzi Cloud Vault Operator instructions and I failed to create it!

Return to your normally scheduled permissions

To properly spin this up creating a service account is the easiest mechanism to get the unsealer up and running.

kind: ServiceAccount 
apiVersion: v1
metadata:
  namespace: default 
  name: vault-unsealer
secrets:
- name: vault-unseal-keys
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default 
  name: vault-unsealer
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: vault-unsealer 
  namespace: default
subjects:
- kind: ServiceAccount 
  name: vault-unsealer 
roleRef:
  kind: Role
  name: vault-unsealer
  apiGroup: rbac.authorization.k8s.io

This boot straps the ServiceAccount vault-unsealer which is allowed to openly modify the secrets store for the namespace it was residing in. Additionally on the resource definition the spec.serviceAccount needs to be set vault-unsealer.

To wrap it up, after a few seconds the instance should be up and running. To forward the pod to your machine you should use kubectl port-forward pods/vault-0 8200:8200. From here we need the root token. This was placed in the vault-unseal-keys secret store as data.vault-root. The value can be accessed via kubectl get secret vault-unseal-keys -o jsonpath={.data.vault-root} |base64 -D|xargs echo will give you the token. Using VAULT_SKIP_VERIFY=true vault login with the token at the prompt will result in access to the datastore.

Tune in tomorrow an exploration on how to break and secure the system!