Kubernetes Access Control

As shown above, this is a typical Kubernetes cluster component diagram, through which we can see that all Kubernetes components use APIServer as a gateway to communicate. For security purposes, the APIServer is generally exposed to the public through TLS authentication, and cluster components need the corresponding TLS certificate to access the APIServer.

The following figure shows the access control process of the APIserver. The complete access control requires three modules: authentication, authorization and access control. Figure 4 shows the APIServer accessing the ETCD cluster, which also uses TLS authentication.

It is important to note that APIServer itself supports multiple authentication methods, not just TLS, and by default we use TLS authentication (multiple authentication methods can be enabled). Currently APIServer supports the following authentication methods.

Here we will focus on the TLS authentication method, other authentication methods can be found in Authenticating | Kubernetes.

1. TLS Two-Way Authentication

TLS is a secure transport layer protocol, including two parts: TLS record protocol and TLS handshake protocol. the TLS record protocol mainly ensures the integrity and privacy of information transmission during transmission, and this part encrypts data by negotiated key. the TLS handshake protocol is mainly for authenticating each other’s identity and negotiating key.

APIServer and cluster components communicate using TLS two-way authentication, as the name implies, both the client and the server need to verify each other’s identity, compared with one-way authentication, two-way authentication client needs to download the server’s public key certificate from the server side for verification, but also needs to upload the client’s public key certificate to the server side for verification, and only when both sides have passed the authentication Start to establish a secure communication channel for data transmission, the specific process is as follows.

kube-apiserver involves a lot of certificate options, the following are the certificate options and descriptions after sorting.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

--etcd-cafile                        # etcd CA 证书
--etcd-certfile                      # APIServer 访问 etcd client 公钥
--etcd-keyfile                       # APIServer 访问 etcd client 密钥
--client-ca-file                     # 验证访问 APIServer 的 client CA 证书
--tls-cert-file                      # APIServer 服务的公钥
--tls-private-key-file               # APIServer 服务的密钥
--service-account-key-file           # 验证 ServiceAccount tokens 的公钥
--service-account-signing-key-file   # ServiceAccount tokens 签名密钥
--kubelet-client-certificate         # 访问 kubelet 的公钥
--kubelet-client-key                 # 访问 kubelet 的密钥
--requestheader-client-ca-file       # 用于签名 --proxy-client-cert-file 和--proxy-client-key-file 指定的证书，启用 aggregator 时使用
--proxy-client-cert-file             # 用于请求 aggregator client 公钥
--proxy-client-key-file              # 用于请求 aggregator client 密钥

It is worth noting that APIServer uses the CN and O fields in the certificate as user name and group name identifiers during TLS authentication, and uses these two fields in combination with ClusterRole/ClusterRoleBinding/Role/RoleBinding to achieve the purpose of associated authorization, while it does not have its own user and group management mechanism.

2. RBAC

After the above introduction of TLS authentication, how do we manage resource authorization based on authentication? RBAC, which literally means role-based access control, in Kubernetes mainly involves the above mentioned ClusterRole/ClusterRoleBinding/Role/RoleBinding resources, where ClusterRole/ClusterRoleBinding is a global role and Role/RoleBind is for namespace. A Role is an abstraction of a user’s permissions, and a RoleBinding binds a role to a User, Group, or Service Account. The following diagram illustrates the correspondence more graphically.

Let’s look at the specific resource configuration, here is an example of CoreDNS.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: coredns   # 如果你要绑定权限，这里的 name 需要对应 TLS 证书中的 CN 字段
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  - services
  - pods
  - namespaces
  verbs:
  - list
  - watch

ClusterRole defines the scope of CoreDNS apiGroups, the resources that can be accessed, and the permissions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: coredns
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: coredns
subjects:
- kind: ServiceAccount
  name: default
  namespace: kube-system

The subjects in ClusterRoleBinding define the ServiceAccount and the corresponding space (subjects can also be groups or Service Account, where groups correspond to the O field in the TLS certificate). As mentioned above, CN and O in the certificate are used as user and group identification fields, and ClusterRoleBinding/RoleBinding relates ClusterRole/Role and subjects: serviceaccount: prefix, as in the CoreDNS example, and for TLS access, you can configure the CN as system:serviceaccount:coredns, see Using RBAC Authorization for details.

3. TLS bootstrapping

As mentioned earlier, the reason for using TLS authentication is for inter-cluster communication security purposes. Kubernetes has introduced a certificate request and signing API since 1.4 to simplify this process, which is what we are talking about here: TLS bootstrapping.

The following diagram illustrates the workflow of TLS bootstrapping.

The flow in the diagram above is simply that the kubelet first looks for the kubeconfig file when it starts, and when it exists, it uses kubeconfig to join the cluster directly at startup. If kubeconfig does not exist, the kubelet uses the bootstrap.kubeconfig file to create an authentication request, and then automatically creates a kubeconfig file and joins the cluster by issuing a certificate through the bootstrapping mechanism. The detailed process is described in TLS bootstrapping | Kubernetes , where the generated certificate will be checked for validity and will be re-enrolled when it is about to expire.

To enable TLS bootstrapping for Kubelet, you need to turn on the option --bootstrap-kubeconfig, and we can see a sample configuration in bootstrap.kubeconfig

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/pki/ca.pem      # CA 证书地址
    server: https://<APIServer>:<APIServerPort>            # APIServer 地址
  name: bootstrap
contexts:
- context:
    cluster: bootstrap
    user: kubelet-bootstrap
  name: bootstrap
current-context: bootstrap
preferences: {}
users:
- name: kubelet-bootstrap
  user:
token: 50bc4305185c3c2d8e31cab9223a8107 # APIServer 定义的 bootstrap token

4. Node Authorization

While we know that TLS bootstrapping is mainly for kubelet certificate auto-authentication, Node Authorization is a special authorization for kubelet API request authorization. Node Authorization allows Kubelet to have the following API operation permissions.

• read operations
  • services
  • endpoints
  • nodes
  • pods
  • secrets, configmaps, pvc and the pv bound to the node
• write operations
  • nodes and node state (restrict kubelet from modifying its own nodes via the NodeRestriction access control plugin)
  • pods and pods state (restricts kubelet from modifying its own scheduled pods via the NodeRestriction access control plugin)
  • events
• authentication operations
  • TLS bootstrapping read-write access certificationsigningrequests API
  • ability to check and create tokenreviews and subjectaccessreviews for authentication authorization

In future releases, node authorization will support adding or removing permissions to ensure that the kubelet has the minimum set of permissions needed to operate correctly.

APIServer can enable Node authorization with --authorization-mode=Node, normally Node is enabled along with RBAC, e.g. --authorization-mode=Node,RBAC. APIServer also enables NodeRestriction to restrict kubelet API write operations, --enable-admission-plugins=... , NodeRestriction, ....