1. 理解认证授权
1.1 为什么要认证
想理解认证，我们得从认证解决什么问题、防止什么问题的发生入手。
防止什么问题呢？是防止有人入侵你的集群，root你的机器后让我们集群依然安全吗？不是吧，root都到手了，那就为所欲为，防不胜防了。
其实网络安全本身就是为了解决在某些假设成立的条件下如何防范的问题。比如一个非常重要的假设就是两个节点或者ip之间的通讯网络是不可信任的，可能会被第三方窃取，也可能会被第三方篡改。就像我们上学时候给心仪的女孩传纸条，传送的过程可能会被别的同学偷看，甚至内容可能会从我喜欢你修改成我不喜欢你了。当然这种假设不是随便想出来的，而是从网络技术现状和实际发生的问题中发现、总结出来的。kubernetes的认证也是从这个问题出发来实现的。
1.2 概念梳理
为了解决上面说的问题，kubernetes并不需要自己想办法，毕竟是网络安全层面的问题，是每个服务都会遇到的问题，业内也有成熟的方案来解决。这里我们一起了解一下业内方案和相关的概念。
- 对称加密/非对称加密
这两个概念属于密码学的东西，对于没接触过的同学不太容易理解。可以参考知乎大神的生动讲解：《如何用通俗易懂的话来解释非对称加密》 - SSL/TLS
了解了对称加密和非对称加密后，我们就可以了解一下SSL/TLS了。同样，已经有大神总结了非常好的入门文章：《SSL/TLS协议运行机制的概述》
1.3 什么是授权
授权的概念就简单多了，就是什么人具有什么样的权限，一般通过角色作为纽带把他们组合在一起。也就是一个角色一边拥有多种权限，一边拥有多个人。这样就把人和权限建立了一个关系。
2. kubernetes的认证授权
Kubernetes集群的所有操作基本上都是通过kube-apiserver这个组件进行的，它提供HTTP RESTful形式的API供集群内外客户端调用。需要注意的是：认证授权过程只存在HTTPS形式的API中。也就是说，如果客户端使用HTTP连接到kube-apiserver，那么是不会进行认证授权的。所以说，可以这么设置，在集群内部组件间通信使用HTTP，集群外部就使用HTTPS，这样既增加了安全性，也不至于太复杂。
对APIServer的访问要经过的三个步骤，前面两个是认证和授权，第三个是 Admission Control，它也能在一定程度上提高安全性，不过更多是资源管理方面的作用。
2.1 kubernetes的认证
kubernetes提供了多种认证方式，比如客户端证书、静态token、静态密码文件、ServiceAccountTokens等等。你可以同时使用一种或多种认证方式。只要通过任何一个都被认作是认证通过。下面我们就认识几个常见的认证方式。
- 客户端证书认证
客户端证书认证叫作TLS双向认证，也就是服务器客户端互相验证证书的正确性，在都正确的情况下协调通信加密方案。
为了使用这个方案，api-server需要用–client-ca-file选项来开启。 - 引导Token
当我们有非常多的node节点时，手动为每个node节点配置TLS认证比较麻烦，这时就可以用到引导token的认证方式，前提是需要在api-server开启 experimental-bootstrap-token-auth 特性，客户端的token信息与预先定义的token匹配认证通过后，自动为node颁发证书。当然引导token是一种机制，可以用到各种场景中。 - Service Account Tokens 认证
有些情况下，我们希望在pod内部访问api-server，获取集群的信息，甚至对集群进行改动。针对这种情况，kubernetes提供了一种特殊的认证方式：Service Account。 Service Account 和 pod、service、deployment 一样是 kubernetes 集群中的一种资源，用户也可以创建自己的 Service Account。
ServiceAccount 主要包含了三个内容：namespace、Token 和 CA。namespace 指定了 pod 所在的 namespace，CA 用于验证 apiserver 的证书，token 用作身份验证。它们都通过 mount 的方式保存在 pod 的文件系统中。
2.2 kubernetes的授权
在Kubernetes1.6版本中新增角色访问控制机制（Role-Based Access，RBAC）让集群管理员可以针对特定使用者或服务账号的角色，进行更精确的资源访问控制。在RBAC中，权限与角色相关联，用户通过成为适当角色的成员而得到这些角色的权限。这就极大地简化了权限的管理。在一个组织中，角色是为了完成各种工作而创造，用户则依据它的责任和资格来被指派相应的角色，用户可以很容易地从一个角色被指派到另一个角色。
目前 Kubernetes 中有一系列的鉴权机制，因为Kubernetes社区的投入和偏好，相对于其它鉴权机制而言，RBAC是更好的选择。具体RBAC是如何体现在kubernetes系统中的我们会在后面的部署中逐步的深入了解。
2.3 kubernetes的AdmissionControl
AdmissionControl – 准入控制本质上为一段准入代码，在对kubernetes api的请求过程中，顺序为：先经过认证 & 授权，然后执行准入操作，最后对目标对象进行操作。这个准入代码在api-server中，而且必须被编译到二进制文件中才能被执行。
在对集群进行请求时，每个准入控制代码都按照一定顺序执行。如果有一个准入控制拒绝了此次请求，那么整个请求的结果将会立即返回，并提示用户相应的error信息。
常用组件（控制代码）如下：
- AlwaysAdmit：允许所有请求
- AlwaysDeny：禁止所有请求，多用于测试环境
- ServiceAccount：它将serviceAccounts实现了自动化，它会辅助serviceAccount做一些事情，比如如果pod没有serviceAccount属性，它会自动添加一个default，并确保pod的serviceAccount始终存在
- LimitRanger：他会观察所有的请求，确保没有违反已经定义好的约束条件，这些条件定义在namespace中LimitRange对象中。如果在kubernetes中使用LimitRange对象，则必须使用这个插件。
- NamespaceExists：它会观察所有的请求，如果请求尝试创建一个不存在的namespace，则这个请求被拒绝。
3. 环境准备
3.1 停止原有kubernetes相关服务
开始之前我们要先把基础版本的集群停掉，包括service，deployments，pods以及运行的所有kubernetes组件
<code class="language-bash"><span class="com">#删除services</span><span class="pln"> $ kubectl </span><span class="kwd">delete</span><span class="pln"> services nginx</span><span class="pun">-</span><span class="pln">service </span><span class="com">#删除deployments</span><span class="pln"> $ kubectl </span><span class="kwd">delete</span><span class="pln"> deploy kubernetes</span><span class="pun">-</span><span class="pln">bootcamp $ kubectl </span><span class="kwd">delete</span><span class="pln"> deploy nginx</span><span class="pun">-</span><span class="pln">deployment </span><span class="com">#停掉worker节点的服务</span><span class="pln"> $ service kubelet stop </span><span class="pun">&&</span><span class="pln"> rm </span><span class="pun">-</span><span class="pln">fr </span><span class="pun">/</span><span class="kwd">var</span><span class="pun">/</span><span class="pln">lib</span><span class="pun">/</span><span class="pln">kubelet</span><span class="com">/* $ service kube-proxy stop && rm -fr /var/lib/kube-proxy/* $ service kube-calico stop #停掉master节点的服务 $ service kube-calico stop $ service kube-scheduler stop $ service kube-controller-manager stop $ service kube-apiserver stop $ service etcd stop && rm -fr /var/lib/etcd/*</span></code>
www#gaodaima.com来源gao*daima.com搞@代#码网搞代码
3.2 生成配置（所有节点）
跟基础环境搭建一样，我们需要生成kubernetes-with-ca的所有相关配置文件
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter #按照配置文件的提示编辑好配置 $ vi config.properties #生成配置 $ ./</span><span class="pln">gen</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">sh </span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span></code>
3.3 安装cfssl（所有节点）
cfssl是非常好用的CA工具，我们用它来生成证书和秘钥文件
安装过程比较简单，如下：
<code class="language-bash"><span class="com">#下载</span><span class="pln"> $ wget </span><span class="pun">-</span><span class="pln">q </span><span class="pun">--</span><span class="pln">show</span><span class="pun">-</span><span class="pln">progress </span><span class="pun">--</span><span class="pln">https</span><span class="pun">-</span><span class="pln">only </span><span class="pun">--</span><span class="pln">timestamping https</span><span class="pun">:</span><span class="com">//pkg.cfssl.org/R1.2/cfssl_linux-amd64 </span><span class="pln"> https</span><span class="pun">:</span><span class="com">//pkg.cfssl.org/R1.2/cfssljson_linux-amd64</span> <span class="com">#修改为可执行权限</span><span class="pln"> $ chmod </span><span class="pun">+</span><span class="pln">x cfssl_linux</span><span class="pun">-</span><span class="pln">amd64 cfssljson_linux</span><span class="pun">-</span><span class="pln">amd64 </span><span class="com">#移动到bin目录</span><span class="pln"> $ mv cfssl_linux</span><span class="pun">-</span><span class="pln">amd64 </span><span class="pun">/</span><span class="pln">usr</span><span class="pun">/</span><span class="kwd">local</span><span class="pun">/</span><span class="pln">bin</span><span class="pun">/</span><span class="pln">cfssl $ mv cfssljson_linux</span><span class="pun">-</span><span class="pln">amd64 </span><span class="pun">/</span><span class="pln">usr</span><span class="pun">/</span><span class="kwd">local</span><span class="pun">/</span><span class="pln">bin</span><span class="pun">/</span><span class="pln">cfssljson </span><span class="com">#验证</span><span class="pln"> $ cfssl version</span></code>
3.4 生成根证书（主节点）
根证书是证书信任链的根，各个组件通讯的前提是有一份大家都信任的证书（根证书），每个人使用的证书都是由这个根证书签发的。
<code class="language-bash"><span class="com">#所有证书相关的东西都放在这</span><span class="pln"> $ mkdir </span><span class="pun">-</span><span class="pln">p </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca </span><span class="com">#准备生成证书的配置文件</span><span class="pln"> $ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">json </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca $ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca </span><span class="com">#生成证书和秘钥</span><span class="pln"> $ cd </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca $ cfssl gencert </span><span class="pun">-</span><span class="pln">initca ca</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">|</span><span class="pln"> cfssljson </span><span class="pun">-</span><span class="pln">bare ca </span><span class="com">#生成完成后会有以下文件（我们最终想要的就是ca-key.pem和ca.pem，一个秘钥，一个证书）</span><span class="pln"> $ ls ca</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">json ca</span><span class="pun">.</span><span class="pln">csr ca</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem ca</span><span class="pun">.</span><span class="pln">pem</span></code>
4. 改造etcd
4.1 准备证书
etcd节点需要提供给其他服务访问，就要验证其他服务的身份，所以需要一个标识自己监听服务的server证书，当有多个etcd节点的时候也需要client证书与etcd集群其他节点交互，当然也可以client和server使用同一个证书因为它们本质上没有区别。
<code class="language-bash"><span class="com">#etcd证书放在这</span><span class="pln">
$ mkdir </span><span class="pun">-</span><span class="pln">p </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">etcd
</span><span class="com">#准备etcd证书配置</span><span class="pln">
$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">/</span><span class="pln">
$ cd </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">/</span>
<span class="com">#使用根证书(ca.pem)签发etcd证书</span><span class="pln">
$ cfssl gencert
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">config</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">json
</span><span class="pun">-</span><span class="pln">profile</span><span class="pun">=</span><span class="pln">kubernetes etcd</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">|</span><span class="pln"> cfssljson </span><span class="pun">-</span><span class="pln">bare etcd
</span><span class="com">#跟之前类似生成三个文件etcd.csr是个中间证书请求文件，我们最终要的是etcd-key.pem和etcd.pem</span><span class="pln">
$ ls
etcd</span><span class="pun">.</span><span class="pln">csr etcd</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json etcd</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem etcd</span><span class="pun">.</span><span class="pln">pem</span></code>
4.2 改造etcd服务
建议大家先比较一下增加认证的etcd配置与原有配置的区别，做到心中有数。
可以使用命令比较：
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln"> $ vimdiff kubernetes</span><span class="pun">-</span><span class="pln">simple</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">.</span><span class="pln">service kubernetes</span><span class="pun">-</span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">.</span><span class="pln">service</span></code>
更新etcd服务：
<code class="language-bash"><span class="pln">$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">.</span><span class="pln">service </span><span class="pun">/</span><span class="pln">lib</span><span class="pun">/</span><span class="pln">systemd</span><span class="pun">/</span><span class="pln">system</span><span class="pun">/</span><span class="pln"> $ systemctl daemon</span><span class="pun">-</span><span class="pln">reload $ service etcd start </span><span class="com">#验证etcd服务（endpoints自行替换）</span><span class="pln"> $ ETCDCTL_API</span><span class="pun">=</span><span class="lit">3</span><span class="pln"> etcdctl </span><span class="pun">--</span><span class="pln">endpoints</span><span class="pun">=</span><span class="pln">https</span><span class="pun">:</span><span class="com">//192.168.1.102:2379 </span> <span class="pun">--</span><span class="pln">cacert</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem </span><span class="pun">--</span><span class="pln">cert</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">.</span><span class="pln">pem </span><span class="pun">--</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">/</span><span class="pln">etcd</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem endpoint health</span></code>
5. 改造api-server
5.1 准备证书
<code class="language-bash"><span class="com">#api-server证书放在这，api-server是核心，文件夹叫kubernetes吧，如果想叫apiserver也可以，不过相关的地方都需要修改哦</span><span class="pln">
$ mkdir </span><span class="pun">-</span><span class="pln">p </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kubernetes
</span><span class="com">#准备apiserver证书配置</span><span class="pln">
$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">
$ cd </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span>
<span class="com">#使用根证书(ca.pem)签发kubernetes证书</span><span class="pln">
$ cfssl gencert
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">config</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">json
</span><span class="pun">-</span><span class="pln">profile</span><span class="pun">=</span><span class="pln">kubernetes kubernetes</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">|</span><span class="pln"> cfssljson </span><span class="pun">-</span><span class="pln">bare kubernetes
</span><span class="com">#跟之前类似生成三个文件kubernetes.csr是个中间证书请求文件，我们最终要的是kubernetes-key.pem和kubernetes.pem</span><span class="pln">
$ ls
kubernetes</span><span class="pun">.</span><span class="pln">csr kubernetes</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json kubernetes</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem kubernetes</span><span class="pun">.</span><span class="pln">pem</span></code>
5.2 改造api-server服务
查看diff
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter $ vimdiff kubernetes-simple/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">apiserver</span><span class="pun">.</span><span class="pln">service kubernetes</span><span class="pun">-</span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">apiserver</span><span class="pun">.</span><span class="pln">service</span></code>
生成token认证文件
<code class="language-bash"><span class="com">#生成随机token</span><span class="pln"> $ head </span><span class="pun">-</span><span class="pln">c </span><span class="lit">16</span> <span class="pun">/</span><span class="pln">dev</span><span class="pun">/</span><span class="pln">urandom </span><span class="pun">|</span><span class="pln"> od </span><span class="pun">-</span><span class="typ">An</span> <span class="pun">-</span><span class="pln">t x </span><span class="pun">|</span><span class="pln"> tr </span><span class="pun">-</span><span class="pln">d </span><span class="str">" "</span> <span class="lit">8afdf3c4eb7c74018452423c29433609</span> <span class="com">#按照固定格式写入token.csv，注意替换token内容</span><span class="pln"> $ echo </span><span class="str">"8afdf3c4eb7c74018452423c29433609,kubelet-bootstrap,10001,"system:kubelet-bootstrap""</span> <span class="pun">></span> <span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">token</span><span class="pun">.</span><span class="pln">csv</span></code>
更新api-server服务
<code class="language-bash"><span class="pln">$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">apiserver</span><span class="pun">.</span><span class="pln">service </span><span class="pun">/</span><span class="pln">lib</span><span class="pun">/</span><span class="pln">systemd</span><span class="pun">/</span><span class="pln">system</span><span class="pun">/</span><span class="pln"> $ systemctl daemon</span><span class="pun">-</span><span class="pln">reload $ service kube</span><span class="pun">-</span><span class="pln">apiserver start </span><span class="com">#检查日志</span><span class="pln"> $ journalctl </span><span class="pun">-</span><span class="pln">f </span><span class="pun">-</span><span class="pln">u kube</span><span class="pun">-</span><span class="pln">apiserver</span></code>
6. 改造controller-manager
controller-manager一般与api-server在同一台机器上，所以可以使用非安全端口与api-server通讯，不需要生成证书和私钥。
6.1 改造controller-manager服务
查看diff
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln"> $ vimdiff kubernetes</span><span class="pun">-</span><span class="pln">simple</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">controller</span><span class="pun">-</span><span class="pln">manager</span><span class="pun">.</span><span class="pln">service kubernetes</span><span class="pun">-</span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">controller</span><span class="pun">-</span><span class="pln">manager</span><span class="pun">.</span><span class="pln">service</span></code>
更新controller-manager服务
<code class="language-bash"><span class="pln">$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">controller</span><span class="pun">-</span><span class="pln">manager</span><span class="pun">.</span><span class="pln">service </span><span class="pun">/</span><span class="pln">lib</span><span class="pun">/</span><span class="pln">systemd</span><span class="pun">/</span><span class="pln">system</span><span class="pun">/</span><span class="pln"> $ systemctl daemon</span><span class="pun">-</span><span class="pln">reload $ service kube</span><span class="pun">-</span><span class="pln">controller</span><span class="pun">-</span><span class="pln">manager start </span><span class="com">#检查日志</span><span class="pln"> $ journalctl </span><span class="pun">-</span><span class="pln">f </span><span class="pun">-</span><span class="pln">u kube</span><span class="pun">-</span><span class="pln">controller</span><span class="pun">-</span><span class="pln">manager</span></code>
7. 改造scheduler
scheduler一般与apiserver在同一台机器上，所以可以使用非安全端口与apiserver通讯。不需要生成证书和私钥。
7.1 改造scheduler服务
查看diff
比较会发现两个文件并没有区别，不需要改造
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln"> $ vimdiff kubernetes</span><span class="pun">-</span><span class="pln">simple</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">scheduler</span><span class="pun">.</span><span class="pln">service kubernetes</span><span class="pun">-</span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">master</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">scheduler</span><span class="pun">.</span><span class="pln">service</span></code>
启动服务
<code class="language-bash"><span class="pln">$ service kube</span><span class="pun">-</span><span class="pln">scheduler start </span><span class="com">#检查日志</span><span class="pln"> $ journalctl </span><span class="pun">-</span><span class="pln">f </span><span class="pun">-</span><span class="pln">u kube</span><span class="pun">-</span><span class="pln">scheduler</span></code>
8. 改造kubectl
8.1 准备证书
<code class="language-bash"><span class="com">#kubectl证书放在这，由于kubectl相当于系统管理员，我们使用admin命名</span><span class="pln">
$ mkdir </span><span class="pun">-</span><span class="pln">p </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">admin
</span><span class="com">#准备admin证书配置 - kubectl只需客户端证书，因此证书请求中 hosts 字段可以为空</span><span class="pln">
$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">/</span><span class="pln">
$ cd </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">/</span>
<span class="com">#使用根证书(ca.pem)签发admin证书</span><span class="pln">
$ cfssl gencert
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">config</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">json
</span><span class="pun">-</span><span class="pln">profile</span><span class="pun">=</span><span class="pln">kubernetes admin</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">|</span><span class="pln"> cfssljson </span><span class="pun">-</span><span class="pln">bare admin
</span><span class="com">#我们最终要的是admin-key.pem和admin.pem</span><span class="pln">
$ ls
admin</span><span class="pun">.</span><span class="pln">csr admin</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json admin</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem admin</span><span class="pun">.</span><span class="pln">pem</span></code>
8.2 配置kubectl
<code class="language-bash"><span class="com">#指定apiserver的地址和证书位置（ip自行修改）</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">cluster kubernetes
</span><span class="pun">--</span><span class="pln">certificate</span><span class="pun">-</span><span class="pln">authority</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">--</span><span class="pln">embed</span><span class="pun">-</span><span class="pln">certs</span><span class="pun">=</span><span class="kwd">true</span><span class="pln">
</span><span class="pun">--</span><span class="pln">server</span><span class="pun">=</span><span class="pln">https</span><span class="pun">:</span><span class="com">//192.168.1.102:6443</span>
<span class="com">#设置客户端认证参数，指定admin证书和秘钥</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">credentials admin
</span><span class="pun">--</span><span class="pln">client</span><span class="pun">-</span><span class="pln">certificate</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">--</span><span class="pln">embed</span><span class="pun">-</span><span class="pln">certs</span><span class="pun">=</span><span class="kwd">true</span><span class="pln">
</span><span class="pun">--</span><span class="pln">client</span><span class="pun">-</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">/</span><span class="pln">admin</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem
</span><span class="com">#关联用户和集群</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">context kubernetes
</span><span class="pun">--</span><span class="pln">cluster</span><span class="pun">=</span><span class="pln">kubernetes </span><span class="pun">--</span><span class="pln">user</span><span class="pun">=</span><span class="pln">admin
</span><span class="com">#设置当前上下文</span><span class="pln">
$ kubectl config </span><span class="kwd">use</span><span class="pun">-</span><span class="pln">context kubernetes
</span><span class="com">#设置结果就是一个配置文件，可以看看内容</span><span class="pln">
$ cat </span><span class="pun">~</span><span class="str">/.kube/</span><span class="pln">config</span></code>
验证master节点
<code class="language-bash"><span class="com">#可以使用刚配置好的kubectl查看一下组件状态</span><span class="pln">
$ kubectl </span><span class="kwd">get</span><span class="pln"> componentstatus
NAME STATUS MESSAGE ERROR
scheduler </span><span class="typ">Healthy</span><span class="pln"> ok
controller</span><span class="pun">-</span><span class="pln">manager </span><span class="typ">Healthy</span><span class="pln"> ok
etcd</span><span class="pun">-</span><span class="lit">0</span> <span class="typ">Healthy</span> <span class="pun">{</span><span class="str">"health"</span><span class="pun">:</span> <span class="str">"true"</span><span class="pun">}</span></code>
9. 改造calico-node
9.1 准备证书
后续可以看到calico证书用在四个地方：
- calico/node 这个docker 容器运行时访问 etcd 使用证书
- cni 配置文件中，cni 插件需要访问 etcd 使用证书
- calicoctl 操作集群网络时访问 etcd 使用证书
- calico/kube-controllers 同步集群网络策略时访问 etcd 使用证书
<code class="language-bash"><span class="com">#calico证书放在这</span><span class="pln"> $ mkdir </span><span class="pun">-</span><span class="pln">p </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">calico </span><span class="com">#准备calico证书配置 - calico只需客户端证书，因此证书请求中 hosts 字段可以为空</span><span class="pln"> $ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">calico</span><span class="pun">/</span><span class="pln">calico</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">calico</span><span class="pun">/</span><span class="pln"> $ cd </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">calico</span><span class="pun">/</span> <span class="com">#使用根证书(ca.pem)签发calico证书</span><span class="pln"> $ cfssl gencert </span><span class="pun">-</span><span class="pln">ca</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem </span><span class="pun">-</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem </span><span class="pun">-</span><span class="pln">config</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">json </span><span class="pun">-</span><span class="pln">profile</span><span class="pun">=</span><span class="pln">kubernetes calico</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">|</span><span class="pln"> cfssljson </span><span class="pun">-</span><span class="pln">bare calico </span><span class="com">#我们最终要的是calico-key.pem和calico.pem</span><span class="pln"> $ ls calico</span><span class="pun">.</span><span class="pln">csr calico</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json calico</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem calico</span><span class="pun">.</span><span class="pln">pem</span></code>
9.2 改造calico服务
查看diff
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter $ vimdiff kubernetes-simple/</span><span class="pln">all</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">calico</span><span class="pun">.</span><span class="pln">service kubernetes</span><span class="pun">-</span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">all</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">calico</span><span class="pun">.</span><span class="pln">service</span></code>
通过diff会发现，calico多了几个认证相关的文件：
/etc/kubernetes/ca/ca.pem
/etc/kubernetes/ca/calico/calico.pem
/etc/kubernetes/ca/calico/calico-key.pem
由于calico服务是所有节点都需要启动的，大家需要把这几个文件拷贝到每台服务器上
更新calico服务
<code class="language-bash"><span class="pln">$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">all</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">calico</span><span class="pun">.</span><span class="pln">service </span><span class="pun">/</span><span class="pln">lib</span><span class="pun">/</span><span class="pln">systemd</span><span class="pun">/</span><span class="pln">system</span><span class="pun">/</span><span class="pln"> $ systemctl daemon</span><span class="pun">-</span><span class="pln">reload $ service kube</span><span class="pun">-</span><span class="pln">calico start </span><span class="com">#验证calico（能看到其他节点的列表就对啦）</span><span class="pln"> $ calicoctl node status</span></code>
10. 改造kubelet
我们这里让kubelet使用引导token的方式认证，所以认证方式跟之前的组件不同，它的证书不是手动生成，而是由工作节点TLS BootStrap 向api-server请求，由主节点的controller-manager 自动签发。
10.1 创建角色绑定（主节点）
引导token的方式要求客户端向api-server发起请求时告诉他你的用户名和token，并且这个用户是具有一个特定的角色：system:node-bootstrapper，所以需要先将 bootstrap token 文件中的 kubelet-bootstrap 用户赋予这个特定角色，然后 kubelet 才有权限发起创建认证请求。
在主节点执行下面命令
<code class="language-bash"><span class="com">#可以通过下面命令查询clusterrole列表</span><span class="pln">
$ kubectl </span><span class="pun">-</span><span class="pln">n kube</span><span class="pun">-</span><span class="pln">system </span><span class="kwd">get</span><span class="pln"> clusterrole
</span><span class="com">#可以回顾一下token文件的内容</span><span class="pln">
$ cat </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">token</span><span class="pun">.</span><span class="pln">csv
</span><span class="lit">8afdf3c4eb7c74018452423c29433609</span><span class="pun">,</span><span class="pln">kubelet</span><span class="pun">-</span><span class="pln">bootstrap</span><span class="pun">,</span><span class="lit">10001</span><span class="pun">,</span><span class="str">"system:kubelet-bootstrap"</span>
<span class="com">#创建角色绑定（将用户kubelet-bootstrap与角色system:node-bootstrapper绑定）</span><span class="pln">
$ kubectl create clusterrolebinding kubelet</span><span class="pun">-</span><span class="pln">bootstrap
</span><span class="pun">--</span><span class="pln">clusterrole</span><span class="pun">=</span><span class="pln">system</span><span class="pun">:</span><span class="pln">node</span><span class="pun">-</span><span class="pln">bootstrapper </span><span class="pun">--</span><span class="pln">user</span><span class="pun">=</span><span class="pln">kubelet</span><span class="pun">-</span><span class="pln">bootstrap</span></code>
10.2 创建bootstrap.kubeconfig（工作节点）
这个配置是用来完成bootstrap token认证的，保存了像用户，token等重要的认证信息，这个文件可以借助kubectl命令生成：（也可以自己写配置）
<code class="language-bash"><span class="com">#设置集群参数(注意替换ip)</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">cluster kubernetes
</span><span class="pun">--</span><span class="pln">certificate</span><span class="pun">-</span><span class="pln">authority</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">--</span><span class="pln">embed</span><span class="pun">-</span><span class="pln">certs</span><span class="pun">=</span><span class="kwd">true</span><span class="pln">
</span><span class="pun">--</span><span class="pln">server</span><span class="pun">=</span><span class="pln">https</span><span class="pun">:</span><span class="com">//192.168.1.102:6443 </span>
<span class="pun">--</span><span class="pln">kubeconfig</span><span class="pun">=</span><span class="pln">bootstrap</span><span class="pun">.</span><span class="pln">kubeconfig
</span><span class="com">#设置客户端认证参数(注意替换token)</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">credentials kubelet</span><span class="pun">-</span><span class="pln">bootstrap
</span><span class="pun">--</span><span class="pln">token</span><span class="pun">=</span><span class="lit">8afdf3c4eb7c74018452423c29433609</span><span class="pln">
</span><span class="pun">--</span><span class="pln">kubeconfig</span><span class="pun">=</span><span class="pln">bootstrap</span><span class="pun">.</span><span class="pln">kubeconfig
</span><span class="com">#设置上下文</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">context </span><span class="kwd">default</span><span class="pln">
</span><span class="pun">--</span><span class="pln">cluster</span><span class="pun">=</span><span class="pln">kubernetes
</span><span class="pun">--</span><span class="pln">user</span><span class="pun">=</span><span class="pln">kubelet</span><span class="pun">-</span><span class="pln">bootstrap
</span><span class="pun">--</span><span class="pln">kubeconfig</span><span class="pun">=</span><span class="pln">bootstrap</span><span class="pun">.</span><span class="pln">kubeconfig
</span><span class="com">#选择上下文</span><span class="pln">
$ kubectl config </span><span class="kwd">use</span><span class="pun">-</span><span class="pln">context </span><span class="kwd">default</span> <span class="pun">--</span><span class="pln">kubeconfig</span><span class="pun">=</span><span class="pln">bootstrap</span><span class="pun">.</span><span class="pln">kubeconfig
</span><span class="com">#将刚生成的文件移动到合适的位置</span><span class="pln">
$ mv bootstrap</span><span class="pun">.</span><span class="pln">kubeconfig </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span></code>
10.3 准备cni配置
查看diff
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter $ vimdiff kubernetes-simple/</span><span class="pln">worker</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="lit">10</span><span class="pun">-</span><span class="pln">calico</span><span class="pun">.</span><span class="pln">conf kubernetes</span><span class="pun">-</span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">worker</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="lit">10</span><span class="pun">-</span><span class="pln">calico</span><span class="pun">.</span><span class="pln">conf</span></code>
copy配置
<code class="language-bash"><span class="pln">$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">worker</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="lit">10</span><span class="pun">-</span><span class="pln">calico</span><span class="pun">.</span><span class="pln">conf </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">cni</span><span class="pun">/</span><span class="pln">net</span><span class="pun">.</span><span class="pln">d</span><span class="pun">/</span></code>
10.4 改造kubelet服务
查看diff
<code class="language-bash"><span class="pln">$ cd </span><span class="pun">~</span><span class="str">/kubernetes-starter $ vimdiff kubernetes-simple/</span><span class="pln">worker</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kubelet</span><span class="pun">.</span><span class="pln">service kubernetes</span><span class="pun">-</span><span class="kwd">with</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">worker</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kubelet</span><span class="pun">.</span><span class="pln">service</span></code>
更新服务
<code class="language-bash"><span class="pln">$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">worker</span><span class="pun">-</span><span class="pln">node</span><span class="pun">/</span><span class="pln">kubelet</span><span class="pun">.</span><span class="pln">service </span><span class="pun">/</span><span class="pln">lib</span><span class="pun">/</span><span class="pln">systemd</span><span class="pun">/</span><span class="pln">system</span><span class="pun">/</span><span class="pln">
$ systemctl daemon</span><span class="pun">-</span><span class="pln">reload
$ service kubelet start
</span><span class="com">#启动kubelet之后到master节点允许worker加入(批准worker的tls证书请求)</span>
<span class="com">#--------*在主节点执行*---------</span><span class="pln">
$ kubectl </span><span class="kwd">get</span><span class="pln"> csr</span><span class="pun">|</span><span class="pln">grep </span><span class="str">"Pending"</span> <span class="pun">|</span><span class="pln"> awk </span><span class="str">"{print $1}"</span><span class="pun">|</span><span class="pln"> xargs kubectl certificate approve
</span><span class="com">#-----------------------------</span>
<span class="com">#检查日志</span><span class="pln">
$ journalctl </span><span class="pun">-</span><span class="pln">f </span><span class="pun">-</span><span class="pln">u kubelet</span></code>
11. 改造kube-proxy
11.1 准备证书
<code class="language-bash"><span class="com">#proxy证书放在这</span><span class="pln">
$ mkdir </span><span class="pun">-</span><span class="pln">p </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy
</span><span class="com">#准备proxy证书配置 - proxy只需客户端证书，因此证书请求中 hosts 字段可以为空。</span>
<span class="com">#CN 指定该证书的 User 为 system:kube-proxy，预定义的 ClusterRoleBinding system:node-proxy 将User system:kube-proxy 与 Role system:node-proxier 绑定，授予了调用 kube-api-server proxy的相关 API 的权限</span><span class="pln">
$ cp </span><span class="pun">~</span><span class="str">/kubernetes-starter/</span><span class="pln">target</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">/</span><span class="pln">
$ cd </span><span class="pun">/</span><span class="pln">etc</span><span class="pun">/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">/</span>
<span class="com">#使用根证书(ca.pem)签发calico证书</span><span class="pln">
$ cfssl gencert
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">-</span><span class="pln">config</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">-</span><span class="pln">config</span><span class="pun">.</span><span class="pln">json
</span><span class="pun">-</span><span class="pln">profile</span><span class="pun">=</span><span class="pln">kubernetes kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json </span><span class="pun">|</span><span class="pln"> cfssljson </span><span class="pun">-</span><span class="pln">bare kube</span><span class="pun">-</span><span class="pln">proxy
</span><span class="com">#我们最终要的是kube-proxy-key.pem和kube-proxy.pem</span><span class="pln">
$ ls
kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">.</span><span class="pln">csr kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">-</span><span class="pln">csr</span><span class="pun">.</span><span class="pln">json kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">.</span><span class="pln">pem</span></code>
11.2 生成kube-proxy.kubeconfig配置
<code class="language-bash"><span class="com">#设置集群参数（注意替换ip）</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">cluster kubernetes
</span><span class="pun">--</span><span class="pln">certificate</span><span class="pun">-</span><span class="pln">authority</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">--</span><span class="pln">embed</span><span class="pun">-</span><span class="pln">certs</span><span class="pun">=</span><span class="kwd">true</span><span class="pln">
</span><span class="pun">--</span><span class="pln">server</span><span class="pun">=</span><span class="pln">https</span><span class="pun">:</span><span class="com">//192.168.1.102:6443 </span>
<span class="pun">--</span><span class="pln">kubeconfig</span><span class="pun">=</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">.</span><span class="pln">kubeconfig
</span><span class="com">#置客户端认证参数</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">credentials kube</span><span class="pun">-</span><span class="pln">proxy
</span><span class="pun">--</span><span class="pln">client</span><span class="pun">-</span><span class="pln">certificate</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">--</span><span class="pln">client</span><span class="pun">-</span><span class="pln">key</span><span class="pun">=</span><span class="str">/etc/</span><span class="pln">kubernetes</span><span class="pun">/</span><span class="pln">ca</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">/</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">-</span><span class="pln">key</span><span class="pun">.</span><span class="pln">pem
</span><span class="pun">--</span><span class="pln">embed</span><span class="pun">-</span><span class="pln">certs</span><span class="pun">=</span><span class="kwd">true</span><span class="pln">
</span><span class="pun">--</span><span class="pln">kubeconfig</span><span class="pun">=</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">.</span><span class="pln">kubeconfig
</span><span class="com">#设置上下文参数</span><span class="pln">
$ kubectl config </span><span class="kwd">set</span><span class="pun">-</span><span class="pln">context </span><span class="kwd">default</span><span class="pln">
</span><span class="pun">--</span><span class="pln">cluster</span><span class="pun">=</span><span class="pln">kubernetes
</span><span class="pun">--</span><span class="pln">user</span><span class="pun">=</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy
</span><span class="pun">--</span><span class="pln">kubeconfig</span><span class="pun">=</span><span class="pln">kube</span><span class="pun">-</span><span class="pln">proxy</span><span class="pun">.</span><span class="pln">kubeconfig
</span><span class="com">#选择上下文</span><span class="pln">
$ kubectl config </span><span class="kwd">use</span><span class="pun">-</span><span class="
</div>
<!-- .entry-wrapper -->
<footer>
<hr>
<ul class="share-buttons ptb-20">
<li> <p>若本号内容有做得不到位的地方（比如：涉及版权或其他问题），请及时联系我们进行整改即可，会在第一时间进行处理。</p>
</li>
</ul>
</span></code>
搞代码网（gaodaima.com）提供的所有资源部分来自互联网，如果有侵犯您的版权或其他权益，请说明详细缘由并提供版权或权益证明然后发送到邮箱[email protected]，我们会在看到邮件的第一时间内为您处理，或直接联系QQ：872152909。本网站采用BY-NC-SA协议进行授权
转载请注明原文链接：纯手工搭建k8s集群-(三)认证授权和服务发现
转载请注明原文链接：纯手工搭建k8s集群-(三)认证授权和服务发现
