Kubernetes共享使用Ceph存储

目录

• 简要概述
• 环境测试
• 结果验证

简要概述

Kubernetes pod 结合Ceph rbd块设备的使用，让Docker 数据存储在Ceph,重启Docker或k8s RC重新调度pod 不会引起数据来回迁移。

工作原理无非就是拿到ceph集群的key作为认证，远程rbdmap映射挂载使用。那么就要启用一个基于key的secret资源，之后k8s pod指定要用的rbd就可以了。

环境测试

我的OS是ubuntu 14.04，一个master,两个minon 节点，一个镜像仓库服务，这样保证了k8s集群跑起来，这个过程就略过了。

1.内核升级

这里的ceph 版本装的是10.2.2，客户端内核过低会引起挂载失败，主要就是minion节点。

<code class=""># curl -sSL https:<span class="">//get.docker.com | sh  && service docker  restart

</span># apt-get install linux-image-<span class="">4.4</span><span class="">.0</span>-<span class="">22</span>-generic  &&  reboot</code>

www#gaodaima.com来源gao@daima#com搞(%代@#码@网搞代码

2.安装ceph-common,让minion调用rbd

<code class="">root@minion2:~# apt-get  install  python-rbd  ceph-common</code>

3.保存ceph-keyring和配置文件

<code class="">root@minion2:~# ll /etc/ceph
total <span class="">20

</span>drwxr-xr-x  <span class="">2</span> root root <span class="">4096</span> Jun <span class="">22</span>  <span class="">2016</span> ./
drwxr-xr-x <span class="">93</span> root root <span class="">4096</span> Nov <span class="">11</span> <span class="">11</span>:<span class="">18</span> ../
-rw-------  <span class="">1</span> root root   <span class="">63</span> Apr <span class="">25</span>  <span class="">2016</span> ceph.client.admin.keyring
-rw-r--r--  <span class="">1</span> root root  <span class="">249</span> Jun <span class="">22</span>  <span class="">2016</span> ceph.conf
-rw-r--r--  <span class="">1</span> root root   <span class="">92</span> Jun <span class="">14</span>  <span class="">2016</span> rbdmap</code>

4.获取ceph keyring 的base64编码字符串

<code class="">root@minion2:~# ceph auth get-key   client.admin
AQAmnRlX2zrqDRAAOiuOs2sIItGhAP6tNDa3Vg==root@minion2:~# 

root@minion2:~# grep key /etc/ceph/ceph.client.admin.keyring |awk <span class="">"{printf "%s", $NF}"
</span>AQAmnRlX2zrqDRAAOiuOs2sIItGhAP6tNDa3Vg==
 root@minion2:~# grep key /etc/ceph/ceph.client.admin.keyring |awk <span class="">"{printf "%s", $NF}"</span>|base64
QVFBbW5SbFgyenJxRFJBQU9pdU9zMnNJSXRHaEFQNnRORGEzVmc9PQ==</code>

5.创建基于keyring 的secret资源

<code class="">root@master:~# cat ceph-secret.yaml 
apiVersion: v1
kind: Secret
metadata:
  name: ceph-secret
data:

  key: QVFBbW5SbFgyenJxRFJBQU9pdU9zMnNJSXRHaEFQNnRORGEzVmc9PQo=</code>

<code class="">root@master:~# kubectl  create -f ceph-secret.yaml
root@master:~# kubectl  get secret
NAME               TYPE                                 DATA      AGE
ceph-secret           Opaque                              <span class="">1</span>         <span class="">187</span>d
<span class="">default</span>-token-rk17v    kubernetes.io/service-account-token       <span class="">3</span>         <span class="">208</span>d</code>

6.编辑一个可用的ReplicationController 让rbdpod跑起来

<code class=""># kubectl create -f  rbd-rc.yaml</code>

验证结果

这样的我们在pod就跑起来了，可以看到mount挂载信息，也可以进入容器查看

<code class="">root@minion2:~# docker ps | grep rbd
<span class="">4</span>b8fc04501b1        nginx          <span class="">"nginx -g "daemon off"</span>   <span class="">6</span> months ago        Up <span class="">6</span> months                             k8s_rbdpod.b1ab9160_rbpod-k3yxn_default_88540575-<span class="">3847</span>-<span class="">11e6</span>-a098-<span class="">0</span>a6a7c3a684c_f8caa4bc<span class="">3</span>c9c1240b72e        gcr.io/google_containers/pause:<span class="">2.0</span>                           <span class="">"/pause"</span>                 <span class="">6</span> months ago        Up <span class="">6</span> months                             k8s_POD<span class="">.25</span>c801ab_rbdpod-k3yxn_default_88540575-<span class="">3847</span>-<span class="">11e6</span>-a098-<span class="">0</span>a6a7c3a684c_c38f4ce7

root@minion2:~# mount | grep rbd
/dev/rbd0 on /<span class="">var</span>/lib/kubelet/plugins/kubernetes.io/rbd/rbd/data-image-wxq type ext4 (rw)

/<span class="">var</span>/lib/kubelet/plugins/kubernetes.io/rbd/rbd/data-image-wxq on /<span class="">var</span>/lib/kubelet/pods/<span class="">88540575</span>-<span class="">3847</span>-<span class="">11e6</span>-a098-<span class="">0</span>a6a7c3a684c/volumes/kubernetes.io~rbd/rbdpd type none (rw)</code>

这样k8s完美的结合了Ceph 块设备，它们又能愉悦的在一起玩耍了，至于有没有真感情且看未来。

喜欢 (0)赏

[搞代码]

分享 (0)