Kubernetes/存储/Ceph/Docker单机部署Ceph集群(无裸盘).md

> 本文作者：丁辉

# Docker单机部署Ceph集群(无裸盘)

> 无压力 C+V 即可部署成功

1. 创建Ceph专用网络

   ```bash
   docker network create --driver bridge --subnet 172.20.0.0/16 ceph-network
   ```

2. 拉取搭建用镜像

   ```bash
   docker pull ceph/daemon:latest
   ```

3. 搭建mon节点

   ```bash
   docker run -d --name ceph-mon \
     --restart=always \
     --network ceph-network --ip 172.20.0.10 \
     -e MON_NAME=ceph-mon \
     -e MON_IP=172.20.0.10 \
     -e CEPH_PUBLIC_NETWORK=172.20.0.0/16 \
     -v /etc/ceph:/etc/ceph \
     -v /var/lib/ceph/:/var/lib/ceph/ \
     -v /var/log/ceph/:/var/log/ceph/ \
     -v /etc/localtime:/etc/localtime:ro \
     ceph/daemon:latest mon
   ```

4. 禁用不安全认证

   > 这个参数控制是否允许在集群中通过全局 ID 回收机制（global ID reclaim）来认证用户身份

   ```bash
   docker exec ceph-mon ceph config set mon auth_allow_insecure_global_id_reclaim false
   ```

5. 创建 osd 秘钥

   ```bash
   docker exec ceph-mon ceph auth get client.bootstrap-osd -o /var/lib/ceph/bootstrap-osd/ceph.keyring
   ```

6. 搭建 osd 节点

   ```bash
   docker run -d --name ceph-osd-1 \
     --privileged=true \
     --restart=always \
     --network ceph-network --ip 172.20.0.11 \
     -e OSD_TYPE=directory \
     -v /etc/ceph:/etc/ceph \
     -v /var/lib/ceph/:/var/lib/ceph/ \
     -v /var/log/ceph/:/var/log/ceph/ \
     -v /var/lib/ceph/osd/osd-1:/var/lib/ceph/osd \
     -v /etc/localtime:/etc/localtime:ro \
     ceph/daemon:latest osd
     
   docker run -d --name ceph-osd-2 \
     --privileged=true \
     --restart=always \
     --network ceph-network --ip 172.20.0.12 \
     -e OSD_TYPE=directory \
     -v /etc/ceph:/etc/ceph \
     -v /var/lib/ceph/:/var/lib/ceph/ \
     -v /var/log/ceph/:/var/log/ceph/ \
     -v /var/lib/ceph/osd/osd-2:/var/lib/ceph/osd \
     -v /etc/localtime:/etc/localtime:ro \
     ceph/daemon:latest osd
     
   docker run -d --name ceph-osd-3 \
     --privileged=true \
     --restart=always \
     --network ceph-network --ip 172.20.0.13 \
     -e OSD_TYPE=directory \
     -v /etc/ceph:/etc/ceph \
     -v /var/lib/ceph/:/var/lib/ceph/ \
     -v /var/log/ceph/:/var/log/ceph/ \
     -v /var/lib/ceph/osd/osd-3:/var/lib/ceph/osd \
     -v /etc/localtime:/etc/localtime:ro \
     ceph/daemon:latest osd
   ```

7. 搭建 mgr 节点

   ```bash
   docker run -d --name ceph-mgr \
     --restart=always \
     --network ceph-network --ip 172.20.0.14 \
     -p 7000:7000 \
     -v /etc/ceph:/etc/ceph \
     -v /var/lib/ceph/:/var/lib/ceph/ \
     -v /var/log/ceph/:/var/log/ceph/ \
     -v /etc/localtime:/etc/localtime:ro \
     ceph/daemon:latest mgr
   ```

8. 开启管理界面(查看此文档)

   [MGR开启监控管理界面](https://gitee.com/offends/Kubernetes/blob/main/%E5%AD%98%E5%82%A8/Ceph/MGR%E5%BC%80%E5%90%AF%E7%9B%91%E6%8E%A7%E7%AE%A1%E7%90%86%E7%95%8C%E9%9D%A2.md)

9. 创建 rgw 秘钥

   ```bash
   docker exec ceph-mon ceph auth get client.bootstrap-rgw -o /var/lib/ceph/bootstrap-rgw/ceph.keyring
   ```

10. 搭建 rgw 节点

    ```bash
    docker run -d --name ceph-rgw \
      --restart=always \
      --network ceph-network --ip 172.20.0.15 \
      -p 7480:7480 \
      -e RGW_NAME=ceph-rgw \
      -v /etc/ceph:/etc/ceph \
      -v /var/lib/ceph/:/var/lib/ceph/ \
      -v /var/log/ceph/:/var/log/ceph/ \
      -v /etc/localtime:/etc/localtime:ro \
      ceph/daemon:latest rgw
    ```

11. 搭建 mds 节点

    ```bash
    docker run -d --name ceph-mds \
      --restart=always \
      --network ceph-network --ip 172.20.0.16 \
      -v /etc/ceph:/etc/ceph \
      -v /var/lib/ceph/:/var/lib/ceph/ \
      -v /var/log/ceph/:/var/log/ceph/ \
      -v /etc/localtime:/etc/localtime:ro \
      -e MDS_NAME=ceph-mds \
      -e CEPHFS_CREATE=1 \
      -e CEPHFS_NAME=cephfs \
      -e CEPHFS_DATA_POOL=cephfs_data \
      -e CEPHFS_DATA_POOL_PG=128 \
      -e CEPHFS_METADATA_POOL=cephfs_metadata \
      -e CEPHFS_METADATA_POOL_PG=64 \
      ceph/daemon:latest mds
    ```

12. 检查Ceph状态

    ```bash
    docker exec ceph-mon ceph -s
    ```

    挂载地址为：172.20.0.10:6789

# 卸载清理

1. 清理容器

   ```bash
   docker rm -f ceph-mon
   docker rm -f ceph-osd-1
   docker rm -f ceph-osd-2
   docker rm -f ceph-osd-3
   docker rm -f ceph-mgr
   docker rm -f ceph-rgw
   docker rm -f ceph-mds
   ```

2. 清理持久化目录

   ```bash
   rm -rf {/etc/ceph,/var/lib/ceph,/var/log/ceph}
   ```

3. 清理 Docker 网络

   ```bash
   docker network rm ceph-network
   ```