synchronization
This commit is contained in:
207
Helm/Helm部署Rook-Ceph.md
Normal file
207
Helm/Helm部署Rook-Ceph.md
Normal file
@@ -0,0 +1,207 @@
|
||||
> 本文作者：丁辉
|
||||
|
||||
# Helm部署Rook-Ceph
|
||||
|
||||
## 介绍
|
||||
|
||||
**Rook-Ceph 是一个开源的云原生存储编排器，旨在简化 Ceph 存储集群在 Kubernetes 环境中的部署和管理**。Rook-Ceph 将复杂的 Ceph 部署流程简化为易于在 Kubernetes 上实施的操作，使得 Ceph 集群能够无缝地与云原生环境集成。它利用 Kubernetes 的资源管理和调度能力，提供了一种高效且可扩展的方式来部署和管理存储解决方案。
|
||||
|
||||
## 基础准备
|
||||
|
||||
[Rook官方主页](https://rook.io/)
|
||||
|
||||
| 节点名称 | IP | 存储盘 |
|
||||
| :---------: | :----------: | :------: |
|
||||
| ceph-node-1 | 192.168.1.10 | /dev/sdb |
|
||||
| ceph-node-2 | 192.168.1.20 | /dev/sdb |
|
||||
| ceph-node-3 | 192.168.1.30 | /dev/sdb |
|
||||
|
||||
> 添加仓库
|
||||
|
||||
```bash
|
||||
helm repo add rook-release https://charts.rook.io/release
|
||||
helm repo update
|
||||
```
|
||||
|
||||
## 部署Rook-Ceph-Operator
|
||||
|
||||
1. 配置 values.yaml 文件
|
||||
|
||||
```bash
|
||||
vi rook-ceph-operator-values.yaml
|
||||
```
|
||||
|
||||
内容如下
|
||||
|
||||
```yaml
|
||||
# 配置镜像加速
|
||||
csi:
|
||||
cephcsi:
|
||||
repository: quay.dockerproxy.com/cephcsi/cephcsi
|
||||
|
||||
registrar:
|
||||
repository: registry.aliyuncs.com/google_containers/csi-node-driver-registrar
|
||||
|
||||
provisioner:
|
||||
repository: registry.aliyuncs.com/google_containers/csi-provisioner
|
||||
|
||||
snapshotter:
|
||||
repository: registry.aliyuncs.com/google_containers/csi-snapshotter
|
||||
|
||||
attacher:
|
||||
repository: registry.aliyuncs.com/google_containers/csi-attacher
|
||||
|
||||
resizer:
|
||||
repository: registry.aliyuncs.com/google_containers/csi-resizer
|
||||
```
|
||||
|
||||
2. 部署
|
||||
|
||||
```bash
|
||||
helm install rook-ceph rook-release/rook-ceph \
|
||||
--namespace rook-ceph --create-namespace \
|
||||
-f rook-ceph-operator-values.yaml
|
||||
```
|
||||
|
||||
## 部署 Rook-Ceph-Cluster
|
||||
|
||||
1. 配置 values.yaml 文件
|
||||
|
||||
```bash
|
||||
vi rook-ceph-cluster-values.yaml
|
||||
```
|
||||
|
||||
内容如下
|
||||
|
||||
```yaml
|
||||
operatorNamespace: rook-ceph
|
||||
|
||||
toolbox:
|
||||
enabled: true
|
||||
|
||||
cephClusterSpec:
|
||||
storage:
|
||||
useAllNodes: false # 关闭使用所有Node
|
||||
useAllDevices: false # 关闭使用所有设备
|
||||
# 指定存储节点和磁盘
|
||||
nodes:
|
||||
- name: "192.168.1.10"
|
||||
devices:
|
||||
- name: "sdb"
|
||||
- name: "192.168.1.20"
|
||||
devices:
|
||||
- name: "sdb"
|
||||
- name: "192.168.1.20"
|
||||
devices:
|
||||
- name: "sdb"
|
||||
#- name: "192.168.1.100"
|
||||
#deviceFilter: "^sd." # 过滤以 "sd." 开头的设备
|
||||
#- name: "nvme0"
|
||||
#config:
|
||||
#osdsPerDevice: "5" # 创建多个 OSD
|
||||
#- name: "/dev/disk/XXXX-XXXX" # 指定实际设备文件的路径
|
||||
|
||||
# 开启监控面板
|
||||
dashboard:
|
||||
enabled: true
|
||||
ssl: false
|
||||
|
||||
# 配置调度策略
|
||||
placement:
|
||||
all:
|
||||
nodeAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
nodeSelectorTerms:
|
||||
- matchExpressions:
|
||||
- key: role
|
||||
operator: In
|
||||
values:
|
||||
- storage-node
|
||||
podAffinity:
|
||||
podAntiAffinity:
|
||||
topologySpreadConstraints:
|
||||
tolerations:
|
||||
- key: storage-node
|
||||
operator: Exists
|
||||
```
|
||||
|
||||
2. Ceph节点添加标签
|
||||
|
||||
```bash
|
||||
kubectl label nodes {ceph-node-1,ceph-node-2,ceph-node-3} role=storage-node
|
||||
```
|
||||
|
||||
3. 部署
|
||||
|
||||
```bash
|
||||
helm install rook-ceph-cluster rook-release/rook-ceph-cluster \
|
||||
--namespace rook-ceph --create-namespace \
|
||||
-f rook-ceph-cluster-values.yaml
|
||||
```
|
||||
|
||||
4. 查看状态
|
||||
|
||||
```bash
|
||||
kubectl -n rook-ceph exec -it $(kubectl get pod -l app=rook-ceph-tools -n rook-ceph | awk '{print $1}' | grep -v NAME) bash
|
||||
```
|
||||
|
||||
进入容器后查看 ceph 状态
|
||||
|
||||
```bash
|
||||
ceph -s
|
||||
```
|
||||
|
||||
> health: HEALTH_OK
|
||||
|
||||
5. 查看密码登录 Dashboard
|
||||
|
||||
```bash
|
||||
kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath="{['data']['password']}" | base64 --decode && echo
|
||||
```
|
||||
|
||||
> 账户：admin
|
||||
|
||||
## 卸载
|
||||
|
||||
[集群清理文档](https://rook.io/docs/rook/latest-release/Storage-Configuration/ceph-teardown/#removing-the-cluster-crd-finalizer)
|
||||
|
||||
1. 卸载 Rook-Ceph-Cluster
|
||||
|
||||
```bash
|
||||
helm uninstall rook-ceph-cluster -n rook-ceph
|
||||
```
|
||||
|
||||
2. 卸载 Rook-Ceph-Operator
|
||||
|
||||
```bash
|
||||
helm uninstall rook-ceph -n rook-ceph
|
||||
```
|
||||
|
||||
3. 删除 CRD 资源
|
||||
|
||||
```bash
|
||||
for CRD in $(kubectl get crd -n rook-ceph | awk '/ceph.rook.io/ {print $1}'); do
|
||||
kubectl get -n rook-ceph "$CRD" -o name | \
|
||||
xargs -I {} kubectl patch -n rook-ceph {} --type merge -p '{"metadata":{"finalizers": []}}'
|
||||
done
|
||||
```
|
||||
|
||||
4. 删除配置资源
|
||||
|
||||
```bash
|
||||
kubectl -n rook-ceph patch configmap rook-ceph-mon-endpoints --type merge -p '{"metadata":{"finalizers": []}}'
|
||||
kubectl -n rook-ceph patch secrets rook-ceph-mon --type merge -p '{"metadata":{"finalizers": []}}'
|
||||
```
|
||||
|
||||
5. 删除命名空间
|
||||
|
||||
```bash
|
||||
kubectl delete ns rook-ceph
|
||||
```
|
||||
|
||||
6. 删除持久化目录
|
||||
|
||||
```bash
|
||||
rm -rf /var/lib/rook
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user