387 lines
9.2 KiB
Markdown
387 lines
9.2 KiB
Markdown
> 本文作者：丁辉
|
||
|
||
# Rke1部署Kubernetes集群
|
||
|
||
[RKE1文档](https://rke.docs.rancher.com/)
|
||
|
||
[Rancher中文文档](https://docs.rancher.cn/)
|
||
|
||
| 节点名称 | IP | Kubernetes角色 |
|
||
| :------------------: | :----------: | :----------------------: |
|
||
| k8s-master-1,Rke管理 | 192.168.1.10 | controlplane,etcd,worker |
|
||
| k8s-master-2 | 192.168.1.20 | controlplane,etcd,worker |
|
||
| k8s-master-3 | 192.168.1.30 | controlplane,etcd,worker |
|
||
|
||
## 环境准备
|
||
|
||
> !!!每次部署都写挺麻烦的索性都放在一个文件内了请查看 [Kubernetes基础环境准备](https://gitee.com/offends/Kubernetes/blob/main/%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3/Kubernetes%E5%9F%BA%E7%A1%80%E7%8E%AF%E5%A2%83%E5%87%86%E5%A4%87.md) ,请按照此文档初始化环境
|
||
|
||
### 所有节点执行
|
||
|
||
1. 配置 SSH
|
||
|
||
```bash
|
||
sed -i 's/#AllowTcpForwarding yes/AllowTcpForwarding yes/g' /etc/ssh/sshd_config
|
||
```
|
||
|
||
重启 SSH
|
||
|
||
```bash
|
||
systemctl restart sshd
|
||
```
|
||
|
||
2. 将用户添加到 docker 组
|
||
|
||
```bash
|
||
groupadd docker
|
||
useradd -m docker -g docker
|
||
```
|
||
|
||
> 使用其他用户
|
||
>
|
||
> ```bash
|
||
> useradd rke # 创建用户
|
||
> usermod -aG docker rke 将rke用户加入docker组
|
||
> ```
|
||
|
||
3. 配置 docker 用户免密登录
|
||
|
||
```bash
|
||
mkdir -p /home/docker/.ssh/
|
||
touch /home/docker/.ssh/authorized_keys
|
||
chmod 700 /home/docker/.ssh/
|
||
chown -R docker.docker /home/docker/.ssh/
|
||
chmod 600 /home/docker/.ssh/authorized_keys
|
||
```
|
||
|
||
### Rke管理节点执行
|
||
|
||
1. 生成密钥
|
||
|
||
```bash
|
||
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa -q
|
||
```
|
||
|
||
2. 查看主节点密钥
|
||
|
||
> 密钥需要到 RKE 初始化节点上获取，所有节点都是用此密钥
|
||
|
||
```bash
|
||
cat ~/.ssh/id_rsa.pub
|
||
```
|
||
|
||
|
||
### 所有节点执行
|
||
|
||
1. 粘贴密钥内容到此文件内(提示：所有节点粘贴Rke管理节密钥)
|
||
|
||
```bash
|
||
vi /home/docker/.ssh/authorized_keys
|
||
```
|
||
|
||
2. 验证是否可以免密登录
|
||
|
||
```bash
|
||
ssh docker@192.168.1.10
|
||
```
|
||
|
||
## Docker安装
|
||
|
||
1. Docker安装
|
||
|
||
```bash
|
||
curl https://releases.rancher.com/install-docker/20.10.sh | sh
|
||
```
|
||
|
||
> 传递参数使用国内源
|
||
>
|
||
> ```bash
|
||
> curl -fsSL https://releases.rancher.com/install-docker/20.10.sh | sh -s -- --mirror Aliyun
|
||
> ```
|
||
|
||
2. 启动 Docker
|
||
|
||
```bash
|
||
systemctl enable docker
|
||
systemctl start docker
|
||
```
|
||
|
||
## 安装并初始化Rke
|
||
|
||
[RKE二进制文件](https://github.com/rancher/rke/releases/)
|
||
|
||
1. 下载 RKE 二进制文件，并添加到可执行路径下
|
||
|
||
```bash
|
||
wget https://github.com/rancher/rke/releases/download/v1.4.10/rke_linux-amd64
|
||
```
|
||
|
||
2. 授权
|
||
|
||
```bash
|
||
chmod 777 rke_linux-amd64 && mv rke_linux-amd64 /usr/local/bin/rke
|
||
```
|
||
|
||
### 方法一 (不推荐怪麻烦的请看"方法二")
|
||
|
||
> 如果 `Number of Hosts` 填的是多节点则会提示输入多次节点信息
|
||
|
||
```bash
|
||
rke config --name cluster.yml
|
||
```
|
||
|
||
```bash
|
||
[+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]: #默认回车
|
||
[+] Number of Hosts [1]: #节点数量
|
||
[+] SSH Address of host (1) [none]: 192.168.1.10 #节点IP地址
|
||
[+] SSH Port of host (1) [22]: #默认回车
|
||
[+] SSH Private Key Path of host (192.168.1.10) [none]: #默认回车
|
||
[-] You have entered empty SSH key path, trying fetch from SSH key parameter
|
||
[+] SSH Private Key of host (192.168.1.10) [none]: #默认回车
|
||
[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~/.ssh/id_rsa
|
||
[+] SSH User of host (192.168.1.10) [ubuntu]: docker #SSH用户
|
||
[+] Is host (192.168.1.10) a Control Plane host (y/n)? [y]: y #是否为控制节点
|
||
[+] Is host (192.168.1.10) a Worker host (y/n)? [n]: y #是否为计算节点
|
||
[+] Is host (192.168.1.10) an etcd host (y/n)? [n]: y #是否为etcd节点
|
||
[+] Override Hostname of host (192.168.1.10) [none]: #默认回车
|
||
[+] Internal IP of host (192.168.1.10) [none]: 192.168.1.10 #主机内部IP
|
||
[+] Docker socket path on host (192.168.1.10) [/var/run/docker.sock]: #默认回车
|
||
[+] Network Plugin Type (flannel, calico, weave, canal, aci) [canal]: flannel #选择网络插件类型
|
||
[+] Authentication Strategy [x509]: #默认回车
|
||
[+] Authorization Mode (rbac, none) [rbac]: #默认回车
|
||
[+] Kubernetes Docker image [rancher/hyperkube:v1.26.8-rancher1]: #选择 k8s 版本
|
||
[+] Cluster domain [cluster.local]: #集群域
|
||
[+] Service Cluster IP Range [10.43.0.0/16]: #服务集群IP范围
|
||
[+] Enable PodSecurityPolicy [n]: #默认回车
|
||
[+] Cluster Network CIDR [10.42.0.0/16]: #集群网络CIDR
|
||
[+] Cluster DNS Service IP [10.43.0.10]: #集群DNS服务IP
|
||
[+] Add addon manifest URLs or YAML files [no]: #默认回车
|
||
```
|
||
|
||
基础参数修改
|
||
|
||
```bash
|
||
sed -i '/^ingress:$/,/^ provider:/ s/provider: ""/provider: "none"/' cluster.yml
|
||
```
|
||
|
||
### 方法二
|
||
|
||
1. 生成初始文件
|
||
|
||
```bash
|
||
rke config --empty --name cluster.yml
|
||
```
|
||
|
||
2. 按需要修改 address 参数
|
||
|
||
> 多节点则写多个 `address` 并通过调整 role 指定节点属性
|
||
|
||
```yml
|
||
nodes:
|
||
- address: 192.168.1.10
|
||
port: "22"
|
||
internal_address: 192.168.1.10
|
||
role:
|
||
- controlplane #管理
|
||
- worker #计算
|
||
- etcd #etcd节点
|
||
hostname_override: ""
|
||
user: docker
|
||
docker_socket: /var/run/docker.sock
|
||
ssh_key: ""
|
||
ssh_key_path: ~/.ssh/id_rsa
|
||
ssh_cert: ""
|
||
ssh_cert_path: ""
|
||
labels: {}
|
||
taints: []
|
||
# 格式一样此处省略 20,30 节点配置 ...
|
||
services:
|
||
...
|
||
```
|
||
|
||
3. 基础参数修改
|
||
|
||
```bash
|
||
sed -i 's/service_cluster_ip_range: ""/service_cluster_ip_range: 10.43.0.0\/16/' cluster.yml
|
||
sed -i 's/cluster_cidr: ""/cluster_cidr: 10.42.0.0\/16/' cluster.yml
|
||
sed -i 's/cluster_domain: ""/cluster_domain: cluster.local/' cluster.yml
|
||
sed -i 's/cluster_dns_server: ""/cluster_dns_server: 10.43.0.10/' cluster.yml
|
||
sed -i 's/plugin: ""/plugin: flannel/' cluster.yml
|
||
sed -i 's/strategy: ""/strategy: x509/' cluster.yml
|
||
sed -i 's/^\s*mode: ""$/ mode: rbac/' cluster.yml
|
||
sed -i '/^ingress:$/,/^ provider:/ s/provider: ""/provider: "none"/' cluster.yml
|
||
sed -i '/^[^ ]/ s/ssh_key_path: ""/ssh_key_path: ~\/.ssh\/id_rsa/g' cluster.yml
|
||
sed -i '0,/^\s*ssh_key_path: ""$/{s,^\s*ssh_key_path: ""$, ssh_key_path: ~/.ssh/id_rsa,}' cluster.yml
|
||
```
|
||
|
||
## 初始化 Kubernetes 集群
|
||
|
||
```bash
|
||
rke up
|
||
```
|
||
|
||
> - 禁用 metrics-server 组件
|
||
>
|
||
> ```bash
|
||
> sed -i '/^monitoring:$/,/^ provider:/ s/provider: ""/provider: "none"/' cluster.yml
|
||
> ```
|
||
>
|
||
> - 调整节点端口范围
|
||
>
|
||
> > 默认端口范围：30000-32767
|
||
>
|
||
> ```bash
|
||
> sed -i 's/service_node_port_range: ""/service_node_port_range: "10000-30000"/' cluster.yml
|
||
> ```
|
||
>
|
||
> - 关闭 Docker 版本检测
|
||
>
|
||
> ```bash
|
||
> sed -i 's/ignore_docker_version: null/ignore_docker_version: true/' cluster.yml
|
||
> ```
|
||
>
|
||
> - 调整部署版本
|
||
>
|
||
> - 查看当前 RKE 支持的Kubernetes版本
|
||
>
|
||
> ```bash
|
||
> rke config --list-version --all
|
||
> ```
|
||
>
|
||
> - 替换版本
|
||
>
|
||
> ```bash
|
||
> sed -i 's/kubernetes_version: ""/kubernetes_version: "v1.24.17-rancher1-1"/' cluster.yml
|
||
> ```
|
||
>
|
||
> - 更新集群
|
||
>
|
||
> ```bash
|
||
> rke up --update-only
|
||
> ```
|
||
|
||
## 安装 kubectl
|
||
|
||
[Kubectl二进制文件](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)
|
||
|
||
1. 下载 kubectl
|
||
|
||
```bash
|
||
curl -LO https://dl.k8s.io/release/v1.26.8/bin/linux/amd64/kubectl
|
||
```
|
||
|
||
2. 授权
|
||
|
||
```bash
|
||
chmod 777 kubectl && mv kubectl /usr/local/bin/
|
||
```
|
||
|
||
3. 添加 kubctl 文件
|
||
|
||
```bash
|
||
mkdir ~/.kube && cp kube_config_cluster.yml ~/.kube/config && chmod 600 ~/.kube/config
|
||
```
|
||
|
||
4. 验证
|
||
|
||
```bash
|
||
kubectl get node
|
||
```
|
||
|
||
> 本文中没有禁用 `monitoring` 所以也可以使用 `kubectl top node` 测试
|
||
|
||
## 卸载
|
||
|
||
1. 卸载 RKE 集群
|
||
|
||
```bash
|
||
rke remove
|
||
```
|
||
|
||
2. 清理残余容器
|
||
|
||
```bash
|
||
for i in $(docker ps -a | grep rancher | awk '{print $1}');do docker rm -f $i;done
|
||
for i in $(docker ps -a | grep rke | awk '{print $1}');do docker rm -f $i;done
|
||
```
|
||
|
||
3. 清除 Docker 引擎的废弃资源和缓存
|
||
|
||
```bash
|
||
docker system prune --all
|
||
```
|
||
|
||
4. 卸载挂载
|
||
|
||
```bash
|
||
mount | grep /var/lib/kubelet/pods/ | awk '{print $1}' | xargs umount -l
|
||
```
|
||
|
||
5. 删除持久化目录
|
||
|
||
```bash
|
||
rm -rf /var/lib/kubelet/
|
||
rm -rf /run/flannel/
|
||
```
|
||
|
||
## 备份和恢复
|
||
|
||
> 非常重要，他奶奶的吃大亏了
|
||
|
||
### 创建一次性快照
|
||
|
||
> RKE 会将节点快照保存在 `/opt/rke/etcd-snapshots` 路径下
|
||
|
||
```bash
|
||
rke etcd snapshot-save --config cluster.yml --name <快照名称>
|
||
```
|
||
|
||
### 恢复集群
|
||
|
||
```bash
|
||
rke etcd snapshot-restore --config cluster.yml --name <快照名称>
|
||
```
|
||
|
||
## 恢复 Rke配置文件
|
||
|
||
> 准备依赖 `jq`
|
||
>
|
||
> - Centos
|
||
>
|
||
> ```
|
||
> yum install jq -y
|
||
> ```
|
||
>
|
||
> - Ubuntu
|
||
>
|
||
> ```bash
|
||
> apt install jq -y
|
||
> ```
|
||
|
||
- 恢复 Kubectl 配置文件
|
||
|
||
> 修改 `--master-ip=` 为任意 K8S Master节点IP
|
||
|
||
```bash
|
||
curl -sfL https://gitee.com/offends/Kubernetes/raw/main/File/Shell/restore-rkestate-kubeconfig.sh | bash -s -- --master-ip=<K8S Master节点IP>
|
||
```
|
||
|
||
- 恢复 rkestate 状态文件
|
||
|
||
- 通过本地 kubectl 找回
|
||
|
||
```bash
|
||
kubectl get configmap -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r . > cluster.rkestate
|
||
```
|
||
|
||
- 通过 master 节点找回
|
||
|
||
```bash
|
||
curl -sfL https://gitee.com/offends/Kubernetes/raw/main/File/Shell/restore-rkestate-config.sh | bash -s
|
||
```
|
||
|
||
|
||
|
||
|
||
|