> 本文作者：丁辉 # Rke1部署Kubernetes集群 [RKE1文档](https://rke.docs.rancher.com/) [Rancher中文文档](https://docs.rancher.cn/) | 节点名称 | IP | Kubernetes角色 | | :------------------: | :----------: | :----------------------: | | k8s-master-1,Rke管理 | 192.168.1.10 | controlplane,etcd,worker | | k8s-master-2 | 192.168.1.20 | controlplane,etcd,worker | | k8s-master-3 | 192.168.1.30 | controlplane,etcd,worker | ## 环境准备 > !!!每次部署都写挺麻烦的索性都放在一个文件内了请查看 [Kubernetes基础环境准备](https://gitee.com/offends/Kubernetes/blob/main/%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3/Kubernetes%E5%9F%BA%E7%A1%80%E7%8E%AF%E5%A2%83%E5%87%86%E5%A4%87.md) ,请按照此文档初始化环境 ### 所有节点执行 1. 配置 SSH ```bash sed -i 's/#AllowTcpForwarding yes/AllowTcpForwarding yes/g' /etc/ssh/sshd_config ``` 重启 SSH ```bash systemctl restart sshd ``` 2. 将用户添加到 docker 组 ```bash groupadd docker useradd -m docker -g docker ``` > 使用其他用户 > > ```bash > useradd rke # 创建用户 > usermod -aG docker rke 将rke用户加入docker组 > ``` 3. 配置 docker 用户免密登录 ```bash mkdir -p /home/docker/.ssh/ touch /home/docker/.ssh/authorized_keys chmod 700 /home/docker/.ssh/ chown -R docker.docker /home/docker/.ssh/ chmod 600 /home/docker/.ssh/authorized_keys ``` ### Rke管理节点执行 1. 生成密钥 ```bash ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa -q ``` 2. 查看主节点密钥 > 密钥需要到 RKE 初始化节点上获取，所有节点都是用此密钥 ```bash cat ~/.ssh/id_rsa.pub ``` ### 所有节点执行 1. 粘贴密钥内容到此文件内(提示：所有节点粘贴Rke管理节密钥) ```bash vi /home/docker/.ssh/authorized_keys ``` 2. 验证是否可以免密登录 ```bash ssh docker@192.168.1.10 ``` ## Docker安装 1. Docker安装 ```bash curl https://releases.rancher.com/install-docker/20.10.sh | sh ``` > 传递参数使用国内源 > > ```bash > curl -fsSL https://releases.rancher.com/install-docker/20.10.sh | sh -s -- --mirror Aliyun > ``` 2. 启动 Docker ```bash systemctl enable docker systemctl start docker ``` ## 安装并初始化Rke [RKE二进制文件](https://github.com/rancher/rke/releases/) 1. 下载 RKE 二进制文件，并添加到可执行路径下 ```bash wget https://github.com/rancher/rke/releases/download/v1.4.10/rke_linux-amd64 ``` 2. 授权 ```bash chmod 777 rke_linux-amd64 && mv rke_linux-amd64 /usr/local/bin/rke ``` ### 方法一 (不推荐怪麻烦的请看"方法二") > 如果 `Number of Hosts` 填的是多节点则会提示输入多次节点信息 ```bash rke config --name cluster.yml ``` ```bash [+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]: #默认回车 [+] Number of Hosts [1]: #节点数量 [+] SSH Address of host (1) [none]: 192.168.1.10 #节点IP地址 [+] SSH Port of host (1) [22]: #默认回车 [+] SSH Private Key Path of host (192.168.1.10) [none]: #默认回车 [-] You have entered empty SSH key path, trying fetch from SSH key parameter [+] SSH Private Key of host (192.168.1.10) [none]: #默认回车 [-] You have entered empty SSH key, defaulting to cluster level SSH key: ~/.ssh/id_rsa [+] SSH User of host (192.168.1.10) [ubuntu]: docker #SSH用户 [+] Is host (192.168.1.10) a Control Plane host (y/n)? [y]: y #是否为控制节点 [+] Is host (192.168.1.10) a Worker host (y/n)? [n]: y #是否为计算节点 [+] Is host (192.168.1.10) an etcd host (y/n)? [n]: y #是否为etcd节点 [+] Override Hostname of host (192.168.1.10) [none]: #默认回车 [+] Internal IP of host (192.168.1.10) [none]: 192.168.1.10 #主机内部IP [+] Docker socket path on host (192.168.1.10) [/var/run/docker.sock]: #默认回车 [+] Network Plugin Type (flannel, calico, weave, canal, aci) [canal]: flannel #选择网络插件类型 [+] Authentication Strategy [x509]: #默认回车 [+] Authorization Mode (rbac, none) [rbac]: #默认回车 [+] Kubernetes Docker image [rancher/hyperkube:v1.26.8-rancher1]: #选择 k8s 版本 [+] Cluster domain [cluster.local]: #集群域 [+] Service Cluster IP Range [10.43.0.0/16]: #服务集群IP范围 [+] Enable PodSecurityPolicy [n]: #默认回车 [+] Cluster Network CIDR [10.42.0.0/16]: #集群网络CIDR [+] Cluster DNS Service IP [10.43.0.10]: #集群DNS服务IP [+] Add addon manifest URLs or YAML files [no]: #默认回车 ``` 基础参数修改 ```bash sed -i '/^ingress:$/,/^ provider:/ s/provider: ""/provider: "none"/' cluster.yml ``` ### 方法二 1. 生成初始文件 ```bash rke config --empty --name cluster.yml ``` 2. 按需要修改 address 参数 > 多节点则写多个 `address` 并通过调整 role 指定节点属性 ```yml nodes: - address: 192.168.1.10 port: "22" internal_address: 192.168.1.10 role: - controlplane #管理 - worker #计算 - etcd #etcd节点 hostname_override: "" user: docker docker_socket: /var/run/docker.sock ssh_key: "" ssh_key_path: ~/.ssh/id_rsa ssh_cert: "" ssh_cert_path: "" labels: {} taints: [] # 格式一样此处省略 20,30 节点配置 ... services: ... ``` 3. 基础参数修改 ```bash sed -i 's/service_cluster_ip_range: ""/service_cluster_ip_range: 10.43.0.0\/16/' cluster.yml sed -i 's/cluster_cidr: ""/cluster_cidr: 10.42.0.0\/16/' cluster.yml sed -i 's/cluster_domain: ""/cluster_domain: cluster.local/' cluster.yml sed -i 's/cluster_dns_server: ""/cluster_dns_server: 10.43.0.10/' cluster.yml sed -i 's/plugin: ""/plugin: flannel/' cluster.yml sed -i 's/strategy: ""/strategy: x509/' cluster.yml sed -i 's/^\s*mode: ""$/ mode: rbac/' cluster.yml sed -i '/^ingress:$/,/^ provider:/ s/provider: ""/provider: "none"/' cluster.yml sed -i '/^[^ ]/ s/ssh_key_path: ""/ssh_key_path: ~\/.ssh\/id_rsa/g' cluster.yml sed -i '0,/^\s*ssh_key_path: ""$/{s,^\s*ssh_key_path: ""$, ssh_key_path: ~/.ssh/id_rsa,}' cluster.yml ``` ## 初始化 Kubernetes 集群 ```bash rke up ``` > - 禁用 metrics-server 组件 > > ```bash > sed -i '/^monitoring:$/,/^ provider:/ s/provider: ""/provider: "none"/' cluster.yml > ``` > > - 调整节点端口范围 > > > 默认端口范围：30000-32767 > > ```bash > sed -i 's/service_node_port_range: ""/service_node_port_range: "10000-30000"/' cluster.yml > ``` > > - 关闭 Docker 版本检测 > > ```bash > sed -i 's/ignore_docker_version: null/ignore_docker_version: true/' cluster.yml > ``` > > - 调整部署版本 > > - 查看当前 RKE 支持的Kubernetes版本 > > ```bash > rke config --list-version --all > ``` > > - 替换版本 > > ```bash > sed -i 's/kubernetes_version: ""/kubernetes_version: "v1.24.17-rancher1-1"/' cluster.yml > ``` > > - 更新集群 > > ```bash > rke up --update-only > ``` ## 安装 kubectl [Kubectl二进制文件](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/) 1. 下载 kubectl ```bash curl -LO https://dl.k8s.io/release/v1.26.8/bin/linux/amd64/kubectl ``` 2. 授权 ```bash chmod 777 kubectl && mv kubectl /usr/local/bin/ ``` 3. 添加 kubctl 文件 ```bash mkdir ~/.kube && cp kube_config_cluster.yml ~/.kube/config && chmod 600 ~/.kube/config ``` 4. 验证 ```bash kubectl get node ``` > 本文中没有禁用 `monitoring` 所以也可以使用 `kubectl top node` 测试 ## 卸载 1. 卸载 RKE 集群 ```bash rke remove ``` 2. 清理残余容器 ```bash for i in $(docker ps -a | grep rancher | awk '{print $1}');do docker rm -f $i;done for i in $(docker ps -a | grep rke | awk '{print $1}');do docker rm -f $i;done ``` 3. 清除 Docker 引擎的废弃资源和缓存 ```bash docker system prune --all ``` 4. 卸载挂载 ```bash mount | grep /var/lib/kubelet/pods/ | awk '{print $1}' | xargs umount -l ``` 5. 删除持久化目录 ```bash rm -rf /var/lib/kubelet/ rm -rf /run/flannel/ ``` ## 备份和恢复 > 非常重要，他奶奶的吃大亏了 ### 创建一次性快照 > RKE 会将节点快照保存在 `/opt/rke/etcd-snapshots` 路径下 ```bash rke etcd snapshot-save --config cluster.yml --name <快照名称> ``` ### 恢复集群 ```bash rke etcd snapshot-restore --config cluster.yml --name <快照名称> ``` ## 恢复 Rke配置文件 > 准备依赖 `jq` > > - Centos > > ``` > yum install jq -y > ``` > > - Ubuntu > > ```bash > apt install jq -y > ``` - 恢复 Kubectl 配置文件 > 修改 `--master-ip=` 为任意 K8S Master节点IP ```bash curl -sfL https://gitee.com/offends/Kubernetes/raw/main/File/Shell/restore-rkestate-kubeconfig.sh | bash -s -- --master-ip= ``` - 恢复 rkestate 状态文件 - 通过本地 kubectl 找回 ```bash kubectl get configmap -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r . > cluster.rkestate ``` - 通过 master 节点找回 ```bash curl -sfL https://gitee.com/offends/Kubernetes/raw/main/File/Shell/restore-rkestate-config.sh | bash -s ```