414 lines
10 KiB
Markdown
414 lines
10 KiB
Markdown
> 本文作者：丁辉
|
||
|
||
# Pod水平自动缩放
|
||
|
||
[官方文档](https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment)
|
||
|
||
## 基础配置准备
|
||
|
||
1. 确保 Metrics Server 已安装
|
||
|
||
[Github](https://github.com/kubernetes-sigs/metrics-server/)
|
||
|
||
检查 metrics-server 是否运行
|
||
|
||
```bash
|
||
kubectl get pods -n kube-system | grep metrics-server
|
||
```
|
||
|
||
如果没有，可以安装它（如果是标准的 Kubernetes 集群）
|
||
|
||
```bash
|
||
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
|
||
```
|
||
|
||
2. 确保 Pod 已设置资源请求
|
||
|
||
[Pod配置资源请求](https://gitee.com/offends/Kubernetes/blob/main/%E4%BD%BF%E7%94%A8%E6%96%87%E6%A1%A3/Pod%E9%85%8D%E7%BD%AE%E8%B5%84%E6%BA%90%E8%AF%B7%E6%B1%82.md)
|
||
|
||
## 介绍
|
||
|
||
> 自动伸缩（HPA）的计算是基于 `requests` 的，而不是 `limits`
|
||
|
||
**详细解释**
|
||
|
||
HPA 使用的计算公式是：
|
||
|
||
- 使用率(Utilization)= Pod 当前实际使用量 ÷ Pod 的 requests 值
|
||
- 绝对值(AverageValue) = 所有 Pod 的实际 CPU 使用量总和 ÷ Pod 数量
|
||
|
||
## 使用条件介绍
|
||
|
||
- ✅ **类型**：`AverageValue`（绝对值）
|
||
- ✅ **目标**：所有 Pod 平均使用 **80m CPU**
|
||
- ✅ **特点**：不依赖 `requests` 设置
|
||
- ⚠️ **注意**：如果 Pod 的 `requests` 设置变化，可能需要调整这个值
|
||
- 🔄 **对比**：如果改为 `Utilization: 80%`，则目标是 CPU 使用量达到 `requests.cpu` 的 80%
|
||
|
||
**选择建议**：
|
||
|
||
- 如果 Pod 的 `requests.cpu` 稳定且合理 → 使用 `Utilization`
|
||
- 如果 `requests` 经常变化或设置不合理 → 使用 `AverageValue`
|
||
- 两者可以互相转换：`averageValue = requests.cpu × (utilization/100)`
|
||
|
||
## 创建 HPA
|
||
|
||
**基础命令格式对比表**
|
||
|
||
| 参数/命令部分 | 基础命令 | 说明 |
|
||
| :------------: | :---------------------------: | :----------------------: |
|
||
| **命令结构** | `kubectl autoscale` | 创建或更新HPA |
|
||
| **目标资源** | `deployment/nginx-deployment` | 要自动扩缩容的Deployment |
|
||
| **最小副本数** | `--min=3` | 最少保持的Pod数量 |
|
||
| **最大副本数** | `--max=6` | 最多允许的Pod数量 |
|
||
| **CPU阈值** | `--cpu=80` | CPU使用率目标值 |
|
||
| **内存阈值** | `--memory=75` | 内存使用率目标值 |
|
||
|
||
### 命令创建(默认使用AverageValue算计)
|
||
|
||
- 绝对值 50m
|
||
|
||
```bash
|
||
kubectl autoscale deployment/nginx-deployment --min=1 --max=3 --cpu=50 --memory=50
|
||
```
|
||
|
||
- 绝对值50Mi
|
||
|
||
```bash
|
||
kubectl autoscale deployment/nginx-deployment --min=1 --max=3 --memory=50
|
||
```
|
||
|
||
### Yaml 创建(使用Utilization)
|
||
|
||
- CPU 80%
|
||
- 内存 75%
|
||
|
||
1. 编辑 Yaml
|
||
|
||
```bash
|
||
vi nginx-deployment-hpa-utilization.yaml
|
||
```
|
||
|
||
内容如下
|
||
|
||
```yaml
|
||
apiVersion: autoscaling/v2
|
||
kind: HorizontalPodAutoscaler
|
||
metadata:
|
||
name: nginx-deployment
|
||
spec:
|
||
scaleTargetRef:
|
||
apiVersion: apps/v1
|
||
kind: Deployment
|
||
name: nginx-deployment
|
||
minReplicas: 1
|
||
maxReplicas: 3
|
||
metrics:
|
||
- type: Resource
|
||
resource:
|
||
name: cpu
|
||
target:
|
||
type: Utilization
|
||
averageUtilization: 80
|
||
- type: Resource
|
||
resource:
|
||
name: memory
|
||
target:
|
||
type: Utilization
|
||
averageUtilization: 75
|
||
```
|
||
|
||
2. 部署
|
||
|
||
```bash
|
||
kubectl apply -f nginx-deployment-hpa-utilization.yaml
|
||
```
|
||
|
||
### Yaml 创建(使用AverageValue)
|
||
|
||
- CPU 绝对值 50m
|
||
- 内存 绝对值50Mi
|
||
|
||
1. 编辑 Yaml
|
||
|
||
```bash
|
||
vi nginx-deployment-hpa-averagevalue.yaml
|
||
```
|
||
|
||
内容如下
|
||
|
||
```yaml
|
||
apiVersion: autoscaling/v2
|
||
kind: HorizontalPodAutoscaler
|
||
metadata:
|
||
name: nginx-deployment
|
||
spec:
|
||
scaleTargetRef:
|
||
apiVersion: apps/v1
|
||
kind: Deployment
|
||
name: nginx-deployment
|
||
minReplicas: 1
|
||
maxReplicas: 3
|
||
metrics:
|
||
- type: Resource
|
||
resource:
|
||
name: cpu
|
||
target:
|
||
type: AverageValue
|
||
averageValue: 50m
|
||
- type: Resource
|
||
resource:
|
||
name: memory
|
||
target:
|
||
type: AverageValue
|
||
averageValue: 50Mi
|
||
```
|
||
|
||
2. 部署
|
||
|
||
```bash
|
||
kubectl apply -f nginx-deployment-hpa-averagevalue.yaml
|
||
```
|
||
|
||
## 验证 HPA
|
||
|
||
创建后，可以检查 HPA 状态：
|
||
|
||
- 查看 HPA
|
||
|
||
```bash
|
||
kubectl get hpa
|
||
```
|
||
|
||
- 详细查看
|
||
|
||
```bash
|
||
kubectl describe hpa nginx-deployment
|
||
```
|
||
|
||
- 查看当前 CPU 使用率
|
||
|
||
```bash
|
||
kubectl top pods
|
||
```
|
||
|
||
## 增加负载验证
|
||
|
||
1. 模拟负载
|
||
|
||
```bash
|
||
kubectl run -i --tty load-generator --rm --image=busybox:1.28 --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://nginx-deployment; done"
|
||
```
|
||
|
||
2. 观察
|
||
|
||
```bash
|
||
kubectl get hpa nginx-deployment --watch
|
||
```
|
||
|
||
## 其他类型
|
||
|
||
### 根据每个 Pod 每秒处理的数据包数量来扩缩容
|
||
|
||
具体解释：
|
||
|
||
- **监控指标**：`packets-per-second`（每秒数据包数）
|
||
- **目标值**：每个 Pod 平均每秒处理 **1000 个数据包**（1k = 1000）
|
||
- **扩缩逻辑**：
|
||
- 如果所有 Pod 的平均值 **超过** 1000 个包/秒 → 增加 Pod 数量
|
||
- 如果所有 Pod 的平均值 **低于** 1000 个包/秒 → 减少 Pod 数量
|
||
|
||
1. 编辑 Yaml
|
||
|
||
```bash
|
||
vi nginx-deployment-hpa-pod.yaml
|
||
```
|
||
|
||
内容如下
|
||
|
||
```yaml
|
||
apiVersion: autoscaling/v2
|
||
kind: HorizontalPodAutoscaler
|
||
metadata:
|
||
name: nginx-deployment
|
||
spec:
|
||
scaleTargetRef:
|
||
apiVersion: apps/v1
|
||
kind: Deployment
|
||
name: nginx-deployment
|
||
minReplicas: 1
|
||
maxReplicas: 3
|
||
metrics:
|
||
- type: Pods
|
||
pods:
|
||
metric:
|
||
name: packets-per-second
|
||
target:
|
||
type: AverageValue
|
||
averageValue: 1k
|
||
```
|
||
|
||
2. 部署
|
||
|
||
```bash
|
||
kubectl apply -f nginx-deployment-hpa-pod.yaml
|
||
```
|
||
|
||
### 根据 Ingress 的每秒请求数来扩缩容后端应用
|
||
|
||
具体解释：
|
||
|
||
- **监控对象**：名为 `main-route` 的 Ingress
|
||
- **监控指标**：`requests-per-second`（Ingress 每秒处理的请求数）
|
||
- **目标值**：整个 Ingress 每秒处理 **2000 个请求**（2k = 2000）
|
||
- **扩缩逻辑**：
|
||
- 如果这个 Ingress 的请求数 **超过** 2000 请求/秒 → 增加后端 Pod 数量
|
||
- 如果这个 Ingress 的请求数 **低于** 2000 请求/秒 → 减少后端 Pod 数量
|
||
|
||
1. 编辑 Yaml
|
||
|
||
```bash
|
||
vi nginx-deployment-hpa-object.yaml
|
||
```
|
||
|
||
内容如下
|
||
|
||
```yaml
|
||
apiVersion: autoscaling/v2
|
||
kind: HorizontalPodAutoscaler
|
||
metadata:
|
||
name: nginx-deployment
|
||
spec:
|
||
scaleTargetRef:
|
||
apiVersion: apps/v1
|
||
kind: Deployment
|
||
name: nginx-deployment
|
||
minReplicas: 1
|
||
maxReplicas: 3
|
||
metrics:
|
||
- type: Object
|
||
object:
|
||
metric:
|
||
name: requests-per-second
|
||
describedObject:
|
||
apiVersion: networking.k8s.io/v1
|
||
kind: Ingress
|
||
name: main-route # 但监控的是这个 Ingress 的流量
|
||
target:
|
||
type: Value
|
||
value: 2k # 整个 Ingress 的目标：2000 请求/秒
|
||
```
|
||
|
||
2. 部署
|
||
|
||
```bash
|
||
kubectl apply -f nginx-deployment-hpa-object.yaml
|
||
```
|
||
|
||
### 根据带 GET 标签的 HTTP 请求指标来扩缩容
|
||
|
||
具体解释：
|
||
|
||
- **监控对象**：某个 Kubernetes 资源（如 Ingress、Service 等）
|
||
- **监控指标**：`http_requests`（HTTP 请求总数或速率）
|
||
- **指标筛选**：只选择带有 `verb: GET` 标签的指标
|
||
- **目标值**：需要在其他地方指定（这里没有显示完整的 target 部分）
|
||
|
||
1. 编辑 Yaml
|
||
|
||
```bash
|
||
vi nginx-deployment-hpa-object-api.yaml
|
||
```
|
||
|
||
内容如下
|
||
|
||
```yaml
|
||
apiVersion: autoscaling/v2
|
||
kind: HorizontalPodAutoscaler
|
||
metadata:
|
||
name: api-hpa
|
||
spec:
|
||
scaleTargetRef:
|
||
apiVersion: apps/v1
|
||
kind: Deployment
|
||
name: api-server
|
||
minReplicas: 2
|
||
maxReplicas: 10
|
||
metrics:
|
||
- type: Object
|
||
object:
|
||
describedObject:
|
||
apiVersion: v1
|
||
kind: Service
|
||
name: api-service
|
||
metric:
|
||
name: http_requests
|
||
selector:
|
||
matchLabels:
|
||
verb: GET # 只监控 GET 请求
|
||
path: "/api/v1/users" # 可以进一步指定路径
|
||
status: "200" # 还可以按状态码筛选
|
||
target:
|
||
type: AverageValue
|
||
averageValue: 500 # 目标：每秒 500 个 GET 请求
|
||
```
|
||
|
||
2. 部署
|
||
|
||
```bash
|
||
kubectl apply -f nginx-deployment-hpa-object-api.yaml
|
||
```
|
||
|
||
### 根据外部消息队列中的任务数量来扩缩容
|
||
|
||
具体解释：
|
||
|
||
- **监控来源**：外部系统（不是 Kubernetes 内部）
|
||
- **监控指标**：`queue_messages_ready`（队列中待处理的消息数量）
|
||
- **队列筛选**：只监控名为 `worker_tasks` 的队列
|
||
- **目标值**：保持队列中大约有 **30 个待处理消息**
|
||
- **扩缩逻辑**：
|
||
- 如果队列消息 **超过 30 个** → 增加 Worker Pod 数量
|
||
- 如果队列消息 **低于 30 个** → 减少 Worker Pod 数量
|
||
|
||
1. 编辑 Yaml
|
||
|
||
```bash
|
||
vi nginx-deployment-hpa-object-worker.yaml
|
||
```
|
||
|
||
内容如下
|
||
|
||
```yaml
|
||
apiVersion: autoscaling/v2
|
||
kind: HorizontalPodAutoscaler
|
||
metadata:
|
||
name: worker-hpa
|
||
spec:
|
||
scaleTargetRef:
|
||
apiVersion: apps/v1
|
||
kind: Deployment
|
||
name: task-worker # 处理队列任务的 Worker 应用
|
||
minReplicas: 1
|
||
maxReplicas: 20
|
||
metrics:
|
||
- type: External
|
||
external:
|
||
metric:
|
||
name: queue_messages_ready # 队列中"就绪"的消息数
|
||
selector:
|
||
matchLabels:
|
||
queue: "worker_tasks" # 只监控这个特定队列
|
||
# 还可以按其他标签筛选，如：
|
||
# vhost: "production"
|
||
# exchange: "orders"
|
||
target:
|
||
type: AverageValue
|
||
averageValue: 30 # 目标：保持队列中有30个待处理消息
|
||
```
|
||
|
||
2. 部署
|
||
|
||
```bash
|
||
kubectl apply -f nginx-deployment-hpa-object-worker.yaml
|
||
``` |