### 1.2 高可用设计原则

**高可用设计原则**：

```yaml
ha_design_principles:
  elimination_of_single_point:
    - "每个组件都需要高可用"
    - "无单点故障"
    - "冗余部署"
    
  failure_isolation:
    - "故障隔离"
    - "故障域控制"
    - "级联失败防止"
    
  automatic_recovery:
    - "故障自动检测"
    - "故障自动恢复"
    - "服务自动转移"
    
  consistency:
    - "数据一致性保证"
    - "状态同步"
    - "选举机制"

### 2.3 etcd运维

**etcd备份恢复**：

```markdown
### etcd运维

**etcd备份**：

```bash
#!/bin/bash
# backup-etcd.sh

ETCD_ENDPOINTS=${ETCD_ENDPOINTS:-"https://127.0.0.1:2379"}
BACKUP_DIR=${BACKUP_DIR:-"/var/backups/etcd"}
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="${BACKUP_DIR}/etcd-snapshot-${DATE}.db"

# 创建备份目录
mkdir -p ${BACKUP_DIR}

# 执行备份
ETCDCTL_API=3 etcdctl \
  --endpoints=${ETCD_ENDPOINTS} \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  snapshot save ${BACKUP_FILE}

# 压缩备份
gzip ${BACKUP_FILE}

# 保留最近30天备份
find ${BACKUP_DIR} -name "*.db.gz" -mtime +30 -delete

echo "备份完成: ${BACKUP_FILE}.gz"

**etcd健康检查**：

```bash
# 检查etcd集群健康
ETCDCTL_API=3 etcdctl \
  --endpoints=https://192.168.1.101:2379,https://192.168.1.102:2379,https://192.168.1.103:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  endpoint health

# 检查etcd集群状态
ETCDCTL_API=3 etcdctl \
  --endpoints=https://192.168.1.101:2379,https://192.168.1.102:2379,https://192.168.1.103:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  endpoint status -w table

### 3.2 scheduler高可用

**scheduler选举机制**：

```markdown
### scheduler高可用

**scheduler选举**：

```yaml
scheduler_ha:
  mechanism: "Lease锁竞争"
  leader_election:
    leaseDuration: 15s
    renewDuration: 10s
    retryPeriod: 5s
    retryLimit: 3
  behavior: "只有一个scheduler活跃，其他处于待命状态"

### 3.3 controller-manager高可用

**controller-manager选举**：

```markdown
### controller-manager高可用

**controller-manager选举**：

```yaml
controller_ha:
  mechanism: "Leader锁竞争"
  leader_election:
    leaseDuration: 15s
    renewDuration: 10s
    retryPeriod: 5s
  components:
    - "deployment-controller"
    - "replicaset-controller"
    - "endpoints-controller"
    - "namespace-controller"
    - "serviceaccount-controller"

## 四、Worker Plane高可用

### 4.1 节点高可用

**节点高可用架构**：

```markdown
## Worker Plane高可用

### 节点高可用

**节点高可用要求**：

```yaml
node_ha_requirements:
  minimum_nodes: 3
  recommended_nodes: 5
  per_az:
    minimum: 1
    recommended: 2
    
  node_pool_design:
    - "按业务类型划分NodePool"
    - "按可用区划分NodePool"
    - "按规格大小划分NodePool"

### 4.2 Pod高可用

**Pod高可用策略**：

```markdown
### Pod高可用

**Pod反亲缘性**：

```yaml
# Pod反亲缘性配置
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values:
              - frontend
          topologyKey: kubernetes.io/hostname
  containers:
  - name: app
    image: app:v1

### 4.3 应用层高可用

**HPA高可用**：

```markdown
### 应用层高可用

**HorizontalPodAutoscaler**：

```yaml
# HPA配置
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60

### 5.2 网络高可用

**网络高可用配置**：

```markdown
### 网络高可用

**Ingress高可用**：

```yaml
# Ingress高可用配置
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "route"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
spec:
  ingressClassName: nginx
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 80

### 5.3 监控与告警

**高可用监控指标**：

```yaml
# 高可用监控指标
ha_monitoring:
  etcd:
    - "节点健康状态"
    - "Leader是否存在"
    - "磁盘性能"
    - "Raft propose延迟"
    
  apiserver:
    - "请求延迟"
    - "请求错误率"
    - "并发请求数"
    
  scheduler:
    - "调度延迟"
    - "调度失败数"
    
  nodes:
    - "Node Ready状态"
    - "Pod数量分布"
    - "资源使用率"

### 5.4 灾备方案

**灾备架构**：

```yaml
# 灾备方案设计
disaster_recovery:
  backup_types:
    - "etcd数据备份"
    - "资源清单备份"
    - "Secrets和ConfigMaps备份"
    
  backup_schedule:
    etcd: "每小时一次"
    resources: "每天一次"
    
  recovery_point_objective: "1小时"
  recovery_time_objective: "30分钟"

K8s高可用三平面，etcd奇数防脑裂，
Control Plane多实例，apiserver负载均衡，
scheduler选举Leader，controller竞争Leader，
Worker多节点部署，Pod反亲缘性分散，
可用区AZ要跨域，托管方案最省心。