353 lines
7.6 KiB
Markdown
353 lines
7.6 KiB
Markdown
# Trivy Operator 部署指南
|
||
|
||
> 创建时间: 2025-12-30
|
||
> 状态: 待从 EU Harbor 服务器执行
|
||
|
||
---
|
||
|
||
## 🚨 问题说明
|
||
|
||
### 现状
|
||
|
||
Trivy Operator 部署失败，原因是镜像拉取超时：
|
||
|
||
```
|
||
Failed to pull image "mirror.gcr.io/aquasec/trivy-operator:0.29.0":
|
||
Error response from daemon: Get "https://mirror.gcr.io/v2/": context deadline exceeded
|
||
```
|
||
|
||
### 根本原因
|
||
|
||
- 本地网络无法访问 `mirror.gcr.io`
|
||
- 本地网络无法访问 EU Harbor (`harbor.eu.lingjingwanxiang.cn`)
|
||
- 必须从 **EU Harbor 服务器** 执行镜像缓存和部署
|
||
|
||
---
|
||
|
||
## ✅ 解决方案
|
||
|
||
### 步骤 1: 在 EU Harbor 服务器上缓存镜像
|
||
|
||
登录到 EU Harbor 服务器并执行：
|
||
|
||
```bash
|
||
# 1. 进入脚本目录
|
||
cd /Users/brunogao/work/codes/tools/macos/k8s
|
||
|
||
# 2. 缓存 Trivy Operator 主镜像
|
||
./cache-image.sh aquasec/trivy-operator:0.29.0
|
||
|
||
# 3. 缓存 Trivy 扫描器镜像（如果使用自定义版本）
|
||
./cache-image.sh aquasec/trivy:0.50.0
|
||
|
||
# 4. 验证镜像已缓存
|
||
docker images | grep trivy
|
||
```
|
||
|
||
**预期输出**：
|
||
```
|
||
harbor.omniverseai.net:8080/library/aquasec-trivy-operator 0.29.0
|
||
harbor.omniverseai.net:8080/library/aquasec-trivy 0.50.0
|
||
```
|
||
|
||
### 步骤 2: 在本地部署 Trivy Operator
|
||
|
||
回到本地机器，使用缓存的镜像部署：
|
||
|
||
```bash
|
||
# 1. 创建命名空间
|
||
kubectl create namespace trivy-system
|
||
|
||
# 2. 添加 Helm 仓库
|
||
helm repo add aqua https://aquasecurity.github.io/helm-charts/
|
||
helm repo update
|
||
|
||
# 3. 创建自定义 values 文件（使用本地 Harbor 镜像）
|
||
cat > /tmp/trivy-operator-local.yaml << 'EOF'
|
||
# 使用本地 Harbor 缓存的镜像
|
||
image:
|
||
registry: harbor.omniverseai.net:8080
|
||
repository: library/aquasec-trivy-operator
|
||
tag: "0.29.0"
|
||
|
||
trivy:
|
||
image:
|
||
registry: harbor.omniverseai.net:8080
|
||
repository: library/aquasec-trivy
|
||
tag: "0.50.0"
|
||
|
||
# 扫描配置
|
||
ignoreUnfixed: true
|
||
severity: "CRITICAL,HIGH,MEDIUM"
|
||
|
||
operator:
|
||
# 并发扫描限制
|
||
scanJobsConcurrentLimit: 2
|
||
scanJobTimeout: 10m
|
||
metricsBindAddress: ":8080"
|
||
|
||
# 资源限制
|
||
resources:
|
||
limits:
|
||
cpu: 500m
|
||
memory: 512Mi
|
||
requests:
|
||
cpu: 100m
|
||
memory: 128Mi
|
||
|
||
# ServiceMonitor（可选，如果需要 Prometheus 监控）
|
||
serviceMonitor:
|
||
enabled: true
|
||
namespace: monitoring
|
||
EOF
|
||
|
||
# 4. 安装 Trivy Operator
|
||
helm install trivy-operator aqua/trivy-operator \
|
||
--namespace trivy-system \
|
||
--values /tmp/trivy-operator-local.yaml \
|
||
--timeout 5m \
|
||
--wait
|
||
|
||
# 5. 验证部署
|
||
kubectl get pods -n trivy-system
|
||
kubectl get deployment -n trivy-system
|
||
```
|
||
|
||
### 步骤 3: 验证 Trivy 扫描功能
|
||
|
||
```bash
|
||
# 1. 等待扫描完成（约 5-10 分钟）
|
||
watch kubectl get vulnerabilityreports -A
|
||
|
||
# 2. 查看漏洞报告
|
||
kubectl get vulnerabilityreports -A
|
||
|
||
# 3. 查看特定 Pod 的漏洞详情
|
||
kubectl describe vulnerabilityreport <report-name> -n <namespace>
|
||
|
||
# 4. 查看配置审计报告
|
||
kubectl get configauditreports -A
|
||
|
||
# 5. 导出漏洞报告为 JSON
|
||
kubectl get vulnerabilityreports -A -o json > /tmp/trivy-vulnerabilities.json
|
||
|
||
# 6. 统计漏洞严重程度
|
||
kubectl get vulnerabilityreports -A -o json | \
|
||
jq '.items[].report.summary'
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 部署验证清单
|
||
|
||
完成部署后，验证以下项目：
|
||
|
||
- [ ] **Trivy Operator Pod 运行正常**
|
||
```bash
|
||
kubectl get pods -n trivy-system
|
||
# 预期: trivy-operator-xxx 1/1 Running
|
||
```
|
||
|
||
- [ ] **扫描任务自动启动**
|
||
```bash
|
||
kubectl get jobs -A | grep scan
|
||
# 预期: 看到多个 scan-vulnerabilityreport-xxx 任务
|
||
```
|
||
|
||
- [ ] **漏洞报告生成**
|
||
```bash
|
||
kubectl get vulnerabilityreports -A | wc -l
|
||
# 预期: 数量 > 0
|
||
```
|
||
|
||
- [ ] **配置审计报告生成**
|
||
```bash
|
||
kubectl get configauditreports -A | wc -l
|
||
# 预期: 数量 > 0
|
||
```
|
||
|
||
- [ ] **ServiceMonitor 已创建（如果启用）**
|
||
```bash
|
||
kubectl get servicemonitor -n monitoring | grep trivy
|
||
# 预期: trivy-operator ServiceMonitor
|
||
```
|
||
|
||
---
|
||
|
||
## 🔍 故障排查
|
||
|
||
### 问题 1: Pod 仍然是 ImagePullBackOff
|
||
|
||
**检查**：
|
||
```bash
|
||
kubectl describe pod -n trivy-system <pod-name> | grep -A 5 "Events:"
|
||
```
|
||
|
||
**可能原因**：
|
||
- 镜像未正确缓存到本地 Harbor
|
||
- Helm values 中镜像路径配置错误
|
||
|
||
**解决方案**：
|
||
```bash
|
||
# 验证本地 Harbor 中的镜像
|
||
docker pull harbor.omniverseai.net:8080/library/aquasec-trivy-operator:0.29.0
|
||
|
||
# 如果失败，重新从 EU Harbor 服务器缓存
|
||
ssh user@eu-harbor-server
|
||
cd /Users/brunogao/work/codes/tools/macos/k8s
|
||
./cache-image.sh aquasec/trivy-operator:0.29.0
|
||
```
|
||
|
||
### 问题 2: 扫描任务未启动
|
||
|
||
**检查**：
|
||
```bash
|
||
kubectl logs -n trivy-system deployment/trivy-operator
|
||
```
|
||
|
||
**可能原因**：
|
||
- Operator 权限不足
|
||
- Trivy 扫描器镜像拉取失败
|
||
|
||
**解决方案**：
|
||
```bash
|
||
# 检查 RBAC 权限
|
||
kubectl get clusterrole | grep trivy
|
||
kubectl get clusterrolebinding | grep trivy
|
||
|
||
# 检查扫描器镜像
|
||
./cache-image.sh aquasec/trivy:0.50.0
|
||
```
|
||
|
||
### 问题 3: 扫描任务失败
|
||
|
||
**检查**：
|
||
```bash
|
||
kubectl get jobs -A | grep scan
|
||
kubectl logs job/<failed-job-name> -n <namespace>
|
||
```
|
||
|
||
**可能原因**：
|
||
- 扫描超时（scanJobTimeout 太短）
|
||
- 资源不足
|
||
|
||
**解决方案**：
|
||
```bash
|
||
# 增加超时时间和资源
|
||
helm upgrade trivy-operator aqua/trivy-operator \
|
||
--namespace trivy-system \
|
||
--reuse-values \
|
||
--set operator.scanJobTimeout=15m \
|
||
--set resources.limits.memory=1Gi
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 预期扫描结果
|
||
|
||
部署成功后，Trivy Operator 会自动扫描：
|
||
|
||
### 1. 容器漏洞 (VulnerabilityReports)
|
||
|
||
扫描所有 Pod 中的容器镜像，检测：
|
||
- 操作系统包漏洞
|
||
- 应用依赖漏洞
|
||
- 已知 CVE
|
||
|
||
**示例**：
|
||
```yaml
|
||
apiVersion: aquasecurity.github.io/v1alpha1
|
||
kind: VulnerabilityReport
|
||
metadata:
|
||
name: pod-nginx-deployment-xxx
|
||
spec:
|
||
summary:
|
||
criticalCount: 2
|
||
highCount: 5
|
||
mediumCount: 10
|
||
lowCount: 20
|
||
```
|
||
|
||
### 2. 配置审计 (ConfigAuditReports)
|
||
|
||
检查 Kubernetes 资源配置的安全最佳实践：
|
||
- 特权容器
|
||
- 缺少资源限制
|
||
- 不安全的 capabilities
|
||
- Root 用户运行
|
||
|
||
**示例**：
|
||
```yaml
|
||
apiVersion: aquasecurity.github.io/v1alpha1
|
||
kind: ConfigAuditReport
|
||
metadata:
|
||
name: pod-nginx-deployment-xxx
|
||
spec:
|
||
summary:
|
||
criticalCount: 0
|
||
highCount: 1
|
||
mediumCount: 3
|
||
lowCount: 5
|
||
```
|
||
|
||
---
|
||
|
||
## 🎯 下一步行动
|
||
|
||
### 立即执行（必须从 EU Harbor 服务器）
|
||
|
||
1. **登录 EU Harbor 服务器**
|
||
```bash
|
||
ssh user@eu-harbor-server
|
||
```
|
||
|
||
2. **缓存所需镜像**
|
||
```bash
|
||
cd /Users/brunogao/work/codes/tools/macos/k8s
|
||
./cache-image.sh aquasec/trivy-operator:0.29.0
|
||
./cache-image.sh aquasec/trivy:0.50.0
|
||
```
|
||
|
||
3. **回到本地执行部署**
|
||
```bash
|
||
# 使用上面的 Helm 安装命令
|
||
```
|
||
|
||
### 可选优化（部署成功后）
|
||
|
||
1. **配置定期扫描**
|
||
- 默认情况下，Trivy Operator 会自动扫描新 Pod
|
||
- 可配置定期重新扫描间隔
|
||
|
||
2. **集成到 Grafana**
|
||
- 创建 Dashboard 展示漏洞统计
|
||
- 使用 ServiceMonitor 收集指标
|
||
|
||
3. **配置告警**
|
||
- 为高危漏洞配置 Prometheus 告警规则
|
||
- 发送通知到 AlertManager
|
||
|
||
---
|
||
|
||
## 📝 相关文件
|
||
|
||
- **cache-image.sh**: 镜像缓存脚本（已修复组织名问题）
|
||
- **CACHE-IMAGE-FIX.md**: cache-image.sh 修复说明
|
||
- **deploy-trivy-operator-cn.sh**: 备用部署脚本（使用阿里云镜像，成功率低）
|
||
|
||
---
|
||
|
||
## ⚠️ 重要提示
|
||
|
||
1. **必须从 EU Harbor 服务器缓存镜像** - 本地网络无法访问 EU Harbor 和 GCR
|
||
2. **镜像命名规则** - 缓存后的镜像名会将 `/` 替换为 `-`：
|
||
- `aquasec/trivy-operator` → `aquasec-trivy-operator`
|
||
3. **资源消耗** - Trivy 扫描会创建临时 Job，建议在非高峰期部署
|
||
4. **扫描时间** - 首次扫描所有 Pod 需要 5-10 分钟
|
||
|
||
---
|
||
|
||
**创建时间**: 2025-12-30
|
||
**状态**: 待执行
|
||
**优先级**: P2（非关键，但推荐部署以提升安全性）
|