## 🏗️ Architecture & Design - Add ADR collection (15 Architecture Decision Records) - Comprehensive system design documentation - Implementation roadmap and project planning ## 📋 Requirements & Documentation - Detailed functional requirements - API specifications (OpenAPI & AsyncAPI) - Postman collections and optimization reports ## 🗄️ Database & Configuration - Database migration scripts (CDC, core schema) - Resilience4j configuration for resilience patterns - Alert manager and Prometheus monitoring rules ## 🎯 Project Templates - Admin frontend template (Arco Pro + Vue 3) - Big screen data visualization template (ECharts) - Enterprise backend template (Jeecg Boot + Spring Boot) ## 🧪 Testing & Quality - K6 performance testing scenarios - Load testing benchmark scripts - Quality assurance framework setup ## 📦 Infrastructure & Deployment - Complete infrastructure configurations - Deployment strategies and CI/CD pipelines - Monitoring and observability setup This commit establishes the complete foundation for the MVP Sprint 01 development cycle with all necessary architectural decisions, templates, and documentation ready for team collaboration. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
部署方案与运维指南
概述
基于云原生架构和容器化技术,制定光明科学城科技成果转化平台的完整部署方案,包括环境规划、容器化部署、自动化运维和监控告警。
1. 部署架构总览
1.1 环境规划
graph TB
subgraph "开发环境 (Development)"
Dev[本地开发环境]
DevDB[开发数据库]
DevCache[开发缓存]
end
subgraph "测试环境 (Testing)"
Test[测试集群]
TestDB[测试数据库]
TestCache[测试缓存]
TestMQ[测试消息队列]
end
subgraph "预发布环境 (Staging)"
Stage[预发布集群]
StageDB[预发布数据库]
StageCache[预发布缓存]
StageMQ[预发布消息队列]
end
subgraph "生产环境 (Production)"
Prod[生产集群]
ProdDB[生产数据库集群]
ProdCache[生产缓存集群]
ProdMQ[生产消息队列集群]
ProdStorage[生产对象存储]
end
Dev --> Test
Test --> Stage
Stage --> Prod
1.2 基础设施规划
| 环境 | 计算资源 | 存储资源 | 网络配置 | 高可用 |
|---|---|---|---|---|
| 开发环境 | 2C4G × 3台 | 100GB SSD | 单网络 | 否 |
| 测试环境 | 4C8G × 5台 | 500GB SSD | 双网络 | 否 |
| 预发布环境 | 8C16G × 7台 | 1TB SSD | 双网络 | 是 |
| 生产环境 | 16C32G × 15台 | 10TB SSD + 50TB HDD | 三网络 | 是 |
1.3 技术栈部署清单
# 部署技术栈
infrastructure:
container_platform: Kubernetes 1.28+
container_runtime: containerd 1.7+
service_mesh: Istio 1.19+
ingress: Ingress NGINX 1.8+
database:
primary: 达梦 DM8 (生产)
development: PostgreSQL 15 (开发/测试)
cache: Redis 7.0 Cluster
search: OpenSearch 2.11
vector: Milvus 2.3
storage:
object_storage: MinIO Cluster
persistent_storage: Longhorn 1.5
messaging:
primary: RabbitMQ 3.12 Cluster
streaming: Kafka 3.6 Cluster
monitoring:
metrics: Prometheus + Grafana
logging: Loki + Grafana
tracing: Jaeger
apm: OpenTelemetry
cicd:
git: GitLab 16.0
cicd: GitLab CI + Argo CD
registry: Harbor 2.9
scanning: Trivy + Grype
2. 容器化部署
2.1 Docker 镜像构建
2.1.1 后端服务 Dockerfile
# gmkx-user-service/Dockerfile
FROM openjdk:17-jdk-slim AS builder
WORKDIR /app
COPY pom.xml .
COPY src src/
# 下载依赖并构建
RUN apt-get update && apt-get install -y maven && \
mvn dependency:go-offline -B && \
mvn clean package -DskipTests -B
# 运行阶段
FROM openjdk:17-jre-slim
# 安装必要工具和监控代理
RUN apt-get update && \
apt-get install -y curl wget dumb-init && \
rm -rf /var/lib/apt/lists/*
# 创建应用用户
RUN groupadd -r gmkx && useradd -r -g gmkx gmkx
# 设置工作目录
WORKDIR /app
# 复制应用JAR
COPY --from=builder /app/target/gmkx-user-service-*.jar app.jar
# 复制配置文件
COPY docker/entrypoint.sh .
COPY docker/application-docker.yml .
# 设置权限
RUN chown -R gmkx:gmkx /app && \
chmod +x entrypoint.sh
# 健康检查
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
CMD curl -f http://localhost:8080/actuator/health || exit 1
# 切换到非root用户
USER gmkx
# 暴露端口
EXPOSE 8080 9090
# 入口点
ENTRYPOINT ["dumb-init", "--", "./entrypoint.sh"]
2.1.2 前端应用 Dockerfile
# gmkx-web/Dockerfile
# 构建阶段
FROM node:18-alpine AS builder
WORKDIR /app
# 复制依赖文件
COPY package*.json ./
RUN npm ci --only=production && npm cache clean --force
# 复制源代码
COPY . .
# 构建应用
RUN npm run build
# 运行阶段
FROM nginx:1.25-alpine
# 安装必要工具
RUN apk add --no-cache curl
# 复制构建产物
COPY --from=builder /app/dist /usr/share/nginx/html
# 复制nginx配置
COPY docker/nginx.conf /etc/nginx/nginx.conf
COPY docker/default.conf /etc/nginx/conf.d/default.conf
# 复制启动脚本
COPY docker/entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
# 健康检查
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
CMD curl -f http://localhost || exit 1
# 暴露端口
EXPOSE 80
# 启动
ENTRYPOINT ["/entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]
2.1.3 AI 服务 Dockerfile
# gmkx-ai/Dockerfile
FROM python:3.11-slim AS builder
WORKDIR /app
# 安装系统依赖
RUN apt-get update && \
apt-get install -y build-essential curl && \
rm -rf /var/lib/apt/lists/*
# 复制依赖文件
COPY requirements/ requirements/
RUN pip install --no-cache-dir -r requirements/prod.txt
# 复制应用代码
COPY app/ app/
COPY scripts/ scripts/
# 运行阶段
FROM python:3.11-slim
# 安装运行时依赖
RUN apt-get update && \
apt-get install -y curl && \
rm -rf /var/lib/apt/lists/*
# 创建应用用户
RUN groupadd -r gmkx && useradd -r -g gmkx gmkx
WORKDIR /app
# 从构建阶段复制
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY --from=builder /app .
# 设置权限
RUN chown -R gmkx:gmkx /app
# 健康检查
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
CMD curl -f http://localhost:8000/health || exit 1
# 切换用户
USER gmkx
# 暴露端口
EXPOSE 8000
# 启动命令
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
2.2 Docker Compose 开发环境
2.2.1 开发环境编排
# docker-compose.dev.yml
version: '3.8'
services:
# 数据库
postgres:
image: postgres:15-alpine
environment:
POSTGRES_DB: gmkx_dev
POSTGRES_USER: gmkx
POSTGRES_PASSWORD: gmkx123
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
- ./sql/init:/docker-entrypoint-initdb.d
healthcheck:
test: ["CMD-SHELL", "pg_isready -U gmkx"]
interval: 10s
timeout: 5s
retries: 5
# Redis 缓存
redis:
image: redis:7-alpine
command: redis-server --appendonly yes --requirepass redis123
ports:
- "6379:6379"
volumes:
- redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "auth", "redis123", "ping"]
interval: 10s
timeout: 5s
retries: 5
# RabbitMQ 消息队列
rabbitmq:
image: rabbitmq:3.12-management-alpine
environment:
RABBITMQ_DEFAULT_USER: gmkx
RABBITMQ_DEFAULT_PASS: gmkx123
RABBITMQ_DEFAULT_VHOST: /gmkx
ports:
- "5672:5672"
- "15672:15672"
volumes:
- rabbitmq_data:/var/lib/rabbitmq
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
interval: 30s
timeout: 10s
retries: 5
# OpenSearch 搜索引擎
opensearch:
image: opensearchproject/opensearch:2.11.0
environment:
- discovery.type=single-node
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=Admin123!
- plugins.security.disabled=true
ports:
- "9200:9200"
volumes:
- opensearch_data:/usr/share/opensearch/data
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9200/_cluster/health"]
interval: 30s
timeout: 10s
retries: 5
# Milvus 向量数据库
etcd:
image: quay.io/coreos/etcd:v3.5.0
environment:
- ETCD_AUTO_COMPACTION_MODE=revision
- ETCD_AUTO_COMPACTION_RETENTION=1000
- ETCD_QUOTA_BACKEND_BYTES=4294967296
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
minio:
image: minio/minio:latest
environment:
MINIO_ACCESS_KEY: minioadmin
MINIO_SECRET_KEY: minioadmin
command: minio server /data
volumes:
- minio_data:/data
milvus:
image: milvusdb/milvus:v2.3.2
command: ["milvus", "run", "standalone"]
environment:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
ports:
- "19530:19530"
depends_on:
- "etcd"
- "minio"
# 用户服务
user-service:
build:
context: ./gmkx-user-service
dockerfile: Dockerfile
environment:
SPRING_PROFILES_ACTIVE: docker
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/gmkx_dev
SPRING_REDIS_HOST: redis
SPRING_RABBITMQ_HOST: rabbitmq
ports:
- "8081:8080"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
rabbitmq:
condition: service_healthy
volumes:
- ./logs:/app/logs
# 成果服务
achievement-service:
build:
context: ./gmkx-achievement-service
dockerfile: Dockerfile
environment:
SPRING_PROFILES_ACTIVE: docker
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/gmkx_dev
SPRING_REDIS_HOST: redis
OPENSEARCH_URIS: http://opensearch:9200
ports:
- "8082:8080"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
opensearch:
condition: service_healthy
# AI 服务
ai-service:
build:
context: ./gmkx-ai
dockerfile: Dockerfile
environment:
DATABASE_URL: postgresql://gmkx:gmkx123@postgres:5432/gmkx_dev
REDIS_URL: redis://redis:6379/0
MILVUS_HOST: milvus
MILVUS_PORT: 19530
ports:
- "8000:8000"
depends_on:
- postgres
- redis
- milvus
# API 网关
gateway:
build:
context: ./gmkx-gateway
dockerfile: Dockerfile
environment:
SPRING_PROFILES_ACTIVE: docker
USER_SERVICE_URL: http://user-service:8080
ACHIEVEMENT_SERVICE_URL: http://achievement-service:8080
AI_SERVICE_URL: http://ai-service:8000
ports:
- "8080:8080"
depends_on:
- user-service
- achievement-service
- ai-service
# 前端应用
web-app:
build:
context: ./gmkx-web
dockerfile: Dockerfile
environment:
API_BASE_URL: http://gateway:8080
ports:
- "3000:80"
depends_on:
- gateway
volumes:
postgres_data:
redis_data:
rabbitmq_data:
opensearch_data:
minio_data:
networks:
default:
name: gmkx-dev-network
2.3 Kubernetes 生产部署
2.3.1 命名空间和资源配额
# k8s/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: gmkx-platform
labels:
name: gmkx-platform
environment: production
---
apiVersion: v1
kind: ResourceQuota
metadata:
name: gmkx-resource-quota
namespace: gmkx-platform
spec:
hard:
requests.cpu: "20"
requests.memory: 40Gi
limits.cpu: "50"
limits.memory: 100Gi
persistentvolumeclaims: "20"
services: "30"
secrets: "20"
configmaps: "20"
---
apiVersion: v1
kind: LimitRange
metadata:
name: gmkx-limit-range
namespace: gmkx-platform
spec:
limits:
- default:
cpu: "1"
memory: "2Gi"
defaultRequest:
cpu: "100m"
memory: "256Mi"
type: Container
2.3.2 配置管理
# k8s/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: gmkx-config
namespace: gmkx-platform
data:
# 应用配置
application.yml: |
server:
port: 8080
spring:
profiles:
active: production
datasource:
url: jdbc:dm://dm8-service:5236/GMKX_PROD
username: ${DB_USERNAME}
password: ${DB_PASSWORD}
driver-class-name: dm.jdbc.driver.DmDriver
redis:
cluster:
nodes: ${REDIS_CLUSTER_NODES}
password: ${REDIS_PASSWORD}
timeout: 3000ms
lettuce:
pool:
max-active: 8
max-idle: 8
min-idle: 0
rabbitmq:
addresses: ${RABBITMQ_ADDRESSES}
username: ${RABBITMQ_USERNAME}
password: ${RABBITMQ_PASSWORD}
virtual-host: /gmkx
# 日志配置
logging:
level:
com.gmkx: INFO
org.springframework.security: DEBUG
pattern:
console: "%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"
file: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
# 监控配置
management:
endpoints:
web:
exposure:
include: health,info,metrics,prometheus
endpoint:
health:
show-details: always
metrics:
export:
prometheus:
enabled: true
# Nginx 配置
nginx.conf: |
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_proxied any;
gzip_comp_level 6;
gzip_types
application/javascript
application/json
application/xml
text/css
text/javascript
text/plain
text/xml;
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# API 代理
location /api/ {
proxy_pass http://gmkx-gateway:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# 静态资源
location / {
try_files $uri $uri/ /index.html;
add_header Cache-Control "public, max-age=31536000" always;
}
# 健康检查
location /health {
access_log off;
return 200 "healthy\n";
add_header Content-Type text/plain;
}
}
}
---
apiVersion: v1
kind: Secret
metadata:
name: gmkx-secrets
namespace: gmkx-platform
type: Opaque
data:
db-username: Z21reA== # base64 encoded
db-password: Z21reDEyMw==
redis-password: cmVkaXMxMjM=
rabbitmq-username: Z21reA==
rabbitmq-password: Z21reDEyMw==
jwt-secret: bXktand0LXNlY3JldC1rZXk=
2.3.3 服务部署
# k8s/user-service.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: gmkx-user-service
namespace: gmkx-platform
labels:
app: gmkx-user-service
version: v1.0.0
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app: gmkx-user-service
template:
metadata:
labels:
app: gmkx-user-service
version: v1.0.0
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9090"
prometheus.io/path: "/actuator/prometheus"
spec:
serviceAccountName: gmkx-service-account
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
containers:
- name: user-service
image: harbor.gmsc.gov.cn/gmkx/user-service:v1.0.0
imagePullPolicy: Always
ports:
- containerPort: 8080
name: http
- containerPort: 9090
name: metrics
env:
- name: SPRING_PROFILES_ACTIVE
value: "production"
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: gmkx-secrets
key: db-username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: gmkx-secrets
key: db-password
- name: REDIS_CLUSTER_NODES
value: "redis-cluster:6379"
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: gmkx-secrets
key: redis-password
- name: RABBITMQ_ADDRESSES
value: "rabbitmq-cluster:5672"
- name: RABBITMQ_USERNAME
valueFrom:
secretKeyRef:
name: gmkx-secrets
key: rabbitmq-username
- name: RABBITMQ_PASSWORD
valueFrom:
secretKeyRef:
name: gmkx-secrets
key: rabbitmq-password
- name: JAVA_OPTS
value: "-Xms1g -Xmx2g -XX:+UseG1GC -XX:MaxGCPauseMillis=200"
volumeMounts:
- name: config-volume
mountPath: /app/config
- name: logs-volume
mountPath: /app/logs
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 2
memory: 4Gi
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 60
periodSeconds: 30
timeoutSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
startupProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 30
volumes:
- name: config-volume
configMap:
name: gmkx-config
- name: logs-volume
emptyDir: {}
imagePullSecrets:
- name: harbor-registry-secret
---
apiVersion: v1
kind: Service
metadata:
name: gmkx-user-service
namespace: gmkx-platform
labels:
app: gmkx-user-service
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: 8080
protocol: TCP
name: http
- port: 9090
targetPort: 9090
protocol: TCP
name: metrics
selector:
app: gmkx-user-service
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: gmkx-user-service-hpa
namespace: gmkx-platform
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: gmkx-user-service
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 10
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 60
policies:
- type: Percent
value: 50
periodSeconds: 60
2.3.4 网关和入口配置
# k8s/gateway.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: gmkx-gateway
namespace: gmkx-platform
spec:
replicas: 3
selector:
matchLabels:
app: gmkx-gateway
template:
metadata:
labels:
app: gmkx-gateway
spec:
containers:
- name: gateway
image: harbor.gmsc.gov.cn/gmkx/gateway:v1.0.0
ports:
- containerPort: 8080
env:
- name: USER_SERVICE_URL
value: "http://gmkx-user-service:8080"
- name: ACHIEVEMENT_SERVICE_URL
value: "http://gmkx-achievement-service:8080"
- name: DEMAND_SERVICE_URL
value: "http://gmkx-demand-service:8080"
- name: AI_SERVICE_URL
value: "http://gmkx-ai-service:8000"
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 1
memory: 2Gi
---
apiVersion: v1
kind: Service
metadata:
name: gmkx-gateway
namespace: gmkx-platform
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: 8080
selector:
app: gmkx-gateway
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gmkx-ingress
namespace: gmkx-platform
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "100m"
nginx.ingress.kubernetes.io/rate-limit: "100"
nginx.ingress.kubernetes.io/rate-limit-window: "1m"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- platform.gmsc.gov.cn
secretName: gmkx-tls-secret
rules:
- host: platform.gmsc.gov.cn
http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: gmkx-gateway
port:
number: 8080
- path: /
pathType: Prefix
backend:
service:
name: gmkx-web-app
port:
number: 80
3. 数据库部署
3.1 达梦数据库集群部署
3.1.1 主从复制配置
# k8s/dm8-database.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: dm8-config
namespace: gmkx-platform
data:
dm.ini: |
INSTANCE_NAME = DMSERVER
PORT_NUM = 5236
MEMORY_TARGET = 4096
MEMORY_POOL = 2048
MAX_SESSIONS = 1000
MAX_OS_FILE_SIZE = 134217728
BUFFER = 2000
LOG_SIZE = 1024
SYSTEM_LOG_SIZE = 1024
RLOG_SIZE = 1024
REDO_LOG_SIZE = 1024
TEMP_SIZE = 500
SORT_BUF_SIZE = 100
MAX_BUFFER_SIZE = 4096
COMPATIBLE_MODE = 2
BLANK_PAD_MODE = 1
LENGTH_IN_CHAR = 1
CHARSET = 1
DB_TIMEZONE = +8:00
ENABLE_ENCRYPT = 1
ENCRYPT_NAME = AES256
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: dm8-primary
namespace: gmkx-platform
spec:
serviceName: dm8-primary
replicas: 1
selector:
matchLabels:
app: dm8-primary
template:
metadata:
labels:
app: dm8-primary
spec:
containers:
- name: dm8
image: dm8-server:v8.1.2.128
ports:
- containerPort: 5236
env:
- name: DM_USER
value: SYSDBA
- name: DM_PWD
valueFrom:
secretKeyRef:
name: dm8-secret
key: password
- name: INSTANCE_NAME
value: GMKX_PROD
volumeMounts:
- name: dm8-data
mountPath: /opt/dmdbms/data
- name: dm8-config
mountPath: /opt/dmdbms/bin/dm.ini
subPath: dm.ini
resources:
requests:
cpu: 2
memory: 8Gi
limits:
cpu: 8
memory: 32Gi
volumes:
- name: dm8-config
configMap:
name: dm8-config
volumeClaimTemplates:
- metadata:
name: dm8-data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: fast-ssd
resources:
requests:
storage: 500Gi
---
apiVersion: v1
kind: Service
metadata:
name: dm8-service
namespace: gmkx-platform
spec:
type: ClusterIP
ports:
- port: 5236
targetPort: 5236
selector:
app: dm8-primary
3.2 Redis 集群部署
3.2.1 Redis Cluster 配置
# k8s/redis-cluster.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-config
namespace: gmkx-platform
data:
redis.conf: |
# 网络配置
bind 0.0.0.0
port 6379
tcp-backlog 511
tcp-keepalive 300
# 内存配置
maxmemory 2gb
maxmemory-policy allkeys-lru
# 集群配置
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip-file announce-ip
cluster-announce-port 6379
cluster-announce-bus-port 16379
# 持久化配置
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
# 安全配置
requirepass redis123
masterauth redis123
# 日志配置
loglevel notice
logfile ""
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-cluster
namespace: gmkx-platform
spec:
serviceName: redis-cluster
replicas: 6
selector:
matchLabels:
app: redis-cluster
template:
metadata:
labels:
app: redis-cluster
spec:
initContainers:
- name: init-redis
image: redis:7-alpine
command:
- sh
- -c
- |
set -ex
# 获取Pod IP
POD_IP=$(hostname -i)
echo $POD_IP > /shared/announce-ip
volumeMounts:
- name: shared-data
mountPath: /shared
containers:
- name: redis
image: redis:7-alpine
command:
- redis-server
- /etc/redis/redis.conf
ports:
- containerPort: 6379
name: client
- containerPort: 16379
name: gossip
volumeMounts:
- name: redis-config
mountPath: /etc/redis
- name: shared-data
mountPath: /shared
- name: redis-data
mountPath: /data
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 1
memory: 4Gi
volumes:
- name: redis-config
configMap:
name: redis-config
- name: shared-data
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: redis-data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
name: redis-cluster
namespace: gmkx-platform
spec:
type: ClusterIP
clusterIP: None
ports:
- port: 6379
targetPort: 6379
name: client
- port: 16379
targetPort: 16379
name: gossip
selector:
app: redis-cluster
4. 监控与运维
4.1 Prometheus + Grafana 监控
4.1.1 Prometheus 配置
# k8s/monitoring/prometheus.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-config
namespace: gmkx-platform
data:
prometheus.yml: |
global:
scrape_interval: 15s
evaluation_interval: 15s
alerting:
alertmanagers:
- static_configs:
- targets:
- alertmanager:9093
rule_files:
- "/etc/prometheus/rules/*.yml"
scrape_configs:
# Kubernetes API Server
- job_name: 'kubernetes-apiservers'
kubernetes_sd_configs:
- role: endpoints
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
action: keep
regex: default;kubernetes;https
# Kubernetes Nodes
- job_name: 'kubernetes-nodes'
kubernetes_sd_configs:
- role: node
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
# GMKX 应用服务
- job_name: 'gmkx-services'
kubernetes_sd_configs:
- role: endpoints
namespaces:
names:
- gmkx-platform
relabel_configs:
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_service_name]
action: replace
target_label: kubernetes_name
# Redis 集群
- job_name: 'redis-cluster'
static_configs:
- targets: ['redis-cluster:6379']
# 数据库监控
- job_name: 'dm8-database'
static_configs:
- targets: ['dm8-service:5236']
# 告警规则
alert-rules.yml: |
groups:
- name: gmkx-platform-alerts
rules:
# 服务可用性告警
- alert: ServiceDown
expr: up == 0
for: 1m
labels:
severity: critical
annotations:
summary: "服务 {{ $labels.job }} 不可用"
description: "服务 {{ $labels.job }} 在 {{ $labels.instance }} 上已经停止运行超过1分钟"
# 高错误率告警
- alert: HighErrorRate
expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.1
for: 5m
labels:
severity: warning
annotations:
summary: "高错误率检测"
description: "服务 {{ $labels.job }} 错误率超过10%"
# 响应时间告警
- alert: HighResponseTime
expr: histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m])) > 0.5
for: 5m
labels:
severity: warning
annotations:
summary: "响应时间过长"
description: "服务 {{ $labels.job }} 95%分位响应时间超过500ms"
# 内存使用率告警
- alert: HighMemoryUsage
expr: (container_memory_usage_bytes / container_spec_memory_limit_bytes) > 0.9
for: 5m
labels:
severity: warning
annotations:
summary: "内存使用率过高"
description: "容器 {{ $labels.container }} 内存使用率超过90%"
# CPU使用率告警
- alert: HighCPUUsage
expr: rate(container_cpu_usage_seconds_total[5m]) > 0.8
for: 5m
labels:
severity: warning
annotations:
summary: "CPU使用率过高"
description: "容器 {{ $labels.container }} CPU使用率超过80%"
# 数据库连接数告警
- alert: DatabaseConnectionHigh
expr: mysql_global_status_threads_connected > 80
for: 2m
labels:
severity: warning
annotations:
summary: "数据库连接数过高"
description: "数据库连接数超过80"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus
namespace: gmkx-platform
spec:
replicas: 1
selector:
matchLabels:
app: prometheus
template:
metadata:
labels:
app: prometheus
spec:
serviceAccountName: prometheus
containers:
- name: prometheus
image: prom/prometheus:v2.47.0
args:
- --config.file=/etc/prometheus/prometheus.yml
- --storage.tsdb.path=/prometheus/
- --web.console.libraries=/etc/prometheus/console_libraries
- --web.console.templates=/etc/prometheus/consoles
- --storage.tsdb.retention.time=15d
- --web.enable-lifecycle
- --web.enable-admin-api
ports:
- containerPort: 9090
volumeMounts:
- name: prometheus-config
mountPath: /etc/prometheus
- name: prometheus-storage
mountPath: /prometheus
resources:
requests:
cpu: 500m
memory: 2Gi
limits:
cpu: 2
memory: 8Gi
volumes:
- name: prometheus-config
configMap:
name: prometheus-config
- name: prometheus-storage
persistentVolumeClaim:
claimName: prometheus-storage
4.1.2 Grafana 仪表盘
# k8s/monitoring/grafana.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-datasources
namespace: gmkx-platform
data:
datasources.yaml: |
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
access: proxy
url: http://prometheus:9090
isDefault: true
- name: Loki
type: loki
access: proxy
url: http://loki:3100
---
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-dashboards-config
namespace: gmkx-platform
data:
dashboards.yaml: |
apiVersion: 1
providers:
- name: 'gmkx-dashboards'
orgId: 1
folder: 'GMKX Platform'
type: file
disableDeletion: false
updateIntervalSeconds: 10
allowUiUpdates: true
options:
path: /var/lib/grafana/dashboards
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
namespace: gmkx-platform
spec:
replicas: 1
selector:
matchLabels:
app: grafana
template:
metadata:
labels:
app: grafana
spec:
containers:
- name: grafana
image: grafana/grafana:10.1.0
env:
- name: GF_SECURITY_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: grafana-secret
key: admin-password
- name: GF_USERS_ALLOW_SIGN_UP
value: "false"
ports:
- containerPort: 3000
volumeMounts:
- name: grafana-storage
mountPath: /var/lib/grafana
- name: grafana-datasources
mountPath: /etc/grafana/provisioning/datasources
- name: grafana-dashboards-config
mountPath: /etc/grafana/provisioning/dashboards
- name: grafana-dashboards
mountPath: /var/lib/grafana/dashboards
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 1
memory: 2Gi
volumes:
- name: grafana-storage
persistentVolumeClaim:
claimName: grafana-storage
- name: grafana-datasources
configMap:
name: grafana-datasources
- name: grafana-dashboards-config
configMap:
name: grafana-dashboards-config
- name: grafana-dashboards
configMap:
name: grafana-dashboards
4.2 日志管理
4.2.1 Loki + Promtail 日志收集
# k8s/logging/loki.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: loki-config
namespace: gmkx-platform
data:
loki.yaml: |
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
common:
path_prefix: /loki
storage:
filesystem:
chunks_directory: /loki/chunks
rules_directory: /loki/rules
replication_factor: 1
ring:
instance_addr: 127.0.0.1
kvstore:
store: inmemory
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
ruler:
alertmanager_url: http://alertmanager:9093
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h
ingestion_rate_mb: 16
ingestion_burst_size_mb: 32
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: loki
namespace: gmkx-platform
spec:
replicas: 1
selector:
matchLabels:
app: loki
template:
metadata:
labels:
app: loki
spec:
containers:
- name: loki
image: grafana/loki:2.9.0
args:
- -config.file=/etc/loki/loki.yaml
ports:
- containerPort: 3100
volumeMounts:
- name: loki-config
mountPath: /etc/loki
- name: loki-storage
mountPath: /loki
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 1
memory: 2Gi
volumes:
- name: loki-config
configMap:
name: loki-config
- name: loki-storage
persistentVolumeClaim:
claimName: loki-storage
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: promtail
namespace: gmkx-platform
spec:
selector:
matchLabels:
app: promtail
template:
metadata:
labels:
app: promtail
spec:
serviceAccount: promtail
containers:
- name: promtail
image: grafana/promtail:2.9.0
args:
- -config.file=/etc/promtail/config.yml
volumeMounts:
- name: config
mountPath: /etc/promtail
- name: varlog
mountPath: /var/log
readOnly: true
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
env:
- name: HOSTNAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
volumes:
- name: config
configMap:
name: promtail-config
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
4.3 链路追踪
4.3.1 Jaeger 部署
# k8s/tracing/jaeger.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jaeger
namespace: gmkx-platform
spec:
replicas: 1
selector:
matchLabels:
app: jaeger
template:
metadata:
labels:
app: jaeger
spec:
containers:
- name: jaeger
image: jaegertracing/all-in-one:1.50
env:
- name: COLLECTOR_OTLP_ENABLED
value: "true"
- name: SPAN_STORAGE_TYPE
value: "memory"
ports:
- containerPort: 16686
protocol: TCP
- containerPort: 14250
protocol: TCP
- containerPort: 14268
protocol: TCP
- containerPort: 4317
protocol: TCP
- containerPort: 4318
protocol: TCP
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
---
apiVersion: v1
kind: Service
metadata:
name: jaeger
namespace: gmkx-platform
spec:
type: ClusterIP
ports:
- name: http-query
port: 16686
targetPort: 16686
- name: grpc
port: 14250
targetPort: 14250
- name: http-collector
port: 14268
targetPort: 14268
- name: otlp-grpc
port: 4317
targetPort: 4317
- name: otlp-http
port: 4318
targetPort: 4318
selector:
app: jaeger
5. CI/CD 自动化
5.1 GitLab CI 流水线
5.1.1 构建和部署流水线
# .gitlab-ci.yml
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
REGISTRY: harbor.gmsc.gov.cn
PROJECT_NAME: gmkx
stages:
- test
- build
- security-scan
- deploy-test
- deploy-staging
- deploy-production
# 测试阶段
test:unit:
stage: test
image: openjdk:17-jdk
services:
- postgres:15-alpine
variables:
POSTGRES_DB: gmkx_test
POSTGRES_USER: gmkx
POSTGRES_PASSWORD: gmkx123
SPRING_PROFILES_ACTIVE: test
script:
- ./mvnw test jacoco:report
coverage: '/Total.*?([0-9]{1,3})%/'
artifacts:
reports:
junit: target/surefire-reports/TEST-*.xml
coverage_report:
coverage_format: jacoco
path: target/site/jacoco/jacoco.xml
only:
- merge_requests
- develop
- main
test:frontend:
stage: test
image: node:18-alpine
script:
- cd gmkx-web
- npm ci
- npm run test:coverage
- npm run lint
- npm run type-check
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: gmkx-web/coverage/cobertura-coverage.xml
only:
- merge_requests
- develop
- main
# 构建阶段
build:backend:
stage: build
image: docker:24-dind
services:
- docker:24-dind
before_script:
- docker login -u $HARBOR_USERNAME -p $HARBOR_PASSWORD $REGISTRY
script:
- |
for service in user-service achievement-service demand-service gateway; do
echo "Building $service..."
docker build -t $REGISTRY/$PROJECT_NAME/$service:$CI_COMMIT_SHA ./gmkx-$service
docker build -t $REGISTRY/$PROJECT_NAME/$service:latest ./gmkx-$service
docker push $REGISTRY/$PROJECT_NAME/$service:$CI_COMMIT_SHA
docker push $REGISTRY/$PROJECT_NAME/$service:latest
done
only:
- develop
- main
build:frontend:
stage: build
image: docker:24-dind
services:
- docker:24-dind
before_script:
- docker login -u $HARBOR_USERNAME -p $HARBOR_PASSWORD $REGISTRY
script:
- docker build -t $REGISTRY/$PROJECT_NAME/web-app:$CI_COMMIT_SHA ./gmkx-web
- docker build -t $REGISTRY/$PROJECT_NAME/web-app:latest ./gmkx-web
- docker push $REGISTRY/$PROJECT_NAME/web-app:$CI_COMMIT_SHA
- docker push $REGISTRY/$PROJECT_NAME/web-app:latest
only:
- develop
- main
build:ai-service:
stage: build
image: docker:24-dind
services:
- docker:24-dind
before_script:
- docker login -u $HARBOR_USERNAME -p $HARBOR_PASSWORD $REGISTRY
script:
- docker build -t $REGISTRY/$PROJECT_NAME/ai-service:$CI_COMMIT_SHA ./gmkx-ai
- docker build -t $REGISTRY/$PROJECT_NAME/ai-service:latest ./gmkx-ai
- docker push $REGISTRY/$PROJECT_NAME/ai-service:$CI_COMMIT_SHA
- docker push $REGISTRY/$PROJECT_NAME/ai-service:latest
only:
- develop
- main
# 安全扫描
security:container-scan:
stage: security-scan
image: aquasec/trivy:latest
script:
- |
for service in user-service achievement-service demand-service gateway web-app ai-service; do
echo "Scanning $service..."
trivy image --exit-code 0 --severity HIGH,CRITICAL --format json -o $service-scan.json $REGISTRY/$PROJECT_NAME/$service:$CI_COMMIT_SHA
done
artifacts:
reports:
container_scanning: "*-scan.json"
only:
- develop
- main
security:dependency-scan:
stage: security-scan
image: owasp/dependency-check:latest
script:
- /usr/share/dependency-check/bin/dependency-check.sh --project $PROJECT_NAME --scan . --format JSON --out dependency-check-report.json
artifacts:
reports:
dependency_scanning: dependency-check-report.json
only:
- develop
- main
# 部署测试环境
deploy:test:
stage: deploy-test
image: bitnami/kubectl:latest
script:
- kubectl config use-context test-cluster
- |
for service in user-service achievement-service demand-service gateway web-app ai-service; do
helm upgrade --install gmkx-$service ./helm/gmkx-service \
--namespace gmkx-test \
--set image.repository=$REGISTRY/$PROJECT_NAME/$service \
--set image.tag=$CI_COMMIT_SHA \
--set environment=test \
--wait --timeout=5m
done
environment:
name: test
url: https://test.platform.gmsc.gov.cn
only:
- develop
# 部署预发布环境
deploy:staging:
stage: deploy-staging
image: bitnami/kubectl:latest
script:
- kubectl config use-context staging-cluster
- |
for service in user-service achievement-service demand-service gateway web-app ai-service; do
helm upgrade --install gmkx-$service ./helm/gmkx-service \
--namespace gmkx-staging \
--set image.repository=$REGISTRY/$PROJECT_NAME/$service \
--set image.tag=$CI_COMMIT_SHA \
--set environment=staging \
--set replicaCount=2 \
--wait --timeout=10m
done
environment:
name: staging
url: https://staging.platform.gmsc.gov.cn
when: manual
only:
- main
# 部署生产环境
deploy:production:
stage: deploy-production
image: bitnami/kubectl:latest
script:
- kubectl config use-context production-cluster
- |
for service in user-service achievement-service demand-service gateway web-app ai-service; do
# 金丝雀部署
helm upgrade --install gmkx-$service-canary ./helm/gmkx-service \
--namespace gmkx-platform \
--set image.repository=$REGISTRY/$PROJECT_NAME/$service \
--set image.tag=$CI_COMMIT_SHA \
--set environment=production \
--set canary.enabled=true \
--set canary.weight=10 \
--wait --timeout=15m
# 验证金丝雀部署
./scripts/validate-canary.sh gmkx-$service
# 完整部署
helm upgrade --install gmkx-$service ./helm/gmkx-service \
--namespace gmkx-platform \
--set image.repository=$REGISTRY/$PROJECT_NAME/$service \
--set image.tag=$CI_COMMIT_SHA \
--set environment=production \
--set replicaCount=3 \
--wait --timeout=15m
done
environment:
name: production
url: https://platform.gmsc.gov.cn
when: manual
only:
- main
5.2 Argo CD GitOps
5.2.1 Application 配置
# argocd/applications/gmkx-platform.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gmkx-platform
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: https://gitlab.gmsc.gov.cn/gmkx/platform-manifests.git
targetRevision: HEAD
path: manifests/production
destination:
server: https://kubernetes.default.svc
namespace: gmkx-platform
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- PrunePropagationPolicy=foreground
- PruneLast=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
revisionHistoryLimit: 3
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gmkx-monitoring
namespace: argocd
spec:
project: default
source:
repoURL: https://gitlab.gmsc.gov.cn/gmkx/platform-manifests.git
targetRevision: HEAD
path: manifests/monitoring
destination:
server: https://kubernetes.default.svc
namespace: gmkx-monitoring
syncPolicy:
automated:
prune: true
selfHeal: true
5.3 Helm Charts
5.3.1 通用服务 Chart
# helm/gmkx-service/Chart.yaml
apiVersion: v2
name: gmkx-service
description: GMKX Platform Service Helm Chart
type: application
version: 1.0.0
appVersion: "1.0.0"
# helm/gmkx-service/values.yaml
# 默认值配置
replicaCount: 3
image:
repository: harbor.gmsc.gov.cn/gmkx/service
pullPolicy: IfNotPresent
tag: ""
imagePullSecrets:
- name: harbor-registry-secret
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
annotations: {}
name: ""
podAnnotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9090"
prometheus.io/path: "/actuator/prometheus"
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
service:
type: ClusterIP
port: 8080
targetPort: 8080
ingress:
enabled: false
className: "nginx"
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
hosts:
- host: chart-example.local
paths:
- path: /
pathType: Prefix
tls: []
resources:
limits:
cpu: 2000m
memory: 4Gi
requests:
cpu: 500m
memory: 1Gi
autoscaling:
enabled: true
minReplicas: 3
maxReplicas: 10
targetCPUUtilizationPercentage: 70
targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- "{{ include "gmkx-service.name" . }}"
topologyKey: kubernetes.io/hostname
# 环境特定配置
environment: production
config:
springProfiles: production
logLevel: INFO
javaOpts: "-Xms1g -Xmx2g -XX:+UseG1GC"
# 健康检查配置
healthCheck:
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 60
periodSeconds: 30
timeoutSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# 金丝雀部署配置
canary:
enabled: false
weight: 10
maxWeight: 50
stepWeight: 10
interval: 30s
这个部署方案提供了完整的容器化部署、监控运维和自动化流水线,确保系统的稳定性和可维护性。