ccgx-workflow 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +22 -0
- package/README.md +469 -0
- package/README.zh-CN.md +466 -0
- package/bin/ccg.mjs +2 -0
- package/dist/cli.d.mts +1 -0
- package/dist/cli.d.ts +1 -0
- package/dist/cli.mjs +173 -0
- package/dist/index.d.mts +1774 -0
- package/dist/index.d.ts +1774 -0
- package/dist/index.mjs +2029 -0
- package/dist/shared/ccgx-workflow.WgUzkiC3.mjs +5248 -0
- package/package.json +129 -0
- package/templates/commands/agents/assumptions-analyzer.md +129 -0
- package/templates/commands/agents/code-fixer.md +292 -0
- package/templates/commands/agents/codebase-mapper.md +152 -0
- package/templates/commands/agents/debug-session-manager.md +247 -0
- package/templates/commands/agents/debugger.md +111 -0
- package/templates/commands/agents/eval-auditor.md +171 -0
- package/templates/commands/agents/framework-selector.md +152 -0
- package/templates/commands/agents/get-current-datetime.md +29 -0
- package/templates/commands/agents/init-architect.md +114 -0
- package/templates/commands/agents/integration-checker.md +163 -0
- package/templates/commands/agents/interface-auditor.md +170 -0
- package/templates/commands/agents/nyquist-auditor.md +131 -0
- package/templates/commands/agents/pattern-mapper.md +111 -0
- package/templates/commands/agents/phase-runner.md +321 -0
- package/templates/commands/agents/plan-checker.md +255 -0
- package/templates/commands/agents/planner.md +320 -0
- package/templates/commands/agents/team-architect.md +186 -0
- package/templates/commands/agents/team-qa.md +121 -0
- package/templates/commands/agents/team-reviewer.md +157 -0
- package/templates/commands/agents/ui-ux-designer.md +573 -0
- package/templates/commands/agents/verifier.md +274 -0
- package/templates/commands/analyze.md +210 -0
- package/templates/commands/autonomous.md +792 -0
- package/templates/commands/cancel.md +132 -0
- package/templates/commands/clean-branches.md +117 -0
- package/templates/commands/codex-exec.md +404 -0
- package/templates/commands/commit.md +151 -0
- package/templates/commands/context.md +332 -0
- package/templates/commands/debate.md +165 -0
- package/templates/commands/debug.md +226 -0
- package/templates/commands/enhance.md +64 -0
- package/templates/commands/execute.md +380 -0
- package/templates/commands/init.md +123 -0
- package/templates/commands/optimize.md +217 -0
- package/templates/commands/plan.md +373 -0
- package/templates/commands/result.md +106 -0
- package/templates/commands/review.md +338 -0
- package/templates/commands/rollback.md +116 -0
- package/templates/commands/spec-impl.md +139 -0
- package/templates/commands/spec-init.md +101 -0
- package/templates/commands/spec-plan.md +210 -0
- package/templates/commands/spec-research.md +152 -0
- package/templates/commands/spec-review.md +120 -0
- package/templates/commands/status.md +206 -0
- package/templates/commands/team-exec.md +265 -0
- package/templates/commands/test.md +236 -0
- package/templates/commands/verify-work.md +338 -0
- package/templates/commands/verify.md +66 -0
- package/templates/commands/workflow.md +190 -0
- package/templates/commands/worktree.md +128 -0
- package/templates/hooks/ccg-context-monitor.js +159 -0
- package/templates/hooks/ccg-session-state.cjs +510 -0
- package/templates/hooks/ccg-statusline.js +142 -0
- package/templates/output-styles/abyss-command.md +56 -0
- package/templates/output-styles/abyss-concise.md +89 -0
- package/templates/output-styles/abyss-cultivator.md +302 -0
- package/templates/output-styles/abyss-ritual.md +70 -0
- package/templates/output-styles/engineer-professional.md +89 -0
- package/templates/output-styles/laowang-engineer.md +127 -0
- package/templates/output-styles/nekomata-engineer.md +120 -0
- package/templates/output-styles/ojousama-engineer.md +121 -0
- package/templates/prompts/claude/analyzer.md +59 -0
- package/templates/prompts/claude/architect.md +54 -0
- package/templates/prompts/claude/debugger.md +71 -0
- package/templates/prompts/claude/optimizer.md +73 -0
- package/templates/prompts/claude/reviewer.md +63 -0
- package/templates/prompts/claude/tester.md +69 -0
- package/templates/prompts/codex/analyzer.md +58 -0
- package/templates/prompts/codex/architect.md +54 -0
- package/templates/prompts/codex/debugger.md +74 -0
- package/templates/prompts/codex/optimizer.md +81 -0
- package/templates/prompts/codex/reviewer.md +73 -0
- package/templates/prompts/codex/tester.md +62 -0
- package/templates/prompts/gemini/analyzer.md +61 -0
- package/templates/prompts/gemini/architect.md +55 -0
- package/templates/prompts/gemini/debugger.md +78 -0
- package/templates/prompts/gemini/frontend.md +64 -0
- package/templates/prompts/gemini/optimizer.md +84 -0
- package/templates/prompts/gemini/reviewer.md +80 -0
- package/templates/prompts/gemini/tester.md +68 -0
- package/templates/rules/ccg-skill-routing.md +83 -0
- package/templates/rules/ccg-skills.md +71 -0
- package/templates/scripts/ccg-phase-runner-launcher.mjs +467 -0
- package/templates/scripts/invoke-model.mjs +949 -0
- package/templates/scripts/repatch-gemini-plugin.mjs +194 -0
- package/templates/skills/SKILL.md +92 -0
- package/templates/skills/domains/ai/SKILL.md +35 -0
- package/templates/skills/domains/ai/agent-dev.md +242 -0
- package/templates/skills/domains/ai/llm-security.md +288 -0
- package/templates/skills/domains/ai/prompt-and-eval.md +279 -0
- package/templates/skills/domains/ai/rag-system.md +542 -0
- package/templates/skills/domains/architecture/SKILL.md +43 -0
- package/templates/skills/domains/architecture/api-design.md +225 -0
- package/templates/skills/domains/architecture/caching.md +299 -0
- package/templates/skills/domains/architecture/cloud-native.md +285 -0
- package/templates/skills/domains/architecture/message-queue.md +329 -0
- package/templates/skills/domains/architecture/security-arch.md +297 -0
- package/templates/skills/domains/data-engineering/SKILL.md +208 -0
- package/templates/skills/domains/development/SKILL.md +47 -0
- package/templates/skills/domains/development/cpp.md +246 -0
- package/templates/skills/domains/development/go.md +323 -0
- package/templates/skills/domains/development/java.md +277 -0
- package/templates/skills/domains/development/python.md +288 -0
- package/templates/skills/domains/development/rust.md +313 -0
- package/templates/skills/domains/development/shell.md +313 -0
- package/templates/skills/domains/development/typescript.md +277 -0
- package/templates/skills/domains/devops/SKILL.md +40 -0
- package/templates/skills/domains/devops/cost-optimization.md +272 -0
- package/templates/skills/domains/devops/database.md +217 -0
- package/templates/skills/domains/devops/devsecops.md +198 -0
- package/templates/skills/domains/devops/git-workflow.md +181 -0
- package/templates/skills/domains/devops/observability.md +280 -0
- package/templates/skills/domains/devops/performance.md +336 -0
- package/templates/skills/domains/devops/testing.md +283 -0
- package/templates/skills/domains/frontend-design/SKILL.md +244 -0
- package/templates/skills/domains/frontend-design/agents/openai.yaml +4 -0
- package/templates/skills/domains/frontend-design/claymorphism/SKILL.md +121 -0
- package/templates/skills/domains/frontend-design/claymorphism/references/tokens.css +52 -0
- package/templates/skills/domains/frontend-design/component-patterns.md +202 -0
- package/templates/skills/domains/frontend-design/engineering.md +287 -0
- package/templates/skills/domains/frontend-design/glassmorphism/SKILL.md +142 -0
- package/templates/skills/domains/frontend-design/glassmorphism/references/tokens.css +32 -0
- package/templates/skills/domains/frontend-design/liquid-glass/SKILL.md +139 -0
- package/templates/skills/domains/frontend-design/liquid-glass/references/tokens.css +81 -0
- package/templates/skills/domains/frontend-design/neubrutalism/SKILL.md +145 -0
- package/templates/skills/domains/frontend-design/neubrutalism/references/tokens.css +44 -0
- package/templates/skills/domains/frontend-design/reference/color-and-contrast.md +132 -0
- package/templates/skills/domains/frontend-design/reference/interaction-design.md +195 -0
- package/templates/skills/domains/frontend-design/reference/motion-design.md +99 -0
- package/templates/skills/domains/frontend-design/reference/responsive-design.md +114 -0
- package/templates/skills/domains/frontend-design/reference/spatial-design.md +100 -0
- package/templates/skills/domains/frontend-design/reference/typography.md +133 -0
- package/templates/skills/domains/frontend-design/reference/ux-writing.md +107 -0
- package/templates/skills/domains/frontend-design/state-management.md +680 -0
- package/templates/skills/domains/frontend-design/ui-aesthetics.md +110 -0
- package/templates/skills/domains/frontend-design/ux-principles.md +156 -0
- package/templates/skills/domains/infrastructure/SKILL.md +201 -0
- package/templates/skills/domains/mobile/SKILL.md +225 -0
- package/templates/skills/domains/orchestration/SKILL.md +30 -0
- package/templates/skills/domains/orchestration/multi-agent.md +263 -0
- package/templates/skills/domains/security/SKILL.md +73 -0
- package/templates/skills/domains/security/blue-team.md +436 -0
- package/templates/skills/domains/security/code-audit.md +265 -0
- package/templates/skills/domains/security/pentest.md +226 -0
- package/templates/skills/domains/security/red-team.md +374 -0
- package/templates/skills/domains/security/threat-intel.md +372 -0
- package/templates/skills/domains/security/vuln-research.md +369 -0
- package/templates/skills/impeccable/adapt/SKILL.md +201 -0
- package/templates/skills/impeccable/animate/SKILL.md +176 -0
- package/templates/skills/impeccable/arrange/SKILL.md +126 -0
- package/templates/skills/impeccable/audit/SKILL.md +149 -0
- package/templates/skills/impeccable/bolder/SKILL.md +118 -0
- package/templates/skills/impeccable/clarify/SKILL.md +185 -0
- package/templates/skills/impeccable/colorize/SKILL.md +144 -0
- package/templates/skills/impeccable/critique/SKILL.md +203 -0
- package/templates/skills/impeccable/critique/reference/cognitive-load.md +106 -0
- package/templates/skills/impeccable/critique/reference/heuristics-scoring.md +234 -0
- package/templates/skills/impeccable/critique/reference/personas.md +178 -0
- package/templates/skills/impeccable/delight/SKILL.md +305 -0
- package/templates/skills/impeccable/distill/SKILL.md +123 -0
- package/templates/skills/impeccable/extract/SKILL.md +94 -0
- package/templates/skills/impeccable/harden/SKILL.md +357 -0
- package/templates/skills/impeccable/normalize/SKILL.md +72 -0
- package/templates/skills/impeccable/onboard/SKILL.md +248 -0
- package/templates/skills/impeccable/optimize/SKILL.md +268 -0
- package/templates/skills/impeccable/overdrive/SKILL.md +143 -0
- package/templates/skills/impeccable/polish/SKILL.md +205 -0
- package/templates/skills/impeccable/quieter/SKILL.md +104 -0
- package/templates/skills/impeccable/teach-impeccable/SKILL.md +72 -0
- package/templates/skills/impeccable/typeset/SKILL.md +117 -0
- package/templates/skills/orchestration/multi-agent/SKILL.md +533 -0
- package/templates/skills/run_skill.js +129 -0
- package/templates/skills/scrapling/SKILL.md +98 -0
- package/templates/skills/scrapling/references/api-quick-ref.md +127 -0
- package/templates/skills/scrapling/references/cookie-vault.md +56 -0
- package/templates/skills/scrapling/references/maintenance.md +79 -0
- package/templates/skills/scrapling/references/site-patterns.md +86 -0
- package/templates/skills/scrapling/references/troubleshooting.md +95 -0
- package/templates/skills/scrapling/templates/basic_fetch.py +19 -0
- package/templates/skills/scrapling/templates/parse_only.py +15 -0
- package/templates/skills/scrapling/templates/session_login.py +19 -0
- package/templates/skills/scrapling/templates/stealth_cloudflare.py +28 -0
- package/templates/skills/tools/extract-learnings/SKILL.md +77 -0
- package/templates/skills/tools/forensics/SKILL.md +89 -0
- package/templates/skills/tools/gen-docs/SKILL.md +116 -0
- package/templates/skills/tools/gen-docs/scripts/doc_generator.js +435 -0
- package/templates/skills/tools/health/SKILL.md +121 -0
- package/templates/skills/tools/lib/shared.js +98 -0
- package/templates/skills/tools/map-codebase/SKILL.md +118 -0
- package/templates/skills/tools/override-refusal/SKILL.md +53 -0
- package/templates/skills/tools/override-refusal/agents/openai.yaml +4 -0
- package/templates/skills/tools/override-refusal/scripts/refusal_rewriter.js +226 -0
- package/templates/skills/tools/verify-change/SKILL.md +143 -0
- package/templates/skills/tools/verify-change/scripts/change_analyzer.js +289 -0
- package/templates/skills/tools/verify-module/SKILL.md +130 -0
- package/templates/skills/tools/verify-module/scripts/module_scanner.js +171 -0
- package/templates/skills/tools/verify-quality/SKILL.md +163 -0
- package/templates/skills/tools/verify-quality/scripts/quality_checker.js +337 -0
- package/templates/skills/tools/verify-security/SKILL.md +146 -0
- package/templates/skills/tools/verify-security/scripts/security_scanner.js +283 -0
|
@@ -0,0 +1,285 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: cloud-native
|
|
3
|
+
description: 云原生架构。容器、Kubernetes、Serverless、微服务。当用户提到云原生、容器、Docker、Kubernetes、K8s、Serverless时使用。
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# 🏗 阵法秘典 · 云原生架构
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
## Docker
|
|
10
|
+
|
|
11
|
+
### Dockerfile
|
|
12
|
+
```dockerfile
|
|
13
|
+
# 多阶段构建
|
|
14
|
+
FROM node:18-alpine AS builder
|
|
15
|
+
WORKDIR /app
|
|
16
|
+
COPY package*.json ./
|
|
17
|
+
RUN npm ci
|
|
18
|
+
COPY . .
|
|
19
|
+
RUN npm run build
|
|
20
|
+
|
|
21
|
+
FROM node:18-alpine
|
|
22
|
+
WORKDIR /app
|
|
23
|
+
COPY --from=builder /app/dist ./dist
|
|
24
|
+
COPY --from=builder /app/node_modules ./node_modules
|
|
25
|
+
EXPOSE 3000
|
|
26
|
+
USER node
|
|
27
|
+
CMD ["node", "dist/main.js"]
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
### Docker Compose
|
|
31
|
+
```yaml
|
|
32
|
+
version: '3.8'
|
|
33
|
+
services:
|
|
34
|
+
app:
|
|
35
|
+
build: .
|
|
36
|
+
ports:
|
|
37
|
+
- "3000:3000"
|
|
38
|
+
environment:
|
|
39
|
+
- DATABASE_URL=postgres://db:5432/mydb
|
|
40
|
+
depends_on:
|
|
41
|
+
- db
|
|
42
|
+
healthcheck:
|
|
43
|
+
test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
|
|
44
|
+
interval: 30s
|
|
45
|
+
timeout: 10s
|
|
46
|
+
retries: 3
|
|
47
|
+
|
|
48
|
+
db:
|
|
49
|
+
image: postgres:15-alpine
|
|
50
|
+
volumes:
|
|
51
|
+
- postgres_data:/var/lib/postgresql/data
|
|
52
|
+
environment:
|
|
53
|
+
POSTGRES_DB: mydb
|
|
54
|
+
POSTGRES_PASSWORD: ${DB_PASSWORD}
|
|
55
|
+
|
|
56
|
+
volumes:
|
|
57
|
+
postgres_data:
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
### 安全最佳实践
|
|
61
|
+
```yaml
|
|
62
|
+
镜像安全:
|
|
63
|
+
- 使用官方基础镜像
|
|
64
|
+
- 最小化镜像 (alpine/distroless)
|
|
65
|
+
- 扫描漏洞 (Trivy)
|
|
66
|
+
- 固定版本标签
|
|
67
|
+
|
|
68
|
+
运行时安全:
|
|
69
|
+
- 非 root 用户运行
|
|
70
|
+
- 只读文件系统
|
|
71
|
+
- 限制资源
|
|
72
|
+
- 禁用特权模式
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Kubernetes
|
|
76
|
+
|
|
77
|
+
### 基础资源
|
|
78
|
+
```yaml
|
|
79
|
+
# Deployment
|
|
80
|
+
apiVersion: apps/v1
|
|
81
|
+
kind: Deployment
|
|
82
|
+
metadata:
|
|
83
|
+
name: myapp
|
|
84
|
+
spec:
|
|
85
|
+
replicas: 3
|
|
86
|
+
selector:
|
|
87
|
+
matchLabels:
|
|
88
|
+
app: myapp
|
|
89
|
+
template:
|
|
90
|
+
metadata:
|
|
91
|
+
labels:
|
|
92
|
+
app: myapp
|
|
93
|
+
spec:
|
|
94
|
+
containers:
|
|
95
|
+
- name: myapp
|
|
96
|
+
image: myapp:1.0.0
|
|
97
|
+
ports:
|
|
98
|
+
- containerPort: 3000
|
|
99
|
+
resources:
|
|
100
|
+
requests:
|
|
101
|
+
memory: "128Mi"
|
|
102
|
+
cpu: "100m"
|
|
103
|
+
limits:
|
|
104
|
+
memory: "256Mi"
|
|
105
|
+
cpu: "200m"
|
|
106
|
+
livenessProbe:
|
|
107
|
+
httpGet:
|
|
108
|
+
path: /health
|
|
109
|
+
port: 3000
|
|
110
|
+
initialDelaySeconds: 10
|
|
111
|
+
periodSeconds: 10
|
|
112
|
+
readinessProbe:
|
|
113
|
+
httpGet:
|
|
114
|
+
path: /ready
|
|
115
|
+
port: 3000
|
|
116
|
+
initialDelaySeconds: 5
|
|
117
|
+
periodSeconds: 5
|
|
118
|
+
---
|
|
119
|
+
# Service
|
|
120
|
+
apiVersion: v1
|
|
121
|
+
kind: Service
|
|
122
|
+
metadata:
|
|
123
|
+
name: myapp
|
|
124
|
+
spec:
|
|
125
|
+
selector:
|
|
126
|
+
app: myapp
|
|
127
|
+
ports:
|
|
128
|
+
- port: 80
|
|
129
|
+
targetPort: 3000
|
|
130
|
+
type: ClusterIP
|
|
131
|
+
---
|
|
132
|
+
# Ingress
|
|
133
|
+
apiVersion: networking.k8s.io/v1
|
|
134
|
+
kind: Ingress
|
|
135
|
+
metadata:
|
|
136
|
+
name: myapp
|
|
137
|
+
annotations:
|
|
138
|
+
nginx.ingress.kubernetes.io/ssl-redirect: "true"
|
|
139
|
+
spec:
|
|
140
|
+
tls:
|
|
141
|
+
- hosts:
|
|
142
|
+
- myapp.example.com
|
|
143
|
+
secretName: myapp-tls
|
|
144
|
+
rules:
|
|
145
|
+
- host: myapp.example.com
|
|
146
|
+
http:
|
|
147
|
+
paths:
|
|
148
|
+
- path: /
|
|
149
|
+
pathType: Prefix
|
|
150
|
+
backend:
|
|
151
|
+
service:
|
|
152
|
+
name: myapp
|
|
153
|
+
port:
|
|
154
|
+
number: 80
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
### 配置管理
|
|
158
|
+
```yaml
|
|
159
|
+
# ConfigMap
|
|
160
|
+
apiVersion: v1
|
|
161
|
+
kind: ConfigMap
|
|
162
|
+
metadata:
|
|
163
|
+
name: myapp-config
|
|
164
|
+
data:
|
|
165
|
+
APP_ENV: production
|
|
166
|
+
LOG_LEVEL: info
|
|
167
|
+
---
|
|
168
|
+
# Secret
|
|
169
|
+
apiVersion: v1
|
|
170
|
+
kind: Secret
|
|
171
|
+
metadata:
|
|
172
|
+
name: myapp-secret
|
|
173
|
+
type: Opaque
|
|
174
|
+
stringData:
|
|
175
|
+
DATABASE_URL: postgres://user:pass@db:5432/mydb
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
### 安全策略
|
|
179
|
+
```yaml
|
|
180
|
+
# NetworkPolicy
|
|
181
|
+
apiVersion: networking.k8s.io/v1
|
|
182
|
+
kind: NetworkPolicy
|
|
183
|
+
metadata:
|
|
184
|
+
name: myapp-network-policy
|
|
185
|
+
spec:
|
|
186
|
+
podSelector:
|
|
187
|
+
matchLabels:
|
|
188
|
+
app: myapp
|
|
189
|
+
policyTypes:
|
|
190
|
+
- Ingress
|
|
191
|
+
- Egress
|
|
192
|
+
ingress:
|
|
193
|
+
- from:
|
|
194
|
+
- podSelector:
|
|
195
|
+
matchLabels:
|
|
196
|
+
app: frontend
|
|
197
|
+
ports:
|
|
198
|
+
- port: 3000
|
|
199
|
+
egress:
|
|
200
|
+
- to:
|
|
201
|
+
- podSelector:
|
|
202
|
+
matchLabels:
|
|
203
|
+
app: database
|
|
204
|
+
ports:
|
|
205
|
+
- port: 5432
|
|
206
|
+
---
|
|
207
|
+
# PodSecurityPolicy (已废弃,使用 Pod Security Standards)
|
|
208
|
+
# Pod Security Admission
|
|
209
|
+
apiVersion: v1
|
|
210
|
+
kind: Namespace
|
|
211
|
+
metadata:
|
|
212
|
+
name: myapp
|
|
213
|
+
labels:
|
|
214
|
+
pod-security.kubernetes.io/enforce: restricted
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
## Serverless
|
|
218
|
+
|
|
219
|
+
### AWS Lambda
|
|
220
|
+
```python
|
|
221
|
+
import json
|
|
222
|
+
|
|
223
|
+
def handler(event, context):
|
|
224
|
+
body = json.loads(event.get('body', '{}'))
|
|
225
|
+
|
|
226
|
+
return {
|
|
227
|
+
'statusCode': 200,
|
|
228
|
+
'headers': {'Content-Type': 'application/json'},
|
|
229
|
+
'body': json.dumps({'message': 'Hello!'})
|
|
230
|
+
}
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
### Serverless Framework
|
|
234
|
+
```yaml
|
|
235
|
+
service: myapp
|
|
236
|
+
|
|
237
|
+
provider:
|
|
238
|
+
name: aws
|
|
239
|
+
runtime: python3.9
|
|
240
|
+
region: us-east-1
|
|
241
|
+
environment:
|
|
242
|
+
TABLE_NAME: ${self:service}-${sls:stage}
|
|
243
|
+
|
|
244
|
+
functions:
|
|
245
|
+
hello:
|
|
246
|
+
handler: handler.hello
|
|
247
|
+
events:
|
|
248
|
+
- http:
|
|
249
|
+
path: /hello
|
|
250
|
+
method: get
|
|
251
|
+
|
|
252
|
+
process:
|
|
253
|
+
handler: handler.process
|
|
254
|
+
events:
|
|
255
|
+
- sqs:
|
|
256
|
+
arn: !GetAtt MyQueue.Arn
|
|
257
|
+
|
|
258
|
+
resources:
|
|
259
|
+
Resources:
|
|
260
|
+
MyQueue:
|
|
261
|
+
Type: AWS::SQS::Queue
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
## 微服务模式
|
|
265
|
+
|
|
266
|
+
```yaml
|
|
267
|
+
服务发现:
|
|
268
|
+
- DNS (Kubernetes Service)
|
|
269
|
+
- Service Mesh (Istio)
|
|
270
|
+
|
|
271
|
+
负载均衡:
|
|
272
|
+
- 客户端负载均衡
|
|
273
|
+
- 服务端负载均衡
|
|
274
|
+
|
|
275
|
+
熔断器:
|
|
276
|
+
- Circuit Breaker
|
|
277
|
+
- Retry with backoff
|
|
278
|
+
- Timeout
|
|
279
|
+
|
|
280
|
+
可观测性:
|
|
281
|
+
- 日志聚合 (ELK)
|
|
282
|
+
- 指标监控 (Prometheus)
|
|
283
|
+
- 分布式追踪 (Jaeger)
|
|
284
|
+
```
|
|
285
|
+
|
|
@@ -0,0 +1,329 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: message-queue
|
|
3
|
+
description: 消息队列秘典。Kafka、RabbitMQ、Redis Streams、事件驱动架构。当用户提到消息队列、Kafka、RabbitMQ、事件驱动、CQRS、Saga时路由到此。
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# 🏗 阵法秘典 · 消息队列
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
## 核心概念
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
Producer → Broker → Consumer
|
|
13
|
+
发送 存储 消费
|
|
14
|
+
|
|
15
|
+
模式:
|
|
16
|
+
点对点 (Queue): 1 Producer → 1 Consumer
|
|
17
|
+
发布订阅 (Topic): 1 Producer → N Consumers
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
| 概念 | 含义 | 类比 |
|
|
21
|
+
|------|------|------|
|
|
22
|
+
| Producer | 消息生产者 | 发令者 |
|
|
23
|
+
| Consumer | 消息消费者 | 执行者 |
|
|
24
|
+
| Broker | 消息中间件 | 传令阵 |
|
|
25
|
+
| Topic/Queue | 消息通道 | 传音符 |
|
|
26
|
+
| Partition | 分区(并行单元) | 阵眼 |
|
|
27
|
+
| Offset | 消费位置 | 修行进度 |
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## Kafka
|
|
32
|
+
|
|
33
|
+
### 架构
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
Producer ──→ Broker Cluster ──→ Consumer Group
|
|
37
|
+
│
|
|
38
|
+
┌────┴────┐
|
|
39
|
+
│ Topic-A │
|
|
40
|
+
│ P0 P1 P2 │ (3 Partitions)
|
|
41
|
+
└──────────┘
|
|
42
|
+
|
|
43
|
+
Replication: Leader + Followers
|
|
44
|
+
ZooKeeper/KRaft: 元数据管理
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### 生产者
|
|
48
|
+
|
|
49
|
+
```python
|
|
50
|
+
from confluent_kafka import Producer
|
|
51
|
+
|
|
52
|
+
conf = {
|
|
53
|
+
'bootstrap.servers': 'kafka:9092',
|
|
54
|
+
'acks': 'all', # 等待所有副本确认
|
|
55
|
+
'retries': 3,
|
|
56
|
+
'retry.backoff.ms': 1000,
|
|
57
|
+
'enable.idempotence': True, # 幂等生产者
|
|
58
|
+
'linger.ms': 5, # 批量发送延迟
|
|
59
|
+
'batch.size': 16384, # 批量大小
|
|
60
|
+
'compression.type': 'snappy', # 压缩
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
producer = Producer(conf)
|
|
64
|
+
|
|
65
|
+
def delivery_report(err, msg):
|
|
66
|
+
if err:
|
|
67
|
+
print(f"Delivery failed: {err}")
|
|
68
|
+
|
|
69
|
+
producer.produce(
|
|
70
|
+
topic='orders',
|
|
71
|
+
key=order_id.encode(), # 相同 key → 相同 partition → 有序
|
|
72
|
+
value=json.dumps(order).encode(),
|
|
73
|
+
callback=delivery_report
|
|
74
|
+
)
|
|
75
|
+
producer.flush()
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### 消费者
|
|
79
|
+
|
|
80
|
+
```python
|
|
81
|
+
from confluent_kafka import Consumer
|
|
82
|
+
|
|
83
|
+
conf = {
|
|
84
|
+
'bootstrap.servers': 'kafka:9092',
|
|
85
|
+
'group.id': 'order-processor',
|
|
86
|
+
'auto.offset.reset': 'earliest',
|
|
87
|
+
'enable.auto.commit': False, # 手动提交
|
|
88
|
+
'max.poll.interval.ms': 300000,
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
consumer = Consumer(conf)
|
|
92
|
+
consumer.subscribe(['orders'])
|
|
93
|
+
|
|
94
|
+
try:
|
|
95
|
+
while True:
|
|
96
|
+
msg = consumer.poll(1.0)
|
|
97
|
+
if msg is None:
|
|
98
|
+
continue
|
|
99
|
+
if msg.error():
|
|
100
|
+
handle_error(msg.error())
|
|
101
|
+
continue
|
|
102
|
+
|
|
103
|
+
process_message(msg.value())
|
|
104
|
+
consumer.commit(asynchronous=False) # 处理成功后提交
|
|
105
|
+
finally:
|
|
106
|
+
consumer.close()
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
### Kafka 关键配置
|
|
110
|
+
|
|
111
|
+
```yaml
|
|
112
|
+
Broker:
|
|
113
|
+
num.partitions: 6 # 默认分区数
|
|
114
|
+
default.replication.factor: 3 # 副本数
|
|
115
|
+
min.insync.replicas: 2 # 最小同步副本
|
|
116
|
+
log.retention.hours: 168 # 保留 7 天
|
|
117
|
+
log.segment.bytes: 1073741824 # 1GB 段文件
|
|
118
|
+
|
|
119
|
+
Topic 设计:
|
|
120
|
+
分区数 = max(生产吞吐/单分区写入能力, 消费者数)
|
|
121
|
+
副本数 = 3 (生产环境)
|
|
122
|
+
Key 选择: 业务ID (保证同一实体有序)
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## RabbitMQ
|
|
128
|
+
|
|
129
|
+
### Exchange 类型
|
|
130
|
+
|
|
131
|
+
| 类型 | 路由规则 | 适用场景 |
|
|
132
|
+
|------|----------|----------|
|
|
133
|
+
| Direct | 精确匹配 routing key | 点对点 |
|
|
134
|
+
| Fanout | 广播到所有绑定队列 | 发布订阅 |
|
|
135
|
+
| Topic | 通配符匹配 routing key | 灵活路由 |
|
|
136
|
+
| Headers | 匹配消息头 | 复杂路由 |
|
|
137
|
+
|
|
138
|
+
```
|
|
139
|
+
Producer → Exchange → Binding → Queue → Consumer
|
|
140
|
+
│
|
|
141
|
+
routing_key 匹配
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
### 可靠性保证
|
|
145
|
+
|
|
146
|
+
```yaml
|
|
147
|
+
生产者:
|
|
148
|
+
- Publisher Confirms (确认模式)
|
|
149
|
+
- 持久化消息 (delivery_mode=2)
|
|
150
|
+
- 事务模式 (性能差,不推荐)
|
|
151
|
+
|
|
152
|
+
Broker:
|
|
153
|
+
- 持久化队列 (durable=True)
|
|
154
|
+
- 镜像队列 / Quorum Queue
|
|
155
|
+
- 磁盘持久化
|
|
156
|
+
|
|
157
|
+
消费者:
|
|
158
|
+
- 手动 ACK (auto_ack=False)
|
|
159
|
+
- 预取限制 (prefetch_count)
|
|
160
|
+
- 死信队列 (DLX) 处理失败消息
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
### 死信队列 (DLQ)
|
|
164
|
+
|
|
165
|
+
```
|
|
166
|
+
正常队列 ──(消费失败/TTL过期/队列满)──→ 死信交换机 → 死信队列
|
|
167
|
+
│
|
|
168
|
+
人工处理 / 重试
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
---
|
|
172
|
+
|
|
173
|
+
## Redis Streams
|
|
174
|
+
|
|
175
|
+
```bash
|
|
176
|
+
# 生产
|
|
177
|
+
XADD orders * user_id "123" amount "99.99"
|
|
178
|
+
|
|
179
|
+
# 消费组
|
|
180
|
+
XGROUP CREATE orders order-group $ MKSTREAM
|
|
181
|
+
XREADGROUP GROUP order-group consumer-1 COUNT 10 BLOCK 5000 STREAMS orders >
|
|
182
|
+
|
|
183
|
+
# 确认
|
|
184
|
+
XACK orders order-group <message-id>
|
|
185
|
+
|
|
186
|
+
# 查看待处理
|
|
187
|
+
XPENDING orders order-group
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
| 特性 | 适用 | 不适用 |
|
|
191
|
+
|------|------|--------|
|
|
192
|
+
| 轻量级 | 中小规模、低延迟 | 海量数据持久化 |
|
|
193
|
+
| 消费组 | 多消费者并行 | 复杂路由 |
|
|
194
|
+
| 内存存储 | 实时处理 | 长期存储 |
|
|
195
|
+
|
|
196
|
+
---
|
|
197
|
+
|
|
198
|
+
## 事件驱动架构
|
|
199
|
+
|
|
200
|
+
### Event Sourcing
|
|
201
|
+
|
|
202
|
+
```
|
|
203
|
+
传统: 只存最终状态
|
|
204
|
+
Account { balance: 100 }
|
|
205
|
+
|
|
206
|
+
Event Sourcing: 存储所有事件
|
|
207
|
+
AccountCreated { initial: 0 }
|
|
208
|
+
MoneyDeposited { amount: 200 }
|
|
209
|
+
MoneyWithdrawn { amount: 100 }
|
|
210
|
+
→ 重放得到 balance: 100
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
### CQRS (Command Query Responsibility Segregation)
|
|
214
|
+
|
|
215
|
+
```
|
|
216
|
+
Command (写) ──→ Write Model ──→ Event Store
|
|
217
|
+
│
|
|
218
|
+
Event Bus
|
|
219
|
+
│
|
|
220
|
+
Query (读) ←── Read Model ←── Projection
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
### Saga 模式
|
|
224
|
+
|
|
225
|
+
```
|
|
226
|
+
分布式事务编排:
|
|
227
|
+
|
|
228
|
+
Choreography (编舞):
|
|
229
|
+
Order → Payment → Inventory → Shipping
|
|
230
|
+
每个服务监听事件,自主决策
|
|
231
|
+
|
|
232
|
+
Orchestration (编排):
|
|
233
|
+
Saga Orchestrator
|
|
234
|
+
├→ Order Service: 创建订单
|
|
235
|
+
├→ Payment Service: 扣款
|
|
236
|
+
├→ Inventory Service: 扣库存
|
|
237
|
+
└→ Shipping Service: 发货
|
|
238
|
+
|
|
239
|
+
失败补偿:
|
|
240
|
+
Shipping失败 → 补偿Inventory → 补偿Payment → 补偿Order
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
## 选型对比
|
|
246
|
+
|
|
247
|
+
| 维度 | Kafka | RabbitMQ | Redis Streams |
|
|
248
|
+
|------|-------|----------|---------------|
|
|
249
|
+
| 吞吐量 | 极高 (百万/s) | 高 (万/s) | 高 (十万/s) |
|
|
250
|
+
| 延迟 | ms 级 | μs-ms 级 | μs 级 |
|
|
251
|
+
| 持久化 | 磁盘 | 磁盘/内存 | 内存+AOF |
|
|
252
|
+
| 消息顺序 | 分区内有序 | 队列内有序 | 流内有序 |
|
|
253
|
+
| 消息回溯 | ✅ 支持 | ❌ 不支持 | ✅ 支持 |
|
|
254
|
+
| 协议 | 自有协议 | AMQP | Redis协议 |
|
|
255
|
+
| 适用 | 大数据/日志/流处理 | 业务消息/RPC | 轻量级实时 |
|
|
256
|
+
|
|
257
|
+
### 选型决策树
|
|
258
|
+
|
|
259
|
+
```
|
|
260
|
+
需要消息回溯?
|
|
261
|
+
├─ 是 → Kafka / Redis Streams
|
|
262
|
+
└─ 否 → 需要复杂路由?
|
|
263
|
+
├─ 是 → RabbitMQ
|
|
264
|
+
└─ 否 → 吞吐量要求?
|
|
265
|
+
├─ 极高 (>10万/s) → Kafka
|
|
266
|
+
├─ 中等 → RabbitMQ
|
|
267
|
+
└─ 轻量 → Redis Streams
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
---
|
|
271
|
+
|
|
272
|
+
## 常见问题
|
|
273
|
+
|
|
274
|
+
### 消息丢失
|
|
275
|
+
|
|
276
|
+
```yaml
|
|
277
|
+
防丢三板斧:
|
|
278
|
+
生产端: acks=all + retries + 幂等
|
|
279
|
+
Broker: replication + 持久化 + min.insync.replicas
|
|
280
|
+
消费端: 手动提交 + 处理后确认
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
### 消息重复
|
|
284
|
+
|
|
285
|
+
```yaml
|
|
286
|
+
幂等处理:
|
|
287
|
+
- 数据库唯一约束 (message_id)
|
|
288
|
+
- Redis SETNX 去重
|
|
289
|
+
- 业务层幂等设计 (状态机)
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
### 消息积压
|
|
293
|
+
|
|
294
|
+
```yaml
|
|
295
|
+
应急:
|
|
296
|
+
- 增加消费者实例
|
|
297
|
+
- 临时扩大分区 (Kafka)
|
|
298
|
+
- 跳过非关键消息
|
|
299
|
+
|
|
300
|
+
根治:
|
|
301
|
+
- 优化消费者处理速度
|
|
302
|
+
- 合理设置分区数
|
|
303
|
+
- 监控消费 lag 告警
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
---
|
|
307
|
+
|
|
308
|
+
## 最佳实践
|
|
309
|
+
|
|
310
|
+
```yaml
|
|
311
|
+
设计:
|
|
312
|
+
- 消息体尽量小,大数据用引用
|
|
313
|
+
- 消息必须包含唯一ID和时间戳
|
|
314
|
+
- 定义清晰的消息 Schema (Avro/Protobuf)
|
|
315
|
+
- 版本兼容 (向后兼容)
|
|
316
|
+
|
|
317
|
+
运维:
|
|
318
|
+
- 监控消费 lag
|
|
319
|
+
- 死信队列告警
|
|
320
|
+
- 定期清理过期消息
|
|
321
|
+
- 容量规划 (磁盘/内存)
|
|
322
|
+
|
|
323
|
+
安全:
|
|
324
|
+
- TLS 加密传输
|
|
325
|
+
- SASL 认证
|
|
326
|
+
- ACL 授权
|
|
327
|
+
- 审计日志
|
|
328
|
+
```
|
|
329
|
+
|