@tmpxinchen/tmp1 1.0.56 → 1.0.57

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,1122 @@
1
+ # issue-bot 多 project 最小改造说明
2
+
3
+ ## 0. 目标
4
+
5
+ 把一个 issue-bot 从“只处理一个 project”改成“一个 bot 进程处理多个 project”。
6
+
7
+ 本次只做最小改造:
8
+
9
+ ```text
10
+ 一个 bot 进程
11
+ 一个 group webhook
12
+ 一个 token(可访问所有目标 project)
13
+ 多个 project 路径配置
14
+ 任务 / state / workspace 按 project 隔离
15
+ ```
16
+
17
+ 本次**不做**:
18
+
19
+ ```text
20
+ 每 project 一个容器
21
+ 每 project 一个 token
22
+ worker pool
23
+ controller + 临时 worker
24
+ 多租户框架
25
+ 复杂权限模型
26
+ ```
27
+
28
+ ---
29
+
30
+ ## 1. 给内部模型的操作要求
31
+
32
+ 内部代码已经和本地分叉,**不要整文件覆盖**。
33
+
34
+ 请按下面的顺序修改:
35
+
36
+ ```text
37
+ 1. 先备份
38
+ 2. 按函数名 / 代码块定位
39
+ 3. 只替换文档中明确指出的函数或代码块
40
+ 4. 保留内部已有的标签名、路径、日志、token 方式
41
+ 5. 最后用 python3 -m py_compile 和测试清单验证
42
+ ```
43
+
44
+ 建议先备份:
45
+
46
+ ```bash
47
+ cp automation/issue_bot.py automation/issue_bot.py.bak-$(date +%Y%m%d%H%M%S)
48
+ cp scripts/gitlab-init.sh scripts/gitlab-init.sh.bak-$(date +%Y%m%d%H%M%S)
49
+ cp scripts/selfcheck.sh scripts/selfcheck.sh.bak-$(date +%Y%m%d%H%M%S)
50
+ ```
51
+
52
+ 本次改造只涉及:
53
+
54
+ ```text
55
+ automation/issue_bot.py
56
+ docker-compose.yml
57
+ .env
58
+ .env.example
59
+ scripts/gitlab-init.sh
60
+ scripts/selfcheck.sh
61
+ README.md
62
+ automation/README.md
63
+ docs/自动化流程.md
64
+ ```
65
+
66
+ 不涉及:
67
+
68
+ ```text
69
+ GitLab Runner
70
+ CI 模板
71
+ Maven/Nexus
72
+ opencode 镜像
73
+ 门禁脚本
74
+ 业务仓库
75
+ ```
76
+
77
+ ---
78
+
79
+ ## 2. 配置改造
80
+
81
+ ### 2.1 `.env` 和 `.env.example`
82
+
83
+ 在 `AUTOMATION_PROJECT` 下面增加:
84
+
85
+ ```bash
86
+ # 多 project:逗号分隔完整路径,例如 observability/a,observability/b
87
+ # 留空时只处理 AUTOMATION_GROUP/AUTOMATION_PROJECT
88
+ AUTOMATION_PROJECTS=
89
+ ```
90
+
91
+ 说明:
92
+
93
+ - 默认留空,保持单项目行为;
94
+ - 要启用多 project,填完整路径;
95
+ - 路径必须和 GitLab webhook 里的 `project.path_with_namespace` 完全一致;
96
+ - 不要只写项目名,必须带 group,例如 `observability/payment-service`。
97
+
98
+ 示例:
99
+
100
+ ```bash
101
+ AUTOMATION_PROJECTS=observability/metric-ingest-service,observability/payment-service
102
+ ```
103
+
104
+ ### 2.2 `docker-compose.yml`
105
+
106
+ 在 issue-bot 的 `environment` 中,`GITLAB_PROJECT_PATH` 后面增加:
107
+
108
+ ```yaml
109
+ GITLAB_PROJECT_PATH: ${AUTOMATION_GROUP:-observability}/${AUTOMATION_PROJECT:-metric-ingest-service}
110
+ # 多 project 时填逗号分隔的完整路径;为空则回退到 GITLAB_PROJECT_PATH
111
+ GITLAB_PROJECTS: ${AUTOMATION_PROJECTS:-}
112
+ ```
113
+
114
+ 不要删除原来的 `GITLAB_PROJECT_PATH`,它用于单项目回退。
115
+
116
+ ---
117
+
118
+ ## 3. `automation/issue_bot.py` 改造
119
+
120
+ ### 3.1 常量:新增 `PROJECTS`,删除 `PROJECT_API`
121
+
122
+ 原代码:
123
+
124
+ ```python
125
+ PROJECT_PATH = os.environ.get("GITLAB_PROJECT_PATH", "observability/metric-ingest-service")
126
+ ```
127
+
128
+ 改为:
129
+
130
+ ```python
131
+ PROJECT_PATH = os.environ.get("GITLAB_PROJECT_PATH", "observability/metric-ingest-service")
132
+ PROJECTS = list(dict.fromkeys(
133
+ p.strip()
134
+ for p in os.environ.get("GITLAB_PROJECTS", "").split(",")
135
+ if p.strip()
136
+ ))
137
+ if not PROJECTS:
138
+ PROJECTS = [PROJECT_PATH]
139
+ ```
140
+
141
+ 然后找到并删除这一行:
142
+
143
+ ```python
144
+ PROJECT_API = "/api/v4/projects/" + urllib.parse.quote(PROJECT_PATH, safe="")
145
+ ```
146
+
147
+ 把队列类型改为:
148
+
149
+ ```python
150
+ TASKS: "queue.Queue[tuple[str, int]]" = queue.Queue()
151
+ SEEN = set()
152
+ SEEN_LOCK = threading.Lock()
153
+ ```
154
+
155
+ 注意:
156
+
157
+ - `SEEN` 不再只放 `iid`,后面会放 `(project, iid)`;
158
+ - `TASKS` 不再只放 `iid`,后面会放 `(project, iid)`。
159
+
160
+ ---
161
+
162
+ ### 3.2 新增 project 工具函数
163
+
164
+ 在 `clean()` 后面、`api_raw()` 前面增加:
165
+
166
+ ```python
167
+ def project_key(project):
168
+ """文件系统安全的 project key。"""
169
+ return urllib.parse.quote(project, safe="")
170
+
171
+
172
+ def project_api(project):
173
+ return "/api/v4/projects/" + urllib.parse.quote(project, safe="")
174
+ ```
175
+
176
+ 用途:
177
+
178
+ - `project_api()` 用于拼 GitLab API;
179
+ - `project_key()` 用于 state / workspace 目录名,避免 `/` 把目录拆开。
180
+
181
+ ---
182
+
183
+ ### 3.3 `api()` 改成按 project 调用
184
+
185
+ 原代码:
186
+
187
+ ```python
188
+ def api(method, path, data=None, timeout=30):
189
+ return api_raw(method, PROJECT_API + path, data, timeout)
190
+ ```
191
+
192
+ 改为:
193
+
194
+ ```python
195
+ def api(project, method, path, data=None, timeout=30):
196
+ return api_raw(method, project_api(project) + path, data, timeout)
197
+ ```
198
+
199
+ ---
200
+
201
+ ### 3.4 `api_get()` 增加 project
202
+
203
+ 原代码:
204
+
205
+ ```python
206
+ def api_get(path, **params):
207
+ if params:
208
+ path += "?" + urllib.parse.urlencode(params)
209
+ return api("GET", path)
210
+ ```
211
+
212
+ 改为:
213
+
214
+ ```python
215
+ def api_get(project, path, **params):
216
+ if params:
217
+ path += "?" + urllib.parse.urlencode(params)
218
+ return api(project, "GET", path)
219
+ ```
220
+
221
+ ---
222
+
223
+ ### 3.5 `note_issue()` / `note_mr()` 增加 project
224
+
225
+ 原代码:
226
+
227
+ ```python
228
+ def note_issue(iid, body):
229
+ try:
230
+ api("POST", "/issues/%d/notes" % iid, {"body": body})
231
+ except Exception as e:
232
+ log.warning("评论 issue #%s 失败: %s", iid, mask(e))
233
+
234
+
235
+ def note_mr(mr_iid, body):
236
+ try:
237
+ api("POST", "/merge_requests/%d/notes" % mr_iid, {"body": body})
238
+ except Exception as e:
239
+ log.warning("评论 MR !%s 失败: %s", mr_iid, mask(e))
240
+ ```
241
+
242
+ 改为:
243
+
244
+ ```python
245
+ def note_issue(project, iid, body):
246
+ try:
247
+ api(project, "POST", "/issues/%d/notes" % iid, {"body": body})
248
+ except Exception as e:
249
+ log.warning("评论 %s issue #%s 失败: %s", project, iid, mask(e))
250
+
251
+
252
+ def note_mr(project, mr_iid, body):
253
+ try:
254
+ api(project, "POST", "/merge_requests/%d/notes" % mr_iid, {"body": body})
255
+ except Exception as e:
256
+ log.warning("评论 %s MR !%s 失败: %s", project, mr_iid, mask(e))
257
+ ```
258
+
259
+ ---
260
+
261
+ ### 3.6 state 相关函数增加 project
262
+
263
+ 原代码:
264
+
265
+ ```python
266
+ def state_path(iid):
267
+ return STATE_DIR / ("issue-%d.json" % iid)
268
+
269
+
270
+ def load_state(iid):
271
+ try:
272
+ return json.loads(state_path(iid).read_text(encoding="utf-8"))
273
+ except Exception:
274
+ return {}
275
+
276
+
277
+ def save_state(iid, data):
278
+ try:
279
+ STATE_DIR.mkdir(parents=True, exist_ok=True)
280
+ state_path(iid).write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
281
+ except Exception as e:
282
+ log.warning("保存 issue #%s 状态失败: %s", iid, e)
283
+
284
+
285
+ def clear_state(iid):
286
+ try:
287
+ state_path(iid).unlink(missing_ok=True)
288
+ except Exception:
289
+ pass
290
+ ```
291
+
292
+ 改为:
293
+
294
+ ```python
295
+ def state_path(project, iid):
296
+ return STATE_DIR / project_key(project) / ("issue-%d.json" % iid)
297
+
298
+
299
+ def load_state(project, iid):
300
+ try:
301
+ return json.loads(state_path(project, iid).read_text(encoding="utf-8"))
302
+ except Exception:
303
+ return {}
304
+
305
+
306
+ def save_state(project, iid, data):
307
+ try:
308
+ path = state_path(project, iid)
309
+ path.parent.mkdir(parents=True, exist_ok=True)
310
+ path.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
311
+ except Exception as e:
312
+ log.warning("保存 %s issue #%s 状态失败: %s", project, iid, e)
313
+
314
+
315
+ def clear_state(project, iid):
316
+ try:
317
+ state_path(project, iid).unlink(missing_ok=True)
318
+ except Exception:
319
+ pass
320
+ ```
321
+
322
+ 最终目录形态:
323
+
324
+ ```text
325
+ data/automation/state/observability%2Fmetric-ingest-service/issue-1.json
326
+ data/automation/state/observability%2Fpayment-service/issue-1.json
327
+ ```
328
+
329
+ 不同项目的 `#1` 不再冲突。
330
+
331
+ ---
332
+
333
+ ### 3.7 `set_labels()` / `set_labels_safe()` / `human_notes()` 增加 project
334
+
335
+ 原代码:
336
+
337
+ ```python
338
+ def set_labels(iid, add=(), remove=()):
339
+ data = {}
340
+ if add:
341
+ data["add_labels"] = ",".join(add)
342
+ if remove:
343
+ data["remove_labels"] = ",".join(remove)
344
+ if data:
345
+ api("PUT", "/issues/%d" % iid, data)
346
+
347
+
348
+ def set_labels_safe(iid, add=(), remove=()):
349
+ try:
350
+ set_labels(iid, add=add, remove=remove)
351
+ except Exception as e:
352
+ log.warning("更新 issue #%s 状态标签失败: %s", iid, mask(e))
353
+ ```
354
+
355
+ 改为:
356
+
357
+ ```python
358
+ def set_labels(project, iid, add=(), remove=()):
359
+ data = {}
360
+ if add:
361
+ data["add_labels"] = ",".join(add)
362
+ if remove:
363
+ data["remove_labels"] = ",".join(remove)
364
+ if data:
365
+ api(project, "PUT", "/issues/%d" % iid, data)
366
+
367
+
368
+ def set_labels_safe(project, iid, add=(), remove=()):
369
+ try:
370
+ set_labels(project, iid, add=add, remove=remove)
371
+ except Exception as e:
372
+ log.warning("更新 %s issue #%s 状态标签失败: %s", project, iid, mask(e))
373
+ ```
374
+
375
+ `human_notes()` 原代码:
376
+
377
+ ```python
378
+ def human_notes(iid):
379
+ try:
380
+ notes = api_get("/issues/%d/notes" % iid, per_page=100, sort="asc")
381
+ except Exception as e:
382
+ log.warning("读取 issue #%s 评论失败: %s", iid, mask(e))
383
+ return []
384
+ ```
385
+
386
+ 改为:
387
+
388
+ ```python
389
+ def human_notes(project, iid):
390
+ try:
391
+ notes = api_get(project, "/issues/%d/notes" % iid, per_page=100, sort="asc")
392
+ except Exception as e:
393
+ log.warning("读取 %s issue #%s 评论失败: %s", project, iid, mask(e))
394
+ return []
395
+ ```
396
+
397
+ ---
398
+
399
+ ### 3.8 `enqueue()` 改为 `(project, iid)`
400
+
401
+ 原代码:
402
+
403
+ ```python
404
+ def enqueue(iid):
405
+ with SEEN_LOCK:
406
+ if iid in SEEN:
407
+ log.info("issue #%s 已在队列/处理中,忽略重复 webhook", iid)
408
+ return
409
+ SEEN.add(iid)
410
+ TASKS.put(iid)
411
+ log.info("issue #%s 入队", iid)
412
+ ```
413
+
414
+ 改为:
415
+
416
+ ```python
417
+ def enqueue(project, iid):
418
+ key = (project, iid)
419
+ with SEEN_LOCK:
420
+ if key in SEEN:
421
+ log.info("%s issue #%s 已在队列/处理中,忽略重复 webhook", project, iid)
422
+ return
423
+ SEEN.add(key)
424
+ TASKS.put(key)
425
+ log.info("%s issue #%s 入队", project, iid)
426
+ ```
427
+
428
+ ---
429
+
430
+ ### 3.9 `clone_url()` 增加 project
431
+
432
+ 原代码:
433
+
434
+ ```python
435
+ def clone_url():
436
+ u = urllib.parse.urlparse(GITLAB_URL)
437
+ tok = urllib.parse.quote(token(), safe="")
438
+ return "%s://oauth2:%s@%s/%s.git" % (u.scheme, tok, u.netloc, PROJECT_PATH)
439
+ ```
440
+
441
+ 改为:
442
+
443
+ ```python
444
+ def clone_url(project):
445
+ u = urllib.parse.urlparse(GITLAB_URL)
446
+ tok = urllib.parse.quote(token(), safe="")
447
+ return "%s://oauth2:%s@%s/%s.git" % (u.scheme, tok, u.netloc, project)
448
+ ```
449
+
450
+ ---
451
+
452
+ ### 3.10 `wait_pipeline()` 增加 project
453
+
454
+ 原代码:
455
+
456
+ ```python
457
+ def wait_pipeline(branch, head_sha):
458
+ ...
459
+ pipes = api_get("/pipelines", ref=branch, per_page=1)
460
+ ```
461
+
462
+ 改为:
463
+
464
+ ```python
465
+ def wait_pipeline(project, branch, head_sha):
466
+ ...
467
+ pipes = api_get(project, "/pipelines", ref=branch, per_page=1)
468
+ ```
469
+
470
+ 日志也改成:
471
+
472
+ ```python
473
+ log.warning("查询 %s pipeline 失败: %s", project, mask(e))
474
+ ```
475
+
476
+ ---
477
+
478
+ ### 3.11 `ensure_runtime_labels()` 增加 project
479
+
480
+ 原代码:
481
+
482
+ ```python
483
+ def ensure_runtime_labels():
484
+ ...
485
+ api("POST", "/labels", {"name": name, "color": color})
486
+ except Exception as e:
487
+ if "-> 409:" not in str(e):
488
+ log.warning("创建标签 %s 失败: %s", name, mask(e))
489
+ ```
490
+
491
+ 改为:
492
+
493
+ ```python
494
+ def ensure_runtime_labels(project):
495
+ ...
496
+ api(project, "POST", "/labels", {"name": name, "color": color})
497
+ except Exception as e:
498
+ if "-> 409:" not in str(e):
499
+ log.warning("创建 %s 标签 %s 失败: %s", project, name, mask(e))
500
+ ```
501
+
502
+ ---
503
+
504
+ ### 3.12 `recover_stale_processing()` 增加 project
505
+
506
+ 关键改动:
507
+
508
+ ```python
509
+ def recover_stale_processing(project):
510
+ try:
511
+ issues = api_get(project, "/issues", labels=PROCESSING_LABEL, state="opened", per_page=100)
512
+ ...
513
+ open_mrs = api_get(project, "/merge_requests", state="opened", per_page=100)
514
+ ...
515
+ set_labels_safe(project, iid, add=[DONE_LABEL], remove=[PROCESSING_LABEL])
516
+ ...
517
+ set_labels_safe(project, iid, add=[FAILED_LABEL], remove=[PROCESSING_LABEL])
518
+ note_issue(project, iid, "⚠️ 上一次自动处理中断,已标记为失败。请重新加 `%s` 重试。" % TRIGGER_LABEL)
519
+ ```
520
+
521
+ 日志中带上 project:
522
+
523
+ ```python
524
+ log.warning("扫描 %s ai-processing issue 失败: %s", project, mask(e))
525
+ log.warning("查询 %s 打开 MR 失败,跳过启动恢复以避免误标: %s", project, mask(e))
526
+ log.info("%s issue #%s 已有打开 MR,恢复为 ai-done", project, iid)
527
+ ```
528
+
529
+ ---
530
+
531
+ ### 3.13 `recover_queued_triggers()` 增加 project
532
+
533
+ 关键改动:
534
+
535
+ ```python
536
+ def recover_queued_triggers(project):
537
+ try:
538
+ issues = api_get(project, "/issues", labels=TRIGGER_LABEL, state="opened", per_page=100)
539
+ ...
540
+ enqueue(project, iid)
541
+ ```
542
+
543
+ ---
544
+
545
+ ### 3.14 `_handle()` 增加 project
546
+
547
+ 函数签名:
548
+
549
+ ```python
550
+ def _handle(project, iid):
551
+ log.info("开始处理 %s issue #%s", project, iid)
552
+ issue = api_get(project, "/issues/%d" % iid)
553
+ ```
554
+
555
+ 所有内部调用都要加 project:
556
+
557
+ ```python
558
+ note_issue(project, iid, ...)
559
+ set_labels(project, iid, ...)
560
+ set_labels_safe(project, iid, ...)
561
+ state = load_state(project, iid)
562
+ api_get(project, "/merge_requests", state="opened", per_page=100)
563
+ project_info = api_get(project, "")
564
+ human_notes(project, iid)
565
+ save_state(project, iid, state)
566
+ clear_state(project, iid)
567
+ wait_pipeline(project, branch, head_sha)
568
+ note_mr(project, mr_iid, ...)
569
+ ```
570
+
571
+ workspace 路径必须带 project:
572
+
573
+ ```python
574
+ ws = WORKSPACE_ROOT / project_key(project) / ("issue-%s-%d" % (iid, int(time.time())))
575
+ ```
576
+
577
+ clone 前创建父目录:
578
+
579
+ ```python
580
+ ws.parent.mkdir(parents=True, exist_ok=True)
581
+ ```
582
+
583
+ clone 使用:
584
+
585
+ ```python
586
+ clone_url(project)
587
+ ```
588
+
589
+ project 默认分支查询:
590
+
591
+ ```python
592
+ project_info = api_get(project, "")
593
+ target = TARGET_BRANCH or project_info.get("default_branch") or "main"
594
+ ```
595
+
596
+ MR 创建:
597
+
598
+ ```python
599
+ mr = api(project, "POST", "/merge_requests", {
600
+ ...
601
+ })
602
+ ```
603
+
604
+ 标签创建:
605
+
606
+ ```python
607
+ api(project, "POST", "/labels", {"name": "path::ai", "color": "#428bca"})
608
+ ```
609
+
610
+ 异常日志:
611
+
612
+ ```python
613
+ log.exception("%s issue #%s 处理失败", project, iid)
614
+ ```
615
+
616
+ ---
617
+
618
+ ### 3.15 `handle()` / `worker()` 改为元组任务
619
+
620
+ 原代码:
621
+
622
+ ```python
623
+ def handle(iid):
624
+ try:
625
+ _handle(iid)
626
+ finally:
627
+ with SEEN_LOCK:
628
+ SEEN.discard(iid)
629
+
630
+
631
+ def worker():
632
+ while True:
633
+ iid = TASKS.get()
634
+ try:
635
+ handle(iid)
636
+ except Exception:
637
+ log.exception("worker 未捕获异常")
638
+ finally:
639
+ TASKS.task_done()
640
+ ```
641
+
642
+ 改为:
643
+
644
+ ```python
645
+ def handle(project, iid):
646
+ try:
647
+ _handle(project, iid)
648
+ finally:
649
+ with SEEN_LOCK:
650
+ SEEN.discard((project, iid))
651
+
652
+
653
+ def worker():
654
+ while True:
655
+ project, iid = TASKS.get()
656
+ try:
657
+ handle(project, iid)
658
+ except Exception:
659
+ log.exception("worker 未捕获异常")
660
+ finally:
661
+ TASKS.task_done()
662
+ ```
663
+
664
+ ---
665
+
666
+ ### 3.16 `Handler.do_POST()` 增加 project 识别
667
+
668
+ 在 payload 解析完成后、判断 `object_kind` 之前增加:
669
+
670
+ ```python
671
+ # 多 project 的关键:先从 webhook 识别 project,并校验白名单。
672
+ project = (payload.get("project") or {}).get("path_with_namespace", "")
673
+ if not project or project not in PROJECTS:
674
+ self._json(202, {"ignored": "unknown-project", "project": project})
675
+ return
676
+ ```
677
+
678
+ Note Hook 中所有调用加 project:
679
+
680
+ ```python
681
+ issue = api_get(project, "/issues/%d" % iid)
682
+ ...
683
+ enqueue(project, iid)
684
+ self._json(202, {"queued": iid, "project": project, "via": "note"})
685
+ ```
686
+
687
+ Issue Hook 中:
688
+
689
+ ```python
690
+ enqueue(project, iid)
691
+ self._json(202, {"queued": iid, "project": project})
692
+ ```
693
+
694
+ 注意:
695
+
696
+ - 不要只校验 label,必须先校验 project;
697
+ - 不在 `PROJECTS` 里的 project 直接返回 202;
698
+ - 不要把 `project` 写死成 `PROJECT_PATH`。
699
+
700
+ ---
701
+
702
+ ### 3.17 `main()` 遍历所有 project
703
+
704
+ 原代码:
705
+
706
+ ```python
707
+ ensure_runtime_labels()
708
+ recover_stale_processing()
709
+ recover_queued_triggers()
710
+ threading.Thread(target=worker, daemon=True).start()
711
+ server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
712
+ log.info("issue-bot 监听 :%s,project=%s,label=%s", PORT, PROJECT_PATH, TRIGGER_LABEL)
713
+ ```
714
+
715
+ 改为:
716
+
717
+ ```python
718
+ for project in PROJECTS:
719
+ ensure_runtime_labels(project)
720
+ recover_stale_processing(project)
721
+ recover_queued_triggers(project)
722
+ threading.Thread(target=worker, daemon=True).start()
723
+ server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
724
+ log.info("issue-bot 监听 :%s,projects=%s,label=%s", PORT, ",".join(PROJECTS), TRIGGER_LABEL)
725
+ ```
726
+
727
+ `/healthz` 可以增加:
728
+
729
+ ```python
730
+ "projects": PROJECTS
731
+ ```
732
+
733
+ 方便确认 bot 实际加载了哪些 project。
734
+
735
+ ---
736
+
737
+ ## 4. `scripts/gitlab-init.sh` 改造
738
+
739
+ ### 4.1 多 project 的 token 提示
740
+
741
+ 在 `setup_automation()` 里,原来的 bot token 逻辑前增加:
742
+
743
+ ```bash
744
+ local multi_project=0
745
+ case ",${AUTOMATION_PROJECTS:-}," in
746
+ *,*,*) multi_project=1 ;;
747
+ esac
748
+ ```
749
+
750
+ 然后:
751
+
752
+ - 如果 `AUTOMATION_BOT_TOKEN` 已设置,继续使用;
753
+ - 如果多 project:
754
+ - 没有 `data/automation/token` 时,提示“project access token 不能跨项目”,暂用 root PAT;
755
+ - 已有 token 时,提示“确认该 token 可访问所有目标项目”;
756
+ - 单 project 继续原逻辑:优先 project access token,失败降级 root PAT。
757
+
758
+ 替换后的核心逻辑:
759
+
760
+ ```bash
761
+ local multi_project=0
762
+ case ",${AUTOMATION_PROJECTS:-}," in
763
+ *,*,*) multi_project=1 ;;
764
+ esac
765
+
766
+ if [ -n "${AUTOMATION_BOT_TOKEN:-}" ]; then
767
+ info "使用 .env 里的 AUTOMATION_BOT_TOKEN"
768
+ elif [ "$multi_project" = "1" ]; then
769
+ if [ ! -s data/automation/token ]; then
770
+ warn "多 project 模式:project access token 不能跨项目,先使用 root PAT;生产请换成可访问全部目标项目的 token"
771
+ cp .gitlab-pat data/automation/token
772
+ chmod 600 data/automation/token
773
+ info "bot token 已写入 data/automation/token"
774
+ else
775
+ warn "多 project 模式:复用已有 data/automation/token,请确认该 token 可访问所有目标项目"
776
+ fi
777
+ elif [ ! -s data/automation/token ]; then
778
+ # 原 project access token 逻辑保持不变
779
+ ...
780
+ else
781
+ info "复用已有 data/automation/token"
782
+ fi
783
+ ```
784
+
785
+ 注意:
786
+
787
+ - 这是最小方案,不引入 per-project token;
788
+ - 生产环境应换成有目标项目权限的最小权限 token;
789
+ - 不要为了多 project 自动创建 group owner token。
790
+
791
+ ### 4.2 webhook 改为 group 级
792
+
793
+ 原代码是项目级:
794
+
795
+ ```bash
796
+ hooks=$(api GET "/projects/${PROJECT_ID}/hooks" ...)
797
+ api PUT "/projects/${PROJECT_ID}/hooks/${hook_id}" ...
798
+ api POST "/projects/${PROJECT_ID}/hooks" ...
799
+ ```
800
+
801
+ 改为 group 级:
802
+
803
+ ```bash
804
+ # 多 project 用 group webhook:一次覆盖组内所有项目,bot 再按 project 过滤。
805
+ local hook_url="http://issue-bot:8080/webhook"
806
+ local hooks hook_id
807
+ hooks=$(api GET "/groups/${GROUP_ID}/hooks" 2>/dev/null || echo '[]')
808
+ hook_id=$(printf '%s' "$hooks" | python3 -c 'import sys,json;d=json.load(sys.stdin);print(next((str(h["id"]) for h in d if h.get("url","").endswith("/webhook")),""))' 2>/dev/null || true)
809
+
810
+ if [ -n "${hook_id:-}" ]; then
811
+ api PUT "/groups/${GROUP_ID}/hooks/${hook_id}" \
812
+ --data-urlencode "url=${hook_url}" \
813
+ --data-urlencode "token=${AUTOMATION_WEBHOOK_SECRET}" \
814
+ --data "issue_events=true" --data "note_events=true" --data "enable_ssl_verification=false" >/dev/null 2>&1 \
815
+ || warn "更新 group webhook 失败,请检查组 Settings → Webhooks"
816
+ else
817
+ api POST "/groups/${GROUP_ID}/hooks" \
818
+ --data-urlencode "url=${hook_url}" \
819
+ --data-urlencode "token=${AUTOMATION_WEBHOOK_SECRET}" \
820
+ --data "issue_events=true" --data "note_events=true" --data "enable_ssl_verification=false" >/dev/null 2>&1 \
821
+ || warn "创建 group webhook 失败,请检查组 Settings → Webhooks"
822
+ fi
823
+
824
+ info "issue-bot 配置完成:group webhook=${hook_url}"
825
+ ```
826
+
827
+ 前提:
828
+
829
+ - `GROUP_ID` 在 `create_project()` 中已经设置;
830
+ - 如果内部分支没有 `GROUP_ID`,先用 group path 查一次:
831
+
832
+ ```bash
833
+ GROUP_ID=$(api GET "/groups/${AUTOMATION_GROUP:-observability}" \
834
+ | python3 -c 'import sys,json;print(json.load(sys.stdin)["id"])')
835
+ ```
836
+
837
+ 注意:
838
+
839
+ - 从单项目升级时,旧的 project webhook 可以手工删除,避免同一事件重复投递;
840
+ - bot 侧已有 `(project, iid)` 去重,重复投递不会重复处理,但会产生无意义请求。
841
+
842
+ ---
843
+
844
+ ## 5. `scripts/selfcheck.sh` 改造
845
+
846
+ 原自检只查项目级 webhook:
847
+
848
+ ```bash
849
+ hooks=$(curl ... "${GITLAB_URL}/api/v4/projects/${pid}/hooks" ...)
850
+ ```
851
+
852
+ 改为优先查 group webhook,失败再回退项目级:
853
+
854
+ ```bash
855
+ local hooks hook_ok group_path
856
+ group_path=$(grep -E '^AUTOMATION_GROUP=' .env 2>/dev/null | cut -d= -f2)
857
+ group_path=${group_path:-observability}
858
+ hooks=$(curl -fsS -m 10 --header "PRIVATE-TOKEN: $pat" \
859
+ "${GITLAB_URL}/api/v4/groups/${group_path}/hooks" 2>/dev/null || echo '[]')
860
+ hook_ok=$(printf '%s' "$hooks" | python3 -c 'import sys,json;d=json.load(sys.stdin);print(any(h.get("url","").endswith("/webhook") for h in d))' 2>/dev/null || echo False)
861
+ if [ "$hook_ok" != "True" ]; then
862
+ # 兼容升级前创建的项目级 webhook
863
+ hooks=$(curl -fsS -m 10 --header "PRIVATE-TOKEN: $pat" \
864
+ "${GITLAB_URL}/api/v4/projects/${pid}/hooks" 2>/dev/null || echo '[]')
865
+ hook_ok=$(printf '%s' "$hooks" | python3 -c 'import sys,json;d=json.load(sys.stdin);print(any(h.get("url","").endswith("/webhook") for h in d))' 2>/dev/null || echo False)
866
+ fi
867
+ [ "$hook_ok" = "True" ] && ok "issue webhook 已配置" || bad "issue webhook 未配置(检查组或项目 Settings → Webhooks)"
868
+ ```
869
+
870
+ 如果 `AUTOMATION_GROUP` 可能是子组(如 `a/b`),需要先 URL encode:
871
+
872
+ ```bash
873
+ group_enc=$(python3 -c 'import urllib.parse,sys;print(urllib.parse.quote(sys.argv[1], safe=""))' "$group_path")
874
+ # 使用 /groups/${group_enc}/hooks
875
+ ```
876
+
877
+ ---
878
+
879
+ ## 6. 文档更新
880
+
881
+ ### 6.1 `automation/README.md`
882
+
883
+ 增加“多 project”章节:
884
+
885
+ ```markdown
886
+ ## 多 project
887
+
888
+ 一个 bot 可以处理多个 project,用同一个 group token 即可,不需要每个项目一个容器。
889
+
890
+ 配置:
891
+
892
+ AUTOMATION_PROJECTS=observability/metric-ingest-service,observability/payment-service
893
+
894
+ 规则:
895
+
896
+ - GITLAB_PROJECTS 是逗号分隔的完整 group/project 路径;
897
+ - 为空时回退到单个 GITLAB_PROJECT_PATH;
898
+ - webhook 必须来自 GITLAB_PROJECTS 中的 project,否则直接忽略;
899
+ - state / workspace 按 project 分目录,不同项目的 #1 不会冲突;
900
+ - 启动恢复会遍历所有配置项目;
901
+ - gitlab-init.sh 使用 group webhook,组内所有 project 的事件都进同一个 bot,再由 bot 过滤;
902
+ - 多 project 时 data/automation/token 必须是可访问全部目标项目的 token。
903
+ ```
904
+
905
+ ### 6.2 `README.md`
906
+
907
+ 在自动化模式说明中增加:
908
+
909
+ ```markdown
910
+ 一个 bot 可处理多个 project:在 .env 配置 AUTOMATION_PROJECTS(逗号分隔完整路径),
911
+ webhook 会按 project 过滤,state / workspace 也按 project 隔离。
912
+ 多项目需要使用可访问全部目标项目的 token。
913
+ ```
914
+
915
+ ### 6.3 `docs/自动化流程.md`
916
+
917
+ 更新:
918
+
919
+ ```text
920
+ - 创建 group webhook(http://issue-bot:8080/webhook)
921
+ - 多 project 时按 AUTOMATION_PROJECTS 逐个创建标签、恢复 ai-processing / ai-fix
922
+ - state 和 workspace 目录带 project 前缀
923
+ ```
924
+
925
+ ---
926
+
927
+ ## 7. 测试清单
928
+
929
+ ### 7.1 语法检查
930
+
931
+ ```bash
932
+ python3 -m py_compile automation/issue_bot.py
933
+ bash -n scripts/gitlab-init.sh
934
+ bash -n scripts/selfcheck.sh
935
+ ```
936
+
937
+ ### 7.2 单项目兼容
938
+
939
+ `.env`:
940
+
941
+ ```bash
942
+ AUTOMATION_PROJECTS=
943
+ AUTOMATION_GROUP=observability
944
+ AUTOMATION_PROJECT=metric-ingest-service
945
+ ```
946
+
947
+ 启动后:
948
+
949
+ ```bash
950
+ docker logs ai-coding-pilot-issue-bot --tail=50
951
+ ```
952
+
953
+ 应看到:
954
+
955
+ ```text
956
+ projects=observability/metric-ingest-service
957
+ ```
958
+
959
+ 在 issue 打 `ai-fix`,应正常出 MR。
960
+
961
+ ### 7.3 多项目
962
+
963
+ `.env`:
964
+
965
+ ```bash
966
+ AUTOMATION_PROJECTS=observability/metric-ingest-service,observability/payment-service
967
+ ```
968
+
969
+ 验证:
970
+
971
+ ```text
972
+ 1. 两个项目各建一个 issue;
973
+ 2. 两个 issue 都打 ai-fix;
974
+ 3. 两个项目都能出各自的 MR;
975
+ 4. 两个 MR 的 source_branch 都是 ai/issue-1-*,但分别在不同仓库;
976
+ 5. state 目录出现:
977
+ data/automation/state/observability%2Fmetric-ingest-service/
978
+ data/automation/state/observability%2Fpayment-service/
979
+ 6. workspace 目录按 project 分开;
980
+ 7. 一个项目失败不影响另一个项目。
981
+ ```
982
+
983
+ ### 7.4 webhook 白名单
984
+
985
+ 构造一个来自未配置项目的 webhook:
986
+
987
+ ```json
988
+ {
989
+ "object_kind": "issue",
990
+ "project": {"path_with_namespace": "observability/unknown"},
991
+ "object_attributes": {"iid": 1, "action": "update", "state": "opened"},
992
+ "labels": [{"title": "ai-fix"}]
993
+ }
994
+ ```
995
+
996
+ 应返回:
997
+
998
+ ```json
999
+ {"ignored": "unknown-project", "project": "observability/unknown"}
1000
+ ```
1001
+
1002
+ ### 7.5 启动恢复
1003
+
1004
+ 给两个项目各放一个 `ai-processing` issue:
1005
+
1006
+ ```text
1007
+ 项目 A:有打开的 MR → 应恢复 ai-done
1008
+ 项目 B:没有 MR → 应恢复 ai-failed 并评论
1009
+ ```
1010
+
1011
+ 确认 PUT 请求路径分别带:
1012
+
1013
+ ```text
1014
+ /api/v4/projects/observability%2Fa/issues/...
1015
+ /api/v4/projects/observability%2Fb/issues/...
1016
+ ```
1017
+
1018
+ ---
1019
+
1020
+ ## 8. 常见问题
1021
+
1022
+ ### 8.1 所有事件都返回 `unknown-project`
1023
+
1024
+ 检查:
1025
+
1026
+ - `AUTOMATION_PROJECTS` 是否是逗号分隔的完整路径;
1027
+ - webhook 里的 `project.path_with_namespace` 是否和配置完全一致;
1028
+ - 是否把项目名写成了短名,例如 `payment-service`,实际应为 `observability/payment-service`;
1029
+ - 子组路径是否完整,例如 `group/subgroup/project`。
1030
+
1031
+ ### 8.2 只有第一个项目能打标签 / 评论
1032
+
1033
+ 原因通常是 token 只能访问一个项目。
1034
+
1035
+ 检查:
1036
+
1037
+ ```bash
1038
+ cat data/automation/token
1039
+ ```
1040
+
1041
+ 多项目需要:
1042
+
1043
+ - 一个能访问全部目标项目的 token;或
1044
+ - 明确设置 `AUTOMATION_BOT_TOKEN`。
1045
+
1046
+ `gitlab-init.sh` 自动创建的 project access token 只覆盖当前项目,不能跨项目。
1047
+
1048
+ ### 8.3 两个项目的 issue #1 状态互相覆盖
1049
+
1050
+ 说明 state / workspace 还没有完全改成 project 维度。
1051
+
1052
+ 检查这些地方是否都传了 project:
1053
+
1054
+ ```text
1055
+ state_path
1056
+ load_state
1057
+ save_state
1058
+ clear_state
1059
+ enqueue
1060
+ SEEN
1061
+ TASKS
1062
+ workspace 路径
1063
+ ```
1064
+
1065
+ 不能只改 API,不改 state。
1066
+
1067
+ ### 8.4 webhook 重复投递
1068
+
1069
+ 如果之前已经有 project webhook,现在又创建了 group webhook:
1070
+
1071
+ - 同一个事件可能进 bot 两次;
1072
+ - bot 的 `SEEN=(project, iid)` 会去重;
1073
+ - 建议删除旧的项目级 webhook。
1074
+
1075
+ ### 8.5 多项目仍然是串行
1076
+
1077
+ 当前还是单 worker:
1078
+
1079
+ ```python
1080
+ threading.Thread(target=worker, daemon=True).start()
1081
+ ```
1082
+
1083
+ 多个 project 的任务会排队执行。
1084
+ 这是本次最小方案的预期行为,不要在这次改造里引入 worker pool。
1085
+
1086
+ ---
1087
+
1088
+ ## 9. 本次不做的内容
1089
+
1090
+ 明确不要在这次改造中做:
1091
+
1092
+ ```text
1093
+ 1. 每 project 一个 token
1094
+ 2. 每 project 一个容器
1095
+ 3. controller + worker 容器
1096
+ 4. 每 project 并发池
1097
+ 5. pipeline watcher 独立线程
1098
+ 6. 多项目 measure.py 汇总
1099
+ 7. 复杂 RBAC / 审计平台
1100
+ ```
1101
+
1102
+ 这些可以作为下一阶段,但不要混进这次最小改造。
1103
+
1104
+ ---
1105
+
1106
+ ## 10. 完成标准
1107
+
1108
+ 满足以下条件即可认为改造完成:
1109
+
1110
+ ```text
1111
+ [ ] 一个 bot 能加载多个 project
1112
+ [ ] webhook 能按 project 路由
1113
+ [ ] 未配置项目被忽略
1114
+ [ ] 两个项目的 issue #1 不冲突
1115
+ [ ] 两个项目能各自 clone / push / 开 MR
1116
+ [ ] state / workspace 按 project 隔离
1117
+ [ ] 启动恢复按 project 执行
1118
+ [ ] 单项目配置仍能工作
1119
+ [ ] gitlab-init.sh 创建 group webhook
1120
+ [ ] selfcheck.sh 能识别 group webhook
1121
+ [ ] 文档已更新
1122
+ ```
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@tmpxinchen/tmp1",
3
- "version": "1.0.56",
3
+ "version": "1.0.57",
4
4
  "description": "gitlab-ce-18.11.12.tar transfer chunk 44/44",
5
5
  "license": "ISC",
6
6
  "author": "xinchen12"