teamai-cli 0.25.0 → 0.26.0-beta.0

package/skill-data/wiki/references/agents/kb-doc-generator.md

@@ -0,0 +1,323 @@
+# Knowledge Base Document Generator Agent
+
+## Responsibility
+
+Generate knowledge base documents for the assigned batch of components/document types, strictly following the nine document type specifications, ensuring code traceability, complete AI Quick Reference tables, and a web of bidirectional links.
+
+**This agent is started batch by batch by the main agent in Phase K2 and supports the parallel sub-agent dispatch mode.**
+
+## Input package
+
+```
+component_list:   list of component names or document types to generate in this batch
+                  e.g. ["Aurora", "Frame", "CCDB", "Dispatcher"] or ["Type-1", "Type-2", "Type-3"]
+architecture_map: full content of _review/k1-architecture-map.md
+repos:            repository list ([{name, path, language}]), replaces the old project_root
+service_map:      service name -> repository map (used to trace call chains across repositories)
+output_dir:       knowledge base output root directory
+project_name:     project name (used for document naming, e.g. "CVM")
+product_docs_dir: product documentation directory (may be empty; if empty, skip product constraint extraction)
+methodology_dir:  {SKILL_DIR}/references/methodology/ directory path
+completed_docs:   list of already completed documents (skipped when resuming from checkpoint)
+parallel_mode:    true | false (default true; Type-4 component documents in parallel, Type-1~3/5~8 serially)
+```
+
+## Execution steps
+
+### Step 0: Load the methodology
+
+Read `{methodology_dir}/phase2-document-types.md` and load the templates and generation rules for the relevant document types.
+
+### Step 1: Checkpoint check
+
+Check the `completed_docs` list, remove completed items from `component_list`, and obtain `pending_list`.
+
+If `pending_list` is empty, return an "all completed" summary immediately and perform no other action.
+
+### Step 2: Dispatch strategy decision
+
+```
+IF component_list consists only of Type-4 component documents AND parallel_mode = true:
+  → parallel mode (Step 2A)
+ELSE (Type-1/2/3/5/6/7/8 or parallel_mode = false):
+  → serial mode (Step 2B)
+```
+
+### Step 2A: Parallel mode (Type-4 component documents)
+
+**MANDATORY: you must use the Agent tool; processing components one by one in sequence is forbidden.**
+
+**Step 2A-1: Chunking**
+
+Split `pending_list` into chunks of **3~5 components** each (component documents are large; do not exceed 5 to avoid context overflow).
+- Prefer placing components from the same architecture layer in the same chunk (reduces cross-layer code reading contention)
+- Skip completed ones (resume from checkpoint)
+
+**Step 2A-2: Start all sub-agents concurrently in a single message**
+
+**Issue all Agent tool calls in the same reply**. This is the only way to run in parallel; issuing them in separate calls degrades to serial execution.
+
+Example (3 chunks concurrently):
+```
+[Agent tool call 1: chunk ["Aurora", "Frame"], subagent_type="general-purpose"]
+[Agent tool call 2: chunk ["CCDB", "VSResource"], subagent_type="general-purpose"]
+[Agent tool call 3: chunk ["Dispatcher", "Compute"], subagent_type="general-purpose"]
+```
+
+Each sub-agent receives the following prompt (replace CHUNK_COMPONENTS, CHUNK_NUM, TOTAL_CHUNKS):
+
+```
+You are the component document generation sub-agent of the wiki skill.
+Generate knowledge base documents for the following components (chunk CHUNK_NUM / TOTAL_CHUNKS):
+CHUNK_COMPONENTS
+
+Architecture reference (condensed; only the components in this chunk and their direct upstream/downstream):
+RELEVANT_COMPONENTS_TABLE
+(format: | Component | Architecture layer | Repository | Language | Upstream | Downstream | Entry file |)
+
+Service map (for cross-repository tracing):
+SERVICE_MAP_RELEVANT_ENTRIES
+
+Project information:
+- repos: REPO_LIST (paths only, no details)
+- output_dir: OUTPUT_DIR
+- project_name: PROJECT_NAME
+- product_docs_dir: PRODUCT_DOCS_DIR (if empty, skip product constraints)
+
+Methodology path: METHODOLOGY_DIR/phase2-document-types.md
+
+For each component:
+1. Scan the code with the Glob→Grep→Read three-step method (see kb-doc-generator.md §Step 2: Code structure scanning rules)
+2. Extract: core responsibility / architecture layer / upstream and downstream / code entry / core mechanisms / data flow / tech stack / data model / config items
+3. Generate a document that follows the Type-4 template and Write it to OUTPUT_DIR/XX_{component}_Design.md
+4. Self-check (see the Checklist below)
+5. Append each completed component name to OUTPUT_DIR/../_review/_chunk_done_CHUNK_NUM.txt (one per line)
+
+Self-check Checklist (after each document is generated):
+- [ ] All 10 dimensions of the AI Quick Reference table filled in and specific (not generic descriptions)?
+- [ ] "Code entry" precise to the function name (not just the file name)?
+- [ ] search-anchor has 5~15 keywords?
+- [ ] Contains a bidirectional link to the main architecture document?
+- [ ] Content that cannot be traced is marked [UNVERIFIED]?
+- [ ] No empty placeholder sections?
+
+[UNVERIFIED] above 20% → add a ⚠️ low-confidence warning at the top of the document.
+
+Write components that could not be generated to OUTPUT_DIR/../_review/_chunk_failed_CHUNK_NUM.txt with the reason.
+```
+
+**Step 2A-3: Wait and collect results**
+
+After all sub-agents finish:
+- Check the `_chunk_done_N.txt` files to confirm completion status
+- If `_chunk_done_N.txt` is missing for a chunk, print a warning: `chunk N may not have completed; check whether the sub-agent ran as the general-purpose type`
+- If more than half of the chunks failed, stop and tell the user to rerun
+- Merge all completed components into `kb_progress.components_done` in `progress.json`
+- Clean up temporary files: `rm -f _review/_chunk_done_*.txt _review/_chunk_failed_*.txt`
+
+### Step 2B: Serial mode (Type-1~3/5~8)
+
+For each document type in `pending_list`, execute **in sequence** (these document types depend on each other and must be serial):
+
+#### 2B-1: Code structure scanning rules
+
+Use the `Glob → Grep → Read` three-step method (**adapt to the language of the component's repository**):
+
+```
+1. Glob: find the entry files of the component's repository (choose the pattern by language)
+   Go:         main.go / cmd/*/main.go
+   Python:     main.py / app.py / manage.py / wsgi.py
+   Java:       *Application.java / *Bootstrap.java / src/main/java/**/Main*.java
+   TypeScript: app.ts / index.ts / main.ts / server.ts
+   Rust:       main.rs / src/main.rs
+   
+2. Grep: locate the core Handlers/Routers (choose the pattern by language + framework)
+   Go:         grep -rn 'func.*Handler\|\.GET\|\.POST\|router\.\|@handler' <dir>
+   Python:     grep -rn '@app\.\|@router\.\|def.*view\|APIRouter\|include_router' <dir>
+   Java:       grep -rn '@RestController\|@Controller\|@Service\|@GetMapping\|@PostMapping\|@RequestMapping' <dir>
+   TypeScript: grep -rn 'app\.get\|app\.post\|router\.\|@Get\|@Post\|@Controller' <dir>
+   Rust:       grep -rn '\.route\|\.get\|\.post\|#\[get\|#\[post\|async fn' <dir>
+
+   ⚠️ Exclude test files: --exclude='*_test.*' --exclude='test_*' --exclude='*_mock.*'
+   
+3. Read: read the core files (by the directory value rating in architecture_map)
+   - ⭐⭐⭐ Must read: business logic layer, core config files, DDL
+   - ⭐⭐ Reference: service context initialisation, config files
+   - ⭐ Skippable: pure binding layers (usually just parameter pass-through)
+   - ✗ Forbidden: generated files (*.pb.go, *_gen.go, *_generated.*, node_modules/, target/, build/)
+```
+
+Extract the following information (**everything must cite a code file:line, no inference**):
+- Core responsibility (one sentence, <=30 words)
+- Architecture layer and upstream/downstream components (communication method: RPC/MQ/DB)
+- Code entry (file name -> core function name)
+- Core mechanisms (the 1~2 most important technical mechanisms)
+- Data flow (where from -> what it passes through -> where to)
+- Tech stack (language + framework + middleware)
+- Data model (tables involved + key DDL fields)
+- Core flows (the steps needed for sequence diagrams)
+- Config items (config key + default value + impact scope)
+- Scheduled tasks (if any)
+- Monitoring metrics (if any)
+
+Mark content that cannot be found in the code as `[UNVERIFIED]`; do not infer.
+
+#### 2B-2: Product documentation extraction (Type-5/6/7, or when product_docs_dir is set)
+
+If `product_docs_dir` is not empty:
+```
+Scan dimensions (from phase2-document-types.md §Type-5 bridge document generation method):
+├── Quantity limits (batch caps, quotas, maximums)
+├── Type constraints (enum values, mutual exclusions)
+├── State preconditions
+├── Billing rules
+├── Security constraints
+└── Compatibility constraints
+```
+
+Trace every product constraint to its validation location in the code (the exact file:line of the `if len() > N`).
+
+#### 2B-3: Document generation
+
+Generate documents following the template for the corresponding type in `phase2-document-types.md`.
+
+**Type-4 component documents must contain (in order)**:
+
+```markdown
+# {component} Internal Design
+<!-- search-anchor: {full name}, {short name}, {abbreviation}, {synonyms}, {common search terms} -->
+> Project: {project_name} | Repository: {repo URL} | Architecture layer: {layer}
+> Position in the overall architecture: [📘 {project_name} Technical Architecture - 4.X {component}](./{project_name} Technical Architecture.md#4x-component)
+
+## 🤖 AI Quick Reference
+| Dimension | Key information |
+|------|---------|
+| **Core responsibility** | {<=30 words, specific} |
+| **Architecture layer** | {layer name} → {role} |
+| **Upstream components** | {ComponentA(RPC)}, {ComponentB(MQ)} |
+| **Downstream components** | {ComponentC(RPC)}, {ComponentD(DB)} |
+| **Code entry** | `{file name}` → `{core function name}()` |
+| **Core mechanisms** | {mechanism 1}; {mechanism 2} |
+| **Mutual exclusion** | {concurrency control method, e.g. "distributed lock key: xx"} |
+| **Data flow** | {source} → {processing} → {destination} |
+| **Tech stack** | {language} + {framework} + {middleware} |
+| **Scheduled tasks** | {N scheduled tasks, or "none"} |
+
+## 📋 Project Overview
+(numbered list of core responsibilities + ASCII architecture position diagram)
+
+## 🏗️ Architecture Design
+(ASCII architecture diagram + core sub-module descriptions + core function signatures)
+
+## 📊 Data Model
+(SQL DDL with comments + data flow diagram)
+
+## 🔌 Interface Design
+(external/internal interface tables + error code definitions)
+
+## ⚙️ Core Flows
+(mermaid sequence diagrams + step descriptions + exception handling)
+
+## 🔧 Configuration
+(config item / default value / description / impact scope)
+
+## 📈 Monitoring and Alerting
+
+## 🐛 Common Issues and Troubleshooting
+
+## 📝 Document Change Log
+### v1.0 ({date})
+- ✅ **Added**: initial version
+> Code baseline: {commit_sha} ({tag})
+```
+
+**Write all documents under `output_dir`; printing the full content in the conversation before writing the file is forbidden.**
+
+### Step 3: Self-check (accuracy verification + interface reconciliation)
+
+Run after each document is generated; **must not be skipped**:
+
+**Structural completeness**:
+- [ ] All 10 dimensions of the AI Quick Reference table filled in, each with specific information (not "see below")?
+- [ ] "Code entry" precise to the function name (`file name:line → function()`)?
+- [ ] search-anchor has 5~15 keywords, including full and short names and synonyms?
+- [ ] Contains a bidirectional link to the main architecture document?
+- [ ] No empty placeholder sections (delete sections with no content)?
+
+**Interface reconciliation** (only for components whose interface verification type in architecture_map is not NONE):
+
+Read the component's scanned baseline count `scanned` from `_review/interface-inventory.json` and count the interfaces actually recorded in the document as `documented`:
+
+```
+HTTP type:  count the routes listed in the document's ## Interface Design section
+MQ type:    count the Topics/Queues/Exchanges explicitly recorded in the document
+RPC type:   count the RPC Methods listed in the document
+```
+
+Compute the difference: `gap = scanned - documented`
+
+Handling rules:
+- `gap = 0`        → ✅ interface coverage complete
+- `0 < gap <= 20%`  → ⚠️ minor gap, append `<!-- INTERFACE_GAP: N interfaces possibly missing -->` at the end of the document
+- `gap > 20%`      → ❌ mark `[INTERFACE_GAP]`, note it in the summary, recommend supplementing and rerunning
+
+Update the component's `interface_coverage.documented` field in `progress.json`.
+
+**Accuracy statistics** (computed per document and returned to the main agent for aggregation):
+```
+Method:
+  total_claims = business rule count + core flow step count + interface description count + config item count
+  verified     = those with a file:line reference
+  unverified   = those marked [UNVERIFIED]
+  ratio        = unverified / total_claims
+```
+
+Handling rules:
+- `ratio > 20%` → add `⚠️ Low-confidence warning: {unverified}/{total_claims} items cannot be traced to code` at the top of the document
+- `ratio > 40%` → mark **[HIGH_UNVERIFIED]** in the summary and recommend focused manual confirmation
+
+### Step 4: Return summary
+
+Return to the main agent (the main agent accumulates the data into `accuracy_stats` and `interface_coverage` in progress.json):
+
+```
+Batch completion summary:
+Files read: {N} (estimated token usage: ~{N}k)
+Documents generated: {N}
+
+Accuracy statistics:
+  Total claims: {N} | Verified: {N} | [UNVERIFIED]: {N} ({X}%)
+
+Interface reconciliation (components with interfaces only):
+  ComponentA [HTTP]: documented {M} / baseline {N} = {X}%  ✅/⚠️/❌
+  ComponentB [MQ]:   documented {M} / baseline {N} = {X}%  ✅/⚠️/❌
+
+Per-document details:
+  - {component}_Design.md: {N}KB, {N} claims, [UNVERIFIED] {N} ({X}%)  [HIGH_UNVERIFIED/INTERFACE_GAP if applicable]
+
+Skipped (already completed): {N}
+Issues found: {issue description or "none"}
+```
+
+## Output
+
+```
+<output_dir>/XX_{component}_Design.md  ← Type-4 component document
+<output_dir>/{project_name} Technical Architecture.md  ← Type-1 (if included in this batch)
+<output_dir>/{project_name} Business Architecture.md  ← Type-2
+<output_dir>/{project_name} Deployment Architecture.md  ← Type-3
+<output_dir>/XX_{project_name}_Core_API_Product_Code_Mapping.md  ← Type-5
+<output_dir>/XX_{project_name}_Product_Rules_Cheat_Sheet.md  ← Type-6
+<output_dir>/XX_{project_name}_Business_Development_SOP.md  ← Type-7
+<output_dir>/{knowledge_enhancement_doc}.md  ← Type-8
+Returned summary string
+```
+
+## Constraints
+
+- **Code is the truth**: every description must cite a code file; unverifiable content must be marked `[UNVERIFIED]`
+- **Templates are mandatory**: read the template for the corresponding section before generating each file type
+- **No empty documents**: do not create a file without substantive content
+- **No redundant output**: Write files directly; do not print the full content in the conversation
+- **Naming convention**: component documents use `XX_{component}_Design.md`; XX is assigned in dependency-chain order (lower-layer components get smaller numbers)
+- **When no API is provided**: Type-5/6 may skip the product constraint mapping and mark constraint values as `[PRODUCT_DOC_MISSING]`

package/skill-data/wiki/references/methodology/phase0-collection.md

@@ -0,0 +1,54 @@
+# Phase 0: Source Material Collection and Preprocessing
+
+## Repository Discovery and Classification
+
+Starting from the entry repository, recursively discover all related repositories:
+
+1. **Dependency analysis**: parse project dependency files (such as `requirements.txt`, `package.json`, `pom.xml`, `Cargo.toml`, `go.mod`, chosen by the detected language)
+2. **Configuration references**: parse module names referenced in workflow orchestration configs → repository mapping
+3. **RPC service discovery**: extract service names from service registry configs → repository mapping
+4. **Classify by architecture layer**: API access layer / workflow engine layer / service execution layer / resource scheduling layer / data adapter layer / base execution layer
+5. **Mark core-ness**: compute priority from lines of code, number of dependents, and Handler count
+
+## Key File Extraction Checklist
+
+| File type | Match pattern | Extraction purpose |
+|---------|---------|---------|
+| **Entry files** | `main.py`, `main.go`, `cmd/*/main.go`, `app.ts` | Service startup and initialization flow |
+| **Routes/Handlers** | `handler.*`, `router.*`, `controller.*` | API endpoints and message handling entry points |
+| **Config files** | `*config*.*`, `conf/`, `*.yaml`, `*.toml` | Workflow orchestration, parameter configuration |
+| **Proto/IDL** | `*.proto`, `*.thrift`, `*schema*` | RPC interface contracts and data structures |
+| **Database operations** | `*db*.*`, `*dao*.*`, `*model*.*`, `*repository*.*` | Data models and table schemas |
+| **Constants/error codes** | `*const*`, `*error*`, `*code*`, `*enum*` | Error code system and business constants |
+| **Test files** | `*_test.*`, `test_*.*` | Expected behavior and edge conditions |
+
+## Building the Code Knowledge Graph
+
+Before generating documents, build a code knowledge graph as an intermediate representation:
+
+**Node types**: `[Service]` / `[Handler]` / `[Config]` / `[Table]` / `[Queue]` / `[API]` / `[ErrorCode]`
+
+**Edge types**: `[CALLS]` (synchronous RPC/HTTP) / `[PUBLISHES]` (asynchronous MQ) / `[CONSUMES]` (MQ consumption) / `[READS]` (DB read) / `[WRITES]` (DB write) / `[CONFIGURES]` (config-driven) / `[MAPS_TO]` (product → code)
+
+**Construction methods** (ordered by availability):
+1. **`teamai codebase --extract`**: Tree-sitter structural edges (**TS/JS/Python/Go** and more) + multi-language heuristic fact pages (writes `teamwiki/`)
+2. Grep + Read (Agent K1/K2): supplement dynamic routes and config-driven calls
+3. Parse orchestration configs → module → command mapping
+4. Parse Proto/IDL/DDL → data structures and table relationships (structured files, can be parsed precisely)
+5. MQ topology inference → Exchange/Topic/Queue/Routing Key
+6. API mapping → external API name → internal Handler entry point
+
+> `code-ast` can produce `DEPENDS_ON` edges for relative imports; package-level and dynamic calls may still be missed, mark them `[UNVERIFIED]` or `AMBIGUOUS`.
+> AST results take precedence over heuristics. There is no separate capabilities doc in this package; use `teamai codebase --extract` output under `teamwiki/`.
+
+## Input Source Priority
+
+| Priority | Input source | Specific content | Output document types |
+|--------|--------|---------|------------|
+| **P0 required** | Code repositories | Directory structure, entry files, configs, Proto | Type-1,4 |
+| **P0 required** | Workflow orchestration configs | workflow_config / state machines | Type-1,4,5 |
+| **P0 required** | Product API docs | Interface parameters, error codes | Type-5,6 |
+| **P1 important** | Database schema | DDL, table schemas | Type-4 |
+| **P1 important** | Product usage docs | Usage limits, FAQ | Type-6,8a |
+| **P2 enhancement** | Git history | Commit/MR records | Type-8b |
+| **P2 enhancement** | Incident records | Incident reports | Type-8d |

package/skill-data/wiki/references/methodology/phase1-reverse-engineering.md

@@ -0,0 +1,89 @@
+# Phase 1: Architecture Reverse-Engineering, From Code to Architectural Understanding
+
+## 1. Bottom-Up Layering Method
+
+```
+Step 1: Identify "leaf nodes" that operate directly on infrastructure
+  ├── Database operations (MySQL/PostgreSQL/Redis/MongoDB)
+  ├── Message queue operations (RabbitMQ/Kafka/RocketMQ)
+  ├── External system calls (third-party APIs / low-level drivers)
+  └── File/object storage operations (S3/OSS/COS)
+
+Step 2: Identify "intermediate nodes" that orchestrate and route
+  ├── Message routing frameworks (consumer routing and dispatch)
+  ├── Task schedulers (cron jobs / delayed tasks)
+  ├── Workflow orchestration engines (Workflow/Saga/state machines)
+  └── Resource schedulers (load balancing / resource allocation)
+
+Step 3: Identify "root nodes", the external entry points
+  ├── API gateway / HTTP Handler / gRPC Server
+  ├── Scheduled task entry points (Cron/Scheduler)
+  └── Event listener entry points (Webhook/EventBus)
+
+Step 4: Layer by call direction
+  External entry → workflow orchestration → service execution → resource scheduling → data operations → infrastructure
+```
+
+### Layer Assignment Rules
+
+| Distinguishing feature | Layer | Typical code pattern |
+|---------|---------|-------------|
+| HTTP/gRPC Server startup | API access layer | `http.ListenAndServe()`, `grpc.NewServer()` |
+| Parameter validation + auth + rate limiting | API access layer | `validate()`, `auth()`, `rateLimit()` |
+| Workflow step configs and state machines | Workflow engine layer | `workflow_config`, `state_machine` |
+| MQ consumption + Handler routing | Service execution layer | `channel.consume()`, `handler.dispatch()` |
+| Scheduling algorithms (Filter/Score) | Resource scheduling layer | `filter()`, `score()`, `schedule()` |
+| DB CRUD + cache operations | Data adapter layer | `db.query()`, `redis.get()` |
+| Low-level system calls/drivers | Base execution layer | `exec()`, `syscall.*`, `driver.*` |
+
+## 2. Three-Layer Penetration Tracing (Core Methodology)
+
+For every user-visible API operation, complete a three-layer penetration trace:
+
+```
+Layer 1: API entry layer
+  ├── Locate the Handler function
+  ├── Extract parameter validation logic
+  ├── Identify hard-coded defaults and whitelists
+  └── Determine the downstream call style (synchronous RPC / asynchronous MQ)
+
+Layer 2: Workflow orchestration layer
+  ├── Find the workflow config (workflow_config / saga_config)
+  ├── Parse the step sequence (step name / execution module / rollback module / timeout / retry)
+  ├── Annotate the execution module and rollback module of each step
+  └── Determine how data is passed between steps
+
+Layer 3: Service execution layer
+  ├── Trace the concrete Handler implementation of each step
+  ├── Identify database operations and state changes
+  ├── Annotate external system calls
+  └── Determine the callback path of the final execution result
+
+Output: complete call chain sequence diagram + state transition diagram + data flow diagram
+```
+
+### Standard Format for Documenting Call Chains
+
+```
+[API name](code entry: {repo}/{path}/{file})
+  → parameter validation + auth and rate limiting
+  → [pre-checks]: {check content}
+  → RPC/MQ → [orchestration layer] ({config file}: {operation name})
+    → [service layer] ({config file}: {flow_name})
+      → [{step 1 module}] {step 1 command} ({details})
+      → [{step 2 module}] {step 2 command} ({details})
+      → ...
+      → callback to [orchestration layer]
+```
+
+## 3. Component Relationship Matrix
+
+Build an N×N relationship matrix annotated with the communication style:
+
+| Caller ↓ / Callee → | ComponentA | ComponentB | ComponentC |
+|---------------------|-------|-------|-------|
+| **ComponentA** | — | RPC | MQ |
+| **ComponentB** | — | — | DB |
+| **ComponentC** | RPC | MQ | — |
+
+Legend: `RPC` (synchronous) / `MQ` (asynchronous) / `DB` (shared database) / `—` (no direct communication)