teamai-cli 0.25.0 → 0.26.0-beta.0

package/skill-data/wiki/references/phases/k4-quality.md

@@ -0,0 +1,190 @@
+## Phase K4: Knowledge Base Quality Assessment and Report
+
+**Methodology**: `{SKILL_DIR}/references/methodology/phase4-quality.md`
+
+### Step 1: Automated validation
+
+```bash
+python3 "{SKILL_DIR}/scripts/validate_kb.py" <output_dir> --verbose
+```
+
+`--verbose` prints the details of every item (missing anchors, the exact location of dead links). This is exactly the full output required below.
+
+Output (**must be shown in full, not only the passing items**):
+```
+Link integrity:          ✅/❌  N dead links
+search-anchor:           ✅/⚠️  coverage N/M (X%)
+AI Quick Reference table: ✅/⚠️  coverage N/M (X%)
+Bidirectional links:     ✅/⚠️  coverage N/M (X%)
+README index:            ✅/⚠️  inclusion rate N/M (X%)
+```
+
+### Step 2: Accuracy audit
+
+Aggregate the credibility of the whole knowledge base from `accuracy_stats`, and the interface coverage from `interface_coverage`:
+
+```
+[Content accuracy]
+Total claims:                 N (business rules + interface descriptions + relationships)
+Verified (with code reference): N (X%)
+[UNVERIFIED]:                 N (X%)
+AMBIGUOUS relationships:      N (X%)
+
+[Interface coverage] (only HTTP/MQ/RPC type components are counted, NONE type is excluded)
+HTTP interfaces: documented M / scan baseline N = X%
+MQ Topics:       documented M / scan baseline N = X%
+RPC Methods:     documented M / scan baseline N = X%
+Overall coverage: X%    target ≥ 90%
+
+⚠️ Interface gap list (components where documented < scan baseline):
+  - ComponentA: documented 8, scan baseline 13, gap 5 → recommend adding
+```
+
+⚠️ Manual confirmation list: (documents with [UNVERIFIED] > 20% + components with interface gaps + AMBIGUOUS relationships)
+
+### Step 3: RAG retrieval spot check
+
+Following `phase4-quality.md §RAG Retrieval Test Cases`, test 1 question from each of the 7 question types (see the methodology for details) and record the hit rate.
+
+### Step 4: AI end-to-end validation (E2E Validation)
+
+**Core idea**: answer a set of standardised questions using the knowledge base, then **trace back to the code to verify the answers**, to detect whether the knowledge base enables the AI to give correct answers.
+
+```
+Step 4A: Generate the standard validation question set (automatic, based on existing documents)
+
+  **Prefer an external validation set provided by the user**:
+  IF the user provided a list of validation questions (3~10 real business questions) in Phase 0 or now:
+    → use the user's questions as the validation set first (source: USER)
+    → top up automatically to 10~15 questions (source: AUTO)
+  ELSE:
+    → generate all automatically (source: AUTO)
+  
+  > User-provided questions are more valuable, because when the AI writes its own questions it tends to test areas it already knows,
+  > and the real blind spots (things the AI did not understand and is unaware of) can only be found by external questions.
+
+  Automatically generate 10~15 validation questions from k1-architecture-map.md and k2-doc-list.md:
+
+  Question type distribution (cover at least the following 5 types):
+
+  ┌────────────────────────────────────────────────────────────────────┐
+  │ Type 1: component responsibility (3 questions)                     │
+  │   Pattern: "What is the core responsibility of <component>? Where is the code entry point?" │
+  │   Verification: the function / file names in the answer must exist in the code │
+  │                                                                    │
+  │ Type 2: call relationships (3 questions)                           │
+  │   Pattern: "What is the relationship between <component A> and <component B>? How do they communicate?" │
+  │   Verification: the answer matches the G1 matrix + the actual imports / calls in the code │
+  │                                                                    │
+  │ Type 3: operation constraints (2 questions)                        │
+  │   Pattern: "Can <operation Y> be executed in <state X>?"           │
+  │   Verification: the answer matches the G9 constraint matrix + the state checks in the code │
+  │                                                                    │
+  │ Type 4: data flow (2 questions)                                    │
+  │   Pattern: "Which tables / queues does <operation Z> ultimately write to?" │
+  │   Verification: the answer matches the G3 data flow + the actual SQL / MQ operations in the code │
+  │                                                                    │
+  │ Type 5: error troubleshooting (2 questions)                        │
+  │   Pattern: "What does error code <XXX> mean? Which component produces it?" │
+  │   Verification: the answer matches the G4 error code map + the error definitions in the code │
+  │                                                                    │
+  │ Type 6 (optional): knowledge boundary test (2 questions)           │
+  │   Pattern: deliberately ask about content the knowledge base does not cover (e.g. third-party SDK internals, historical architecture changes) │
+  │   Verification: the AI should answer "outside the knowledge base coverage" rather than hallucinate │
+  └────────────────────────────────────────────────────────────────────┘
+
+Step 4B: Answer using the knowledge base (simulating the AI usage scenario)
+
+  FOR each validation question:
+    1. Assume only the knowledge base documents can be read, not the code directly
+    2. Find the relevant document following the retrieval routing rules
+    3. Extract the answer from the document
+
+Step 4C: Code trace-back verification
+
+  FOR each answer:
+    1. Verify the key claims directly in the code with Grep/Read
+    2. Judge the result:
+       ✅ CORRECT       : the answer matches the code
+       ⚠️ PARTIAL       : the answer is partially correct, with omissions or imprecision
+       ❌ INCORRECT     : the answer contradicts the code
+       🔇 BOUNDARY_OK   : knowledge boundary question, correctly declined to answer (type 6 only)
+       🔇 BOUNDARY_FAIL : knowledge boundary question, wrongly gave an answer (type 6 only)
+
+Step 4D: Write the validation report
+
+  Append to the ## AI End-to-End Validation section of k4-quality-report.md:
+
+  | Question | Type | Retrieved Document | AI Answer Summary | Code Verification | Result |
+  |------|------|---------|-----------|---------|------|
+  | Core responsibility of Aurora? | Component responsibility | 03_Aurora_Design.md | Scheduling orchestration... | scheduler.go:42 | ✅ |
+  | A→B communication method? | Call relationship | G1 matrix | RPC | import rpc_client | ✅ |
+  | Can operation Y run in state X? | Operation constraint | G9 matrix | No | check_state.go:88 | ✅ |
+  | Third-party SDK internals? | Knowledge boundary | — | Out of scope | — | 🔇 OK |
+
+  Statistics:
+    CORRECT: N/M (X%)
+    PARTIAL: N/M (X%)
+    INCORRECT: N/M (X%), ❌ every INCORRECT must list the specific contradiction
+    BOUNDARY_OK: N/N
+    BOUNDARY_FAIL: N/N
+
+    E2E accuracy = (CORRECT + BOUNDARY_OK) / total questions
+    Target: ≥ 80%
+```
+
+**If E2E accuracy < 80%**: list the documents that need improvement and the specific problems in the "Recommendations" section of the quality report.
+
+### Step 5: Generate the quality report
+
+Write to `_review/k4-quality-report.md`:
+
+```markdown
+# Knowledge Base Quality Report
+
+## Overview
+- Code baseline: <commit SHA> (<tag>)
+- Generated at: <ISO8601>
+- Total documents: N (Type-1~8: N, graph G1~G9: 9)
+
+## Accuracy
+| Metric | Value | Status |
+| Total claims | N | — |
+| With code reference | N (X%) | ✅/❌ |
+| [UNVERIFIED] | N (X%) | ✅/<15% / ⚠️15~25% / ❌>25% |
+| AMBIGUOUS relationships | N | ✅/⚠️ |
+
+## Structural Quality (validate_kb.py output)
+(shown in full, no numbers hidden)
+
+## Cross-Document Consistency (summary of k3-consistency-check.md)
+| Metric | Value | Status |
+| Contradictions | N | ✅=0 / ❌>0 |
+| Missing references | N | ⚠️ |
+| G1 deviations | N | ⚠️ |
+| Consistency rate | X% | target ≥95% |
+
+## RAG Retrieval Spot Check
+| Test Question | Expected Hit | Actual Hit | Result |
+
+## AI End-to-End Validation
+| Metric | Value | Status |
+| CORRECT | N/M (X%) | — |
+| PARTIAL | N/M (X%) | ⚠️ |
+| INCORRECT | N/M (X%) | ❌ |
+| BOUNDARY_OK | N/N | ✅ |
+| E2E accuracy | X% | target ≥80% |
+
+INCORRECT details:
+(the specific contradiction and improvement suggestion for every INCORRECT)
+
+## Manual Confirmation List
+([UNVERIFIED] over-threshold documents + AMBIGUOUS relationships + contradictions + dead links)
+
+## Recommendations
+(improvement directions based on the consistency check + E2E validation)
+```
+
+**When done**: update `current_phase` to `"completed"`. The workflow ends.
+
+---

package/skill-data/wiki/references/phases/phase0-init.md

@@ -0,0 +1,112 @@
+## Phase 0: Initialisation
+
+Ask the user for all of the following in one go (**a single message, not step by step**):
+
+1. **Paths of all code repositories of the project** (the user lists every repository the project involves):
+   - Format: one absolute path per line, or comma-separated
+   - Example:
+     ```
+     /path/to/api-gateway
+     /path/to/order-service
+     /path/to/user-service
+     /path/to/common-lib
+     ```
+   - Note: this is the most critical step. The code of a large project is spread across many repositories, and **all of them** must be provided to build complete architecture awareness. A missing repository = a blind spot in the knowledge base.
+2. **Project name** (used in document names, e.g. "CVM", "E-commerce Platform")
+3. **Product documentation sources** (optional; when provided, the Type-5/6 bridge documents are generated):
+   - API documentation directory path
+   - Usage limits / FAQ document path
+4. **Output path** (default: `knowledge/` under the parent directory of the first repository)
+
+**Step 0A: Repository inventory**
+
+After receiving the user's repository list, build the repository inventory:
+
+```
+FOR each path provided by the user:
+  1. Verify the path exists and is accessible
+  2. Detect whether it is a git repository (has a .git directory)
+  3. Detect the primary language (by file extension distribution)
+  4. Measure code size (file count + estimated line count)
+  5. Record the git commit SHA + tag
+
+Write the result to _review/repo-manifest.json:
+{
+  "repos": [
+    {
+      "path": "/absolute/path/to/repo-a",
+      "name": "repo-a",
+      "language": "go",
+      "files": 320,
+      "lines_estimate": 45000,
+      "commit": "abc123",
+      "tag": "v1.2.0",
+      "accessible": true
+    },
+    ...
+  ],
+  "total_repos": N,
+  "inaccessible": ["path/to/repo-x (permission denied)"]
+}
+```
+
+Show it to the user for confirmation:
+```
+Identified {N} repositories:
+  ✅ repo-a (Go, ~45K lines)
+  ✅ repo-b (Python, ~12K lines)
+  ✅ repo-c (Go, ~28K lines)
+  ❌ repo-x (path does not exist or is not accessible)
+
+Total: ~{N}K lines of code, {N} repositories
+Reply "continue" if this is correct, or add the missing repositories.
+```
+
+**Step 0B: Auto-detect the primary language** (aggregated over the repository list, does not block the flow):
+```
+Detection method: aggregate the file extension distribution of all repositories
+  .go files dominate              → language: "go"
+  .py files dominate              → language: "python"
+  .java files dominate            → language: "java"
+  .ts/.js files dominate          → language: "typescript"
+  .rs files dominate              → language: "rust"
+  Mixed languages (no clear majority) → language: "mixed"
+Note: the language field selects the grep patterns for the interface scan (see Phase K1 Step 5)
+```
+
+**Step 0C: Record the baseline version**:
+```bash
+# Record each repository separately
+FOR repo in repos:
+  git -C <repo.path> rev-parse HEAD 2>/dev/null
+  git -C <repo.path> describe --tags --always 2>/dev/null
+```
+Write to `_review/metadata.json`:
+```json
+{
+  "project_name": "CVM",
+  "scan_time": "<ISO8601>",
+  "repos": [
+    {"name": "repo-a", "commit": "<sha>", "tag": "<tag>"},
+    {"name": "repo-b", "commit": "<sha>", "tag": "<tag>"}
+  ]
+}
+```
+
+**Step 0D: CLI structural baseline (per code repository, recommended)**
+
+Before the K1 deep read, use TeamAI to extract evidence-backed import/call structural edges (Python/Go/TS etc., `code-ast`) and merge them with the regex baseline (`code-heuristic`):
+
+```bash
+# For each repo. Writes <repo>/teamwiki/ (evidence pages + .indices/graph-index.json).
+# Existing flags only: --extract [path], optional --project <slug>, optional --incremental.
+teamai codebase --extract <repo_abs_path> --project <project_slug>
+```
+
+- Output: `teamwiki/evidence/code/<project>/` pages; `teamwiki/.indices/graph-index.json` (structural edges).
+- When K1/K2/K3 write `edges[]` in `_manifest.json`: **prefer citing** the `code-ast` edges from extract + their `evidenceRefs` (`path:line`); label Agent inferences `INFERRED`/`AMBIGUOUS`.
+- After Phase K3, skip any extra graph compile / merge step that is not a `teamai` command. TeamAI does not ship a separate team-wiki CLI. Continue with this skill using `teamai` and the files under this skill directory. No extra plugin is required.
+
+Write the initial progress.json (current_phase: "phase0_done") and enter **Phase K1**.
+
+---

package/skill-data/wiki/references/templates/project-overview.md

@@ -0,0 +1,148 @@
+# Knowledge base overview template
+
+> Used to generate `<output_dir>/README.md`, produced in Phase K2 batch 5 (the top-level index of the knowledge base).
+
+```markdown
+# <Project name>: Deep Knowledge Base
+<!-- search-anchor: <project name>, <project English name>, knowledge base, architecture overview, quick navigation, component documents, Graph RAG, graph -->
+
+> **AI reading guide**: this directory is an AI-Native knowledge base. Read this file first for the global picture and the cognitive boundaries,
+> then follow the retrieval routing rules into the relevant document for details. **Never read the whole knowledge base directory at once.**
+
+## 🤖 Knowledge Base Retrieval Routing Guide (for AI)
+
+### Quick navigation by question type
+
+| I want to know... | Read... | Path |
+|---------|---------|------|
+| Overall system architecture and layering | Technical architecture document | `./{project_name} Technical Architecture.md` |
+| Design and implementation of a component | Component design document | `./XX_{component}_Design.md` |
+| Dependencies between components | G1 dependency matrix | `./graph/G1_*.md` |
+| Which modules an API passes through | G2 call chain overview | `./graph/G2_*.md` |
+| Where data lives, MQ topology | G3 data flow | `./graph/G3_*.md` |
+| Which module an error code belongs to | G4 error code map | `./graph/G4_*.md` |
+| The full flow of a business scenario | G5 interaction scenarios | `./graph/G5_*.md` |
+| Who A depends on indirectly (multi-hop query) | G6 knowledge graph triples | `./graph/G6_*.md` |
+| Blast radius if component X goes down | G7 risk analysis | `./graph/G7_*.md` |
+| How to change a configuration | G8 config parameter index | `./graph/G8_*.md` |
+| Whether an operation is allowed | G9 business rule constraints | `./graph/G9_*.md` |
+| Product constraint → code location mapping | Core API mapping document | `./XX_*_Core_API_Product_Code_Mapping.md` |
+| Business development SOP | Business development guidelines | `./XX_*_Business_Development_SOP.md` |
+
+### Retrieval rules
+
+- **Rule 1, index first, then dig in**: for an unfamiliar component, read this file first to find the right path, then go into the component document
+- **Rule 2, component-internal questions go to the component document**: core mechanisms, code entry points, data models → `XX_{component}_Design.md`
+- **Rule 3, cross-component relation questions go to the graph**: dependency matrix, call chains, impact surface → the `graph/` directory
+- **Rule 4, operation feasibility questions go to G9**: constraint matrix + decision tree → `graph/G9_*.md`
+- **Rule 5, content marked `[UNVERIFIED]` must not be used for code generation** until confirmed by a human
+- **Rule 6, `AMBIGUOUS` relations must not be used for change impact assessment** until clarified
+
+---
+
+## 🚧 Cognitive Boundary Declaration (AI must read)
+
+> This section declares what this knowledge base **does not know**. When a question touches the areas below, the AI
+> **must proactively tell the user "this information is outside the knowledge base coverage; check the source code / product docs / contact the team"**
+> instead of trying to infer or hallucinate.
+
+### Coverage
+
+| Dimension | Coverage | Notes |
+|------|------|------|
+| Code baseline | `<commit SHA>` (`<tag>`) | Changes **after** this version are not covered |
+| Generated at | `<YYYY-MM-DDTHH:MM:SSZ>` | Time anchor between the knowledge base and the code |
+| Core components (P0) | <P0 component list> | Deepest documentation, interface-level coverage |
+| Important components (P1) | <P1 component list> | Medium documentation depth, core mechanisms covered |
+| Auxiliary components (P2) | <P2 component list> | Limited documentation depth, architecture level only |
+
+### Explicitly not covered (AI should not attempt to answer)
+
+| Area | Reason |
+|------|------|
+| Internals of third-party SDKs/libraries | The knowledge base records only how they are called, not third-party source |
+| Operations/deployment details (ansible/k8s config) | Outside the scope of a codebase knowledge base; consult the operations docs |
+| Non-code deliverables (UI design, original product PRDs) | Only the Type-5/6 bridge documents map product constraints |
+| Historical architecture evolution | Only the architecture of the current code baseline is reflected |
+| Performance benchmark data | The knowledge base contains no load-test data |
+| <project-specific uncovered items> | <reason> |
+
+### Low-confidence areas (extra warning needed when answering)
+
+| Area | Reason | Recommendation |
+|------|------|------|
+| Internal details of P2 auxiliary components | Limited documentation depth | Add "based on limited documentation analysis" when citing |
+| Content marked `[UNVERIFIED]` | Cannot be traced back to code | Must tell the user "this information is not verified against code" |
+| `AMBIGUOUS` relations | Confidence < 0.3 | Must tell the user "this relation is uncertain" |
+| Type-5/6 when product docs are missing | No product doc input | Marked `[PRODUCT_DOC_MISSING]` |
+
+### Knowledge base update notes
+
+- **Incremental update**: `teamai codebase --extract <repo> --project <slug> --incremental` re-extracts only the changed files
+- **Full rebuild**: recommended after large-scale code refactoring
+- **Last updated**: `<ISO8601>`
+
+---
+
+## Project introduction
+
+<!-- 1-3 sentences: project background, core business goals, main users -->
+
+## Tech stack
+
+| Category | Technology | Notes |
+|------|------|------|
+| Language | Go / Python | ... |
+| Framework | go-zero / FastAPI | ... |
+| Database | MySQL / PostgreSQL | ... |
+| Cache | Redis | ... |
+| Message queue | Kafka / RabbitMQ | (if any) |
+
+## Knowledge base document index
+
+### Architecture-level documents
+| Document | Type | Size | Notes |
+|------|------|------|------|
+| {project_name} Technical Architecture.md | Type-1 | ~200KB | Architecture overview |
+| {project_name} Business Architecture.md | Type-2 | ~70KB | Product capabilities + lifecycle |
+| {project_name} Deployment Architecture.md | Type-3 | ~40KB | Deployment topology |
+
+### Component design documents
+| No. | Component | Layer | Priority | Size |
+|------|------|--------|--------|------|
+| 01 | <component> | <layer> | P0 | ~NKB |
+
+### Bridge documents (generated when product docs exist)
+| Document | Type | Notes |
+|------|------|------|
+| Core API Product Code Mapping | Type-5 | Product constraint → code location |
+| Product Rules Cheat Sheet | Type-6 | Usage limits / FAQ → code |
+| Business Development SOP | Type-7 | Development / change operation guidelines |
+
+### Graph document set (Graph RAG)
+| Document | Purpose | Size |
+|------|------|------|
+| G1~G9 | Cross-component relation index | See `graph/README.md` |
+
+## Knowledge base quality overview
+
+| Metric | Value | Status |
+|------|------|------|
+| Total documents | N | - |
+| Content accuracy (with code references) | X% | ✅/⚠️ |
+| [UNVERIFIED] ratio | X% | Target <15% |
+| Interface coverage (non-NONE components) | X% | Target ≥90% |
+| AMBIGUOUS relation count | N | Needs human confirmation |
+
+> See `_review/k4-quality-report.md` for the detailed quality report
+
+## Code baseline version
+
+> ⚠️ This knowledge base was generated from the code version below. After the code evolves, run `teamai codebase --extract <repo> --project <slug> --incremental` for an incremental update.
+
+- **Commit**: `<git commit SHA>`
+- **Tag**: `<tag or "no tag">`
+- **Generated at**: `<YYYY-MM-DDTHH:MM:SSZ>`
+
+> Version information source: `_review/metadata.json`
+```