@c4a/extract 0.7.1 → 0.7.5

package/README.md

@@ -14,13 +14,12 @@ knowledge through the normal lifecycle.
 
 ## Package Role
 
-`@c4a/extract` is the protocol and runner layer used by declared Context
-project extraction phases such as `extractTs({ source, collection: "codegraph" })`.
+`@c4a/extract` is the protocol and runner layer used by Code Indexer Providers.
 
 - Language plugins implement `ExtractionPlugin` and return `ExtractionResult` v2.
 - The runner loads one or more plugins, scans repository modules, emits progress/module-error/summary events, and can build an optional low-level code snapshot payload.
-- `@c4a/context-cli` invokes the runner from `context run extract:<source>:codegraph` after the workspace declares an `extractTs` phase.
-- The project flow stages review candidates under `.tmp/context-runtime/lifecycle/candidates.jsonl` and keeps runtime candidate artifacts under `.tmp/context-runtime/extract/`; approved Markdown is written only by review/apply.
+- The selected Provider invokes the runner through the controlled Indexer workset and returns typed facts and artifacts.
+- Context stages Candidate state under `.tmp/context-runtime/`; approved Markdown is written only by Review apply and close.
 
 **Depends on:** `@c4a/core`, `web-tree-sitter`, `zod`
 
@@ -31,12 +30,12 @@ language plugins + repository runner
           ↓
 versioned raw code snapshot
           ↓
-Context candidates → review → approved knowledge
+Indexer facts → reader-oriented Candidates → approved knowledge
 ```
 
 Knowledge-workspace users normally reach this package through the installed
-Agent entry and a declared extraction phase. The runner protocol below is for
-extractor authors, Context maintainers, and project-owned adapters.
+Agent entry and selected Code Indexer Provider. The runner protocol below is
+for parser authors, Provider authors, and Context maintainers.
 
 ## Protocol Layers
 
@@ -108,10 +107,9 @@ Every plugin returns:
 
 ### 3. Repository Runner Protocol
 
-The package exposes `c4a-extract-code`, a low-level NDJSON runner. The current
-Context project flow normally reaches it through a declared `extractTs` phase
-and `context run extract:<source>:codegraph`; agents should not hand-build runner
-input for normal workspace operation.
+The package exposes `c4a-extract-code`, a low-level NDJSON runner. Code Indexer
+Providers may invoke it as an implementation detail; agents should not
+hand-build runner input for normal workspace operation.
 
 Input is JSON on stdin:
 
@@ -192,9 +190,8 @@ Minimum requirements:
 5. Emit `SymbolInfo[]` with stable `name`, `kind`, `visibility`, `file`, `line`, and `endLine`.
 6. Emit `RelationInfo[]` for imports and important type/inheritance/use edges.
 7. Keep paths module-relative inside the plugin; the repository runner prefixes them to repo-relative paths.
-8. Register the plugin in the runner input used by the project extraction
-   phase. User-facing workspaces should declare the matching phase in
-   `src/index.ts` and execute it through `context run <phase-id>`.
+8. Register the plugin in the Code Indexer Provider's runner configuration.
+   User-facing workspaces select the Provider through `src/indexers.yaml`.
 
 Example skeleton:
 
@@ -229,31 +226,30 @@ export class PythonPlugin implements ExtractionPlugin {
 }
 ```
 
-## Relationship to Context Project Extraction
+## Relationship to the Code Indexer
 
-`@c4a/extract` is upstream of the Context project extraction phase; it does not
-render approved knowledge itself.
+`@c4a/extract` is upstream of the Code Indexer Provider; it does not render
+approved knowledge itself.
 
 - `context source add repo <name> --local <repo-or-subdir>` registers the code
   source boundary.
-- `src/index.ts` declares `extractTs({ source: source("<name>"), collection: "codegraph" })`.
-- `context run extract:<name>:codegraph --dry-run` previews resolved modules,
-  candidate estimates, knowledge path examples, and module warnings.
-- `context run extract:<name>:codegraph` writes draft candidates for review. It
-  uses `.tmp/context-runtime/extract/` for candidate snapshots, source
-  fingerprints, and symbol indexes; that scratch directory is not committed
-  state.
+- `src/indexers.yaml` binds the source scope and reader requirements to a Code
+  Indexer Provider.
+- The current Route prepares a controlled workset and the Provider returns the
+  current Indexer result.
+- Context writes draft Candidates and recoverable runtime artifacts under
+  `.tmp/context-runtime/`; that directory is not committed state.
 - `context review html` and `context review apply` own human approval and
   approved Markdown materialization.
 - `context close`, `context verify`, and `context build` own final package
   readiness.
 
 That means language plugins affect published knowledge only through the current
-project flow: better symbols/relations produce better draft candidates, stable
+Indexer flow: better symbols/relations produce better draft Candidates, stable
 `repo:<source>#symbol:...` source refs, review evidence, approved Markdown, and
 package output.
 
-Approved codegraph Markdown localizes canonical refs as
+Approved code knowledge may localize canonical refs as
 `src-N#symbol:<file>:<symbol>:<kind>@<digest>`. The file segment is part of the
 deterministic evidence identity used by verification; agents copy the complete
 ref as an opaque token.

package/README.zh-CN.md

@@ -7,8 +7,8 @@
 Tree-sitter 解析工具。
 
 它不判断代码对产品或读者意味着什么，也不写入正式知识。语言插件只输出结构事实；
-Context 运行时将这些事实绑定到来源身份、暂存审核候选，并通过标准生命周期应用
-审核通过的知识。
+Code Indexer Provider 将这些事实组织成当前 Indexer Result；Context 负责来源身份、
+Candidate 状态和审核通过的知识。
 
 ## 在知识生产链中的职责
 
@@ -19,19 +19,19 @@ Context 运行时将这些事实绑定到来源身份、暂存审核候选，并
        ↓
 版本化原始代码快照
        ↓
-Context 候选 → 审核 → 正式知识
+Indexer 事实 → 面向读者的 Candidate → 正式知识
 ```
 
 - 语言 package 实现 `ExtractionPlugin` 并返回 `ExtractionResult` v2。
 - Runner 加载一个或多个插件，扫描仓库模块，输出 progress、module-error 和
   summary 事件，并可生成底层代码快照。
-- Context runtime 在知识项目声明提取阶段后调用 Runner。
-- 草稿候选和运行快照留在 `.tmp/context-runtime/`；只有 review apply 会写入正式
+- 选中的 Code Indexer Provider 通过受控 workset 调用 Runner。
+- Candidate 和运行快照留在 `.tmp/context-runtime/`；只有 Review apply/close 会写入正式
   Markdown。
 
-知识工作区用户通常通过已安装的 Agent 入口和声明好的提取阶段使用本包，不需要
-手工构造 Runner 输入。下面的协议主要面向提取器作者、Context 维护者和项目自有
-Adapter。
+知识工作区用户通常通过已安装的 Agent 入口和选中的 Code Indexer Provider 使用
+本包，不需要手工构造 Runner 输入。下面的协议主要面向解析器作者、Provider 作者
+和 Context 维护者。
 
 **依赖：** `@c4a/core`、`web-tree-sitter`、`zod`
 
@@ -99,8 +99,8 @@ denominator。
 
 ### 3. Repository Runner
 
-包暴露底层 NDJSON Runner `c4a-extract-code`。正常知识工作区由当前 Route 和已声明
-阶段调用它；Agent 不应手工生成 stdin Payload。
+包暴露底层 NDJSON Runner `c4a-extract-code`。Code Indexer Provider 可以把它作为
+实现细节调用；Agent 不应手工生成 stdin Payload。
 
 Runner 输出一行一个 JSON 对象：
 
@@ -148,20 +148,20 @@ Agent 将完整 `source_ref` 当作不透明 token 复制，不自行解析或
 5. 输出稳定的符号名、kind、visibility、文件和行范围；
 6. 输出导入、调用、类型、继承或使用关系；
 7. 插件内部保持 module-relative 路径，由 Runner 加上 repo-relative 前缀；
-8. 在项目提取阶段的 Runner 配置中注册插件。
+8. 在 Code Indexer Provider 的 Runner 配置中注册插件。
 
-## 与 Context 项目提取的关系
+## 与 Code Indexer 的关系
 
-`@c4a/extract` 位于项目提取阶段上游，不渲染正式知识。典型流程是：
+`@c4a/extract` 位于 Code Indexer Provider 上游，不渲染正式知识。典型流程是：
 
 1. 用户确认代码来源边界；
-2. `src/index.ts` 声明相应提取阶段；
-3. 当前 Route 先运行 dry-run，展示解析出的模块、候选数量和知识路径；
-4. 正式提取生成待审候选和来源指纹；
+2. `src/indexers.yaml` 将来源范围和读者需求绑定到 Code Indexer Provider；
+3. 当前 Route 准备受控 workset，Provider 返回当前 Indexer Result；
+4. Context 生成待审 Candidate 和可恢复运行态；
 5. review apply 将批准内容写入正式知识；
 6. close、verify 和 build 完成本轮并生成知识包。
 
-因此，更好的符号和关系会产生更可靠的候选、证据引用和知识包，但业务分类继续由
+因此，更好的符号和关系会产生更可靠的 Candidate、来源引用和知识包，但业务分类继续由
 Agent 与用户依据证据决定。
 
 ## 开发

package/bin/c4a-extract-code.js

@@ -13163,6 +13163,8 @@ function keyTokens(key) {
   return key.replace(/([a-z0-9])([A-Z])/gu, "$1-$2").replace(/[^A-Za-z0-9]+/gu, "-").toLowerCase().split("-").filter(Boolean);
 }
 function sensitiveKey(key, value) {
+  if (value === INDEXER_OUTPUT_REDACTION_MARKER)
+    return false;
   const tokens = keyTokens(key);
   if (tokens.length === 0)
     return false;
@@ -13204,12 +13206,12 @@ function replaceWithCount(value, pattern, replacement, count) {
 function redactKnownText(value, count) {
   let output = value;
   output = replaceWithCount(output, /-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----[\s\S]*?-----END (?:RSA |EC |OPENSSH )?PRIVATE KEY-----/gu, INDEXER_OUTPUT_REDACTION_MARKER, count);
-  output = replaceWithCount(output, /(\bauthorization\s*:\s*(?:bearer|basic)\s+)[^\s,;]+/giu, (_match, prefix) => `${prefix}${INDEXER_OUTPUT_REDACTION_MARKER}`, count);
+  output = replaceWithCount(output, new RegExp(`(\\bauthorization\\s*:\\s*(?:bearer|basic)\\s+)` + `(?!${escapeRegExp(INDEXER_OUTPUT_REDACTION_MARKER)})[^\\s,;]+`, "giu"), (_match, prefix) => `${prefix}${INDEXER_OUTPUT_REDACTION_MARKER}`, count);
   output = replaceWithCount(output, /([a-z][a-z0-9+.-]*:\/\/)[^\s/@:]+:[^\s/@]+@/giu, (_match, prefix) => `${prefix}${INDEXER_OUTPUT_REDACTION_MARKER}@`, count);
-  output = replaceWithCount(output, /([?&](?:access_token|refresh_token|api_key|password|secret)=)[^&#\s]+/giu, (_match, prefix) => `${prefix}${INDEXER_OUTPUT_REDACTION_MARKER}`, count);
+  output = replaceWithCount(output, new RegExp(`([?&](?:access_token|refresh_token|api_key|password|secret)=)` + `(?!${escapeRegExp(INDEXER_OUTPUT_REDACTION_MARKER)})[^&#\\s]+`, "giu"), (_match, prefix) => `${prefix}${INDEXER_OUTPUT_REDACTION_MARKER}`, count);
   const key = "(?:[A-Za-z0-9_.-]*(?:password|passwd|pwd|secret|token|credential|cookie)[A-Za-z0-9_.-]*|api[-_]?key|access[-_]?(?:key|token)|private[-_]?key|client[-_]?secret|authorization)";
   const assignment = `(?:=\\s*|:\\s+(?=\\S)|:\\s*(?=["']))`;
-  output = replaceWithCount(output, new RegExp(`((?:["']?${key}["']?)\\s*${assignment})(?:"(?:\\\\.|[^"])*"|'(?:\\\\.|[^'])*'|[^\\s,;}\\]]+)`, "giu"), (_match, prefix) => `${prefix}"${INDEXER_OUTPUT_REDACTION_MARKER}"`, count);
+  output = replaceWithCount(output, new RegExp(`((?:["']?${key}["']?)\\s*${assignment})` + `(?!["']?${escapeRegExp(INDEXER_OUTPUT_REDACTION_MARKER)})` + `(?:"(?:\\\\.|[^"])*"|'(?:\\\\.|[^'])*'|[^\\s,;}\\]]+)`, "giu"), (_match, prefix) => `${prefix}"${INDEXER_OUTPUT_REDACTION_MARKER}"`, count);
   return output;
 }
 function redactBlockedText(value, blocked, count) {
@@ -14926,7 +14928,7 @@ function selectedEntryFiles(input) {
     for (const entry of input.detected.entries) {
       const sourcePath = input.module.path === "." ? safeSourceRelativePath(entry.path) : safeSourceRelativePath(`${input.module.path}/${entry.path}`);
       if (!includedSourceFiles.has(sourcePath)) {
-        throw new Error(`Auto-detected entry is outside extractTs include: ${sourcePath}. Update include or configure entries in the Context project.`);
+        throw new Error(`Auto-detected entry is outside the parser include: ${sourcePath}. Update the Provider include or configured entries.`);
       }
     }
     return input.detected.entries;
@@ -14937,7 +14939,7 @@ function selectedEntryFiles(input) {
   }
   return sourceEntries.map((sourcePath, index) => {
     if (!includedSourceFiles.has(sourcePath)) {
-      throw new Error(`Configured extraction entry is missing or outside extractTs include: ${sourcePath}. Update entries or include in the Context project.`);
+      throw new Error(`Configured extraction entry is missing or outside the parser include: ${sourcePath}. Update the Provider entries or include.`);
     }
     const modulePath = moduleRelativeEntryPath(input.module.path, sourcePath);
     return {