dsh-data-domain-knowledge 0.1.0-alpha.14 → 0.1.0-alpha.16

package/README.md

@@ -3,7 +3,7 @@
 本地知识库插件：一个知识库对应一个目录。`knowledge.db` 保存 **SourceRaw**
 （原始可追溯文档）、轴、Knowledge、Event、wiki 卡片、版本与阶段检查点；
 `derived.db` 保存可重建的 **wikiNotes**、知识图谱和向量索引，`assets/` 保存图片与附件。
-知识库目录外的 `<agentRoot>/memory/<库 id>` 保存任务与正式输出、抽取轨迹、操作日志、访问审计与反馈。
+知识库目录外的 `<agentRoot>/memory/<库 id>` 保存任务与正式输出、三段轨迹（抽取 / 归一 / 物化）、操作日志、访问审计与反馈。
 默认使用本地 Python 推理：`BAAI/bge-small-zh-v1.5` 生成 512 维文本向量，
 `BAAI/bge-reranker-base` 对召回候选做交叉编码重排。部署入口必须完成环境自检，
 索引必须匹配当前模型；不满足时提示准备环境或构建检索索引。
@@ -17,7 +17,8 @@ Tita Agent 的固定数据根目录是 `$DSH_HOME/tita-agent`（默认 `~/.dsh/t
 ## 首次运行环境准备
 
 向量模型和重排模型在 Tita 设置 → 环境准备中选择，再安装 / 修复。
-系统用 DSH 自带的 Python 3.12 创建专属依赖环境，不另行下载 Python；再下载所选模型，
+系统用 DSH 自带的 Python 3.12 创建专属依赖环境（macOS 桌面上改用 uv 管理的独立 3.12：
+自带的那份带 hardened runtime，装不了 pip 上的 torch）；再下载所选模型，
 默认是 `BAAI/bge-small-zh-v1.5`（512 维）和 `BAAI/bge-reranker-base`，然后运行真实推理自检。阶段、错误和下载日志
 可查看；下载失败可以重试。就绪检查按所用能力执行：抽取、归一和物化只需要 LLM；构建向量索引需要向量模型；召回按启用的向量和重排能力检查。来源与历史内容仍可保存、读取。
 自检不单独检查 `sentencepiece`；依赖安装失败会显示安装错误，模型真实推理失败会显示对应模型的错误。
@@ -44,7 +45,7 @@ pnpm --filter dsh-data-domain-knowledge doctor
 | `overview` | Agent 概览 | 各设置的挂载节点 |
 | `intent` | 意图 | `intent_record` / `intent_store_view`、`/intent/*` 路由、`titaDataAgentIntent` 服务；会话审计留痕落 `<agentRoot>/intent/` |
 | `knowledge` | 知识库 | `recall_knowledge` 等工具名、`/knowledge/*` 与 `/convert` 路由、`tita-data-agent-knowledge` / `-convert` 设置 |
-| `memory` | 记忆 | 抽取轨迹、处理日志、`feedback.ndjson`、`access-audit.ndjson`，根在 `<agentRoot>/memory/<库 id>`；轴 / Knowledge 真源在主库 |
+| `memory` | 记忆 | 轨迹（抽取 / 归一 / 物化）、处理日志、`feedback.ndjson`、`access-audit.ndjson`，根在 `<agentRoot>/memory/<库 id>`；轴 / Knowledge 真源在主库 |
 | `eval` | 评测 | 问题集格式（开发期能力，不注册运行时入口） |
 
 意图的**会话审计留痕**落在 `<agentRoot>/intent/`：`intent.db` 保存当前记录与追加历史
@@ -342,14 +343,14 @@ HTTP `GET /knowledge/bases` 在返回前先刷新。
 | 段 | 装什么 | 删掉的后果 |
 | --- | --- | --- |
 | `domain_knowledge/` | 知识树，按知识库 id 分叉 | 丢数据 |
-| `memory/<知识库 id>/` | 任务与正式输出、抽取轨迹、处理日志、反馈与访问审计 | 丢历史，不可重建 |
+| `memory/<知识库 id>/` | 任务与正式输出、轨迹（抽取 / 归一 / 物化）、处理日志、反馈与访问审计 | 丢历史，不可重建 |
 | `intent/` | `intent.db`（当前记录 + 审计历史） | 丢审计流 |
 | `output/` | 交付物，二级按日期分桶 | 什么都不损失 |
 | `tmp/` | 运行期临时文件，二级按日期分桶 | 只影响未完成的任务 |
 | `resource/` | 模型、Python 环境、`knowledge-bases.json` | 失去语义检索（嵌入/重排不可用） |
 
 **记忆与意图为什么与知识树平级**：记忆装的是**关于这个 Agent 的动作留痕**（任务、正式输出、
-抽取轨迹、处理日志、审计流水），知识断言及其阶段检查点保存在 `knowledge.db`。把动作留痕放进知识树，会让「备份知识树」
+轨迹、处理日志、审计流水），知识断言及其阶段检查点保存在 `knowledge.db`。把动作留痕放进知识树，会让「备份知识树」
 被迫连审计一起备份，也让「知识树有哪几段」这个问题混进一个横切层。意图则是**会话级**的
 （同一句话在哪个知识库下说，理解方式不该变），所以它不分库。这样知识、记忆、意图三段
 都是可以直接备份、直接 diff 的用户文件。
@@ -362,7 +363,7 @@ HTTP `GET /knowledge/bases` 在返回前先刷新。
 | `derived.db` | wikiNotes、知识图谱、检索索引 | 可重新编译 |
 | `assets/` | 来源图片与附件 | 丢失对应素材 |
 
-知识库提供 `images.publish` 工具作为本地图床。它接收宿主本地 `file_path` 或 Base64 图片，按
+知识库提供 `image_publish` 工具作为本地图床。它接收宿主本地 `file_path` 或 Base64 图片，按
 `assets/<SHA-256 前 12 位>/<文件名>` 写入当前知识库，并返回 `/knowledge/assets/<kb>/<路径>` 地址
 和 Markdown 图片引用。独立图床图片不要求来源或卡片引用，也不会被孤儿清理误删。工具会校验图片格式和 MIME，
 单张图片上限为 20 MiB；写入不需要用户审批。
@@ -401,7 +402,7 @@ POST   /knowledge/bases        { name, description?, rootDir? }    → { base }
 GET    /knowledge/models?kb=:id                                   → DSH 编译模型目录与当前默认模型状态
 PATCH  /knowledge/bases/:id    { name?, description?, extractionNote?, compileModel? } → { base }
 DELETE /knowledge/bases/:id                                        → { deleted: true }
-GET    /knowledge/documents?kb=&query=&limit=&offset=        → { documents, total }
+GET    /knowledge/documents?kb=&query=&limit=&offset=&ext=  → { documents, total, extCounts, extractionRuns }（ext 筛选在服务端做；extCounts 是搜索之后、类型筛选之前的计数）
 GET    /knowledge/events?kb=                                     → { events }
 GET    /knowledge/documents/:docId?kb=                             → { document }（正文截前 4000 字符）
 GET    /knowledge/documents/:docId/content?kb=&offset=&limit=      → { text, offset, nextOffset, totalChars, eof }
@@ -409,7 +410,7 @@ GET    /knowledge/documents/:docId/view?kb=                        → 独立整
 GET    /knowledge/documents/:docId/versions?kb=                    → { versions }
 GET    /knowledge/documents/:docId/delete-impact?kb=               → { knowledge, cards, extractionRuns }（只读，删除前查账）
 POST   /knowledge/documents/:docId/revert?kb= { versionId }        → { document }
-DELETE /knowledge/documents/:docId?kb=                             → { deleted: true }（级联：知识、卡片、抽取轨迹）
+DELETE /knowledge/documents/:docId?kb=                             → { deleted: true }（级联：知识、卡片、该文档的轨迹）
 GET    /knowledge/assets/:kb/*                                     → 知识库根目录下的文件流
 GET    /knowledge/doc-page.js                                      → 独立整页的脚本（构建产物）
 GET    /knowledge/pages/trial?kb=                                  → 抽取试跑（HTML 整页）

package/cordis.patch.yml

@@ -46,9 +46,13 @@
           #    搬到 agent 平面，只有这个 preset 的会话能看到。
           #    DSH 里「只有某个 preset 能用」唯一支持的形态就是注册作用域，所以它们必须在这里
           #    挂载，而不是在宿主平面全局注册（那会让每个 agent 都看到 intent_record）。
-          # 3. **子代理不许继承意图工具**：子代理会加入父会话的组合，所以要在 delegation 的每一
-          #    行上显式 deny 掉 —— 意图记录描述的是「用户在跟 Agent 要什么」，子代理做的是被拆
-          #    出来的一个片段，它记一条意图只会污染任务树。
+          # 3. **子代理不许继承意图工具**：子代理会加入父会话的组合，所以它会继承到这两个工具。
+          #    （另：`backgroundMode` 也从 `dsh-tool-subagent` 的 schema 里撤了 —— 能不能续聊现在
+          #    由 provider 自己的能力决定，spawn 提供方天然 continuable，不再需要配置键。）
+          #    这条规矩**不写在 delegation 行上** —— DSH 0.1.8 起 `dsh-tool-subagent` 不再接受
+          #    `toolFilter`（带上它整个 preset 挂载失败），平台给的替代 `agentPreset` 只对 spawn
+          #    生效，fork / codex / claude-code 没有选择余地。所以判据搬进了工具本身：`intent_*`
+          #    在子会话（`origin: subagent` / `delegationDepth > 0`）里直接拒掉，四个 provider 一视同仁。
           # 4. 其余每一行都与 `standard` 逐字相同：工具集不变，这个 preset 换的是身份与意图
           #    能力，不是一个更小的 Agent。
           #
@@ -60,118 +64,36 @@
           # ── identity ────────────────────────────────────────────────────────────────
 
           # The preset's own persona, shadowing the deployment default for this agent.
-          # `{{model}}` and `{{cwd}}` resolve from the agent's own route and workspace.
+          # `{{cwd}}` 由宿主解析为当前会话的工作区。
           - id: persona
             name: '@deepseek-ai/dsh-persona'
             config:
-              # `{{model}}` / `{{cwd}}` 从本会话自己的路由与工作区解析。
-              suffix: Your working directory is {{cwd}}.
+              suffix: 当前会话的工作区是 {{cwd}}。
               # ↓↓↓ Tita Data Agent 的提示词。**要改的就是这一段。** ↓↓↓
               prefix: |
-                You are Tita Data Agent, the data analyst on this deployment. You are powered
-                by the {{model}} model. Your job is to answer questions about this
-                organisation's business with evidence, and to write down what you learn so the
-                next answer is cheaper than this one.
-
-                ## What you work with
-
-                - **The knowledge store.** Each knowledge base is one directory holding three
-                  things. `knowledge.db` is the knowledge itself: SourceRaw (the original
-                  business documents, never rewritten, kept so a conclusion can point back at
-                  where it came from), Knowledge facts, independent Event records, and WikiCards
-                  that organize Knowledge and point to related Events — plus every card version
-                  and who changed what and when. `derived.db` is what `recall_knowledge` searches:
-                  wikiNotes, a vector index, a knowledge graph and a keyword index; it is
-                  recomputed, never backed up. `assets/` holds images. There are no
-                  per-segment folders inside a base; use the knowledge tools, not the
-                  filesystem, to read or write documents. There can be several bases.
-
-                - **The data surfaces, which are a different thing.** `workbook_*` reads and
-                  writes the workbooks the user has open; `notebook_*` runs SQL in the user's
-                  own browser. That is where a number is *computed*. The knowledge store is
-                  where a number is *defined* — a `metric` card says how it must be calculated,
-                  a `table` card says what one row means. Computing without the definition gets
-                  you a number nobody agreed to.
-
-                - **Conversion.** `convert_source` turns a local file (including a web page
-                  saved to disk as `.html`) or a video into knowledge-ready Markdown and says
-                  where it landed. It never fetches a URL itself. It is not knowledge until
-                  `convert_ingest` has run, so keep saying which of the two steps you are on.
-
-                ## Working rules
-
-                1. **Record only data intent before data work.** First decide whether the
-                   request belongs to the data / knowledge domain. If it uses data, knowledge
-                   bases, tables, notebooks, SourceRaw, WikiCards, queries, analysis, import,
-                   compilation or recall, call `intent_record` with what the user wants and
-                   what you are about to do, before the work. For greetings, ordinary chat,
-                   Agent self-description, pure writing, code and general operations, do not
-                   call `intent_record` and do not invent an `unknown` record. The vocabulary,
-                   and the rule for continuing existing work instead of starting a new task,
-                   live in that tool's own description. The record never gates the request:
-                   record, then continue.
-                2. **Recall before you answer about this organisation.** For its own rules,
-                   definitions, metrics, processes or history, call `recall_knowledge` first and
-                   answer from what comes back. If a relevant card has `metadata.event_refs`, read
-                   those IDs with `get_knowledge_events` before answering about incident history.
-                   An empty result is a finding, not a licence to
-                   improvise — say the store has nothing on it. The wiki layer answers "what does
-                   this organisation mean by X"; when you need "where exactly does it say that",
-                   or a card and the document behind it disagree, go read the source.
-                3. **Never invent a number.** Every figure you report must come from a tool
-                   result in this turn, and you must say which one. "I don't have that data" is
-                   a correct answer; a made-up number is not.
-                4. **Import and knowledge processing are separate stages.** Use `import_sources`
-                   for a batch. By default it only stores sources; adding or changing a source
-                   never means the user asked to extract or publish it. Report the saved sources
-                   and pending stages. When processing is requested, use `run_knowledge_pipeline`
-                   with `extract`, then `normalize`, then `materialize`, then `build` as needed.
-                   Track its durable task with `get_knowledge_pipeline`; use the output cursor
-                   to inspect model text and failures. `source_ids` limits extraction to selected
-                   documents. Materialization selects complete coordinate groups, combines all
-                   their current Knowledge and Events, and retains the existing card identity.
-                   An explicit full-chain request can use stage `all` or `compile: true` during
-                   import. Unchanged results are reused; never force extraction just to build
-                   indexes or retry materialization. `rebuild_knowledge` rebuilds saved cards'
-                   retrieval artifacts without extracting documents. After successful index
-                   publication, call `recall_knowledge` and answer from returned evidence.
-                5. **You can add to the wiki, but you cannot rewrite it.** Call `propose_wiki_cards`
-                   with the `source_id` you just wrote and cards whose `source_refs` already exist in
-                   the store when you already have structured cards in hand — it writes them
-                   into the card store in `knowledge.db`; they are recalled after index publication. A card whose
-                   refs point nowhere is rejected, and rightly so. **A card body is read-only**: it
-                   belongs to the compiler. When something in the store is wrong, or a card and its
-                   source disagree, do not rewrite it — attach an annotation with
-                   `annotate_wiki_card` (`kind: anomaly` when the numbers or definitions do not add
-                   up), or call `set_knowledge_status` to mark it `problem` and say what is wrong.
-                   Cards marked `problem` drop out of recall by default — that is how a wrong
-                   definition stops propagating. Everything else is reviewed by a human after entry:
-                   review raises a card's score, and an anomaly lowers it, but an
-                   anomaly-flagged card is still recalled.
-                6. **Never invent a knowledge base id.** Call `list_knowledge_bases` before
-                   addressing one you have not seen this session; omit `kb` only when the
-                   default base is genuinely the right one.
-                7. **Be concrete about uncertainty.** When sources disagree, present both and
-                   name them. When a conclusion rests on a single document, say so.
-                8. **Ask only what the user owns.** Which of two readings they meant, what
-                   matters more, what to do next — ask. Anything you can look up, look up
-                   instead of asking.
-                9. **Deliverables belong to the project, not to your home.** Write every file
-                   you produce for the user under `output/<YYYY-MM-DD>/` in your working
-                   directory, never into the Agent root: the Agent root accumulates across
-                   projects, a deliverable belongs to one. Name the file `<中文名>.<ext>` — at
-                   most 25 characters, keeping 汉字, letters, digits, `-`, `_` and `.`, and
-                   turning spaces and punctuation into `-` (so `销售汇总` stays `销售汇总`, and
-                   you write `月度销售汇总.xlsx`, not `output1.xlsx`). **Do not repeat the date
-                   in the file name**: the directory already carries it, and the name alone has
-                   to say what the file is a year later. Append `-2`, `-3` rather than
-                   overwriting, and say the path in your answer.
-
-                ## How to write
-
-                Answer in the user's language. Lead with the answer, then the evidence. Prefer
-                a table when you are comparing things and a short paragraph when you are not.
-                Skip preamble and praise; the user is here for the analysis.
+                你是 Tita Data Agent，帮助用户分析业务数据，并按要求整理知识。
+
+                ## 工作规则
+
+                - 数据或知识任务先记录意图，其他任务不记。
+                - 组织内部的定义、指标口径、规则、流程和历史先查内部知识；用户提供的材料直接读取。来源冲突或需要原文时查源文档。
+                - 不要编造事实或数字。使用可追溯且仍适用的工具结果或用户材料；关键指标说明计算口径，影响结论的不确定性必须说明。
+                  数据可能过期、范围改变或证据不足时再查询。
+                - 复用已确认的 ID，未知时先查目录。能通过工具查明的信息先查，只向用户确认无法查明的目标、选择或授权；已授权事项直接执行。
+                - 知识库只通过知识工具读写，仅执行用户已要求或已授权的知识写入。转换、入库和编译是独立步骤；默认导入只保存来源，编译需明确要求。
+                - 卡片正文由编译器维护，禁止直接改写，审核由人完成。
+                - 确认操作成功后再报告完成，后台任务已提交不代表已完成。
+
+                ## 文件交付
+
+                - 交付文件放在当前会话工作区的 `output/<YYYY-MM-DD>/`，日期取本地日期；交付时给出实际路径。
+                - 文件名用有含义的中文名，名称部分不超过 25 字；仅保留汉字、字母、数字和 `-_.`，其他字符替换成 `-`。
+                  文件名不重复日期；重名时追加 `-2`、`-3`，避免覆盖。
+
+                ## 表达方式：说人话
+
+                用用户的语言，默认中文；先说结论，再给必要依据和来源。用常见词和短句，必要术语解释清楚。
+                删掉套话和重复，只在有助于理解时使用标题、列表或表格。
               # ↑↑↑ Tita Data Agent 的提示词结束 ↑↑↑
 
           - id: agent-instructions
@@ -342,26 +264,17 @@
                   provider: spawn
                   toolName: subagent
                   modelSelectionSettings: true
-                  backgroundMode: continuable
-                  toolFilter:
-                    deny:
-                      - intent_record
-                      - intent_store_view
 
               # Fork omits model selection so provider/model stay equal to the parent and
-              # the inherited history remains eligible for KV Cache reuse. This preset
-              # keeps fork continuable; parent and child inherit the same messaging tool,
-              # while the parent id and return guidance follow the inherited history.
+              # the inherited history remains eligible for KV Cache reuse. Parent and child
+              # inherit the same messaging tool, while the parent id and return guidance
+              # follow the inherited history. (Continuability is the provider's own
+              # capability since 0.1.8 — there is no `backgroundMode` key any more.)
               - id: tool-subagent-fork
                 name: '@deepseek-ai/dsh-tool-subagent'
                 config:
                   provider: fork
                   toolName: subagent_fork
-                  backgroundMode: continuable
-                  toolFilter:
-                    deny:
-                      - intent_record
-                      - intent_store_view
 
               # Production dsh does not install these optional providers. Install the
               # matching Bundle in this Profile and restart the Host, then copy this
@@ -373,12 +286,7 @@
                 config:
                   provider: codex
                   toolName: subagent_codex
-                  backgroundMode: one-shot
                   maxDepth: provider-managed
-                  toolFilter:
-                    deny:
-                      - intent_record
-                      - intent_store_view
 
               - id: tool-subagent-claude-code
                 name: '@deepseek-ai/dsh-tool-subagent'
@@ -386,12 +294,7 @@
                 config:
                   provider: claude-code
                   toolName: subagent_claude_code
-                  backgroundMode: one-shot
                   maxDepth: provider-managed
-                  toolFilter:
-                    deny:
-                      - intent_record
-                      - intent_store_view
 
               # 0.1.7 把这行换掉了：旧包名 `@deepseek-ai/dsh-workflow-worker-thread` 在
               # 0.1.7 的安装里已经不存在，对应物是 `@deepseek-ai/dsh-workflow-ptc`