dsh-data-domain-knowledge 0.1.0-alpha.1

package/preset/tita-data-agent/agent.cordis.yml

@@ -0,0 +1,376 @@
+# Tita Data Agent 的 agent 组合（0029）。
+#
+# 这是 `standard` 的一份**拷贝**，只改了四件事。preset 是「一份完整组合」，没有
+# patch 语义（`dsh-agent-presets` 的 authoring 是 copy-only），所以它必须是一整份
+# 文件、而不是对 standard 的增量。
+#
+# 改了什么：
+#
+# 1. **身份**（`persona` 行）：把「你是一个编码 Agent」换成 Tita Data Agent 自己的
+#    提示词。这一行是**这个 preset 存在的理由** —— 它 shadow 掉部署级的 persona，
+#    所以只有加入这个 preset 的会话会以数据 Agent 的身份说话。提示词正文就在下面，
+#    直接改它即可；改完**新建一个会话**才生效（preset 在会话创建时固定）。
+# 2. **意图工具**（`intent-tools` 行）：`intent_record` / `intent_store_view` /
+#    `intent_registry_list` 三个工具搬到 agent 平面，只有这个 preset 的会话能看到。
+#    DSH 里「只有某个 preset 能用」唯一支持的形态就是注册作用域，所以它们必须在这里
+#    挂载，而不是在宿主平面全局注册（那会让每个 agent 都看到 intent_record）。
+# 3. **子代理不许继承意图工具**：子代理会加入父会话的组合，所以要在 delegation 的每一
+#    行上显式 deny 掉 —— 意图记录描述的是「用户在跟 Agent 要什么」，子代理做的是被拆
+#    出来的一个片段，它记一条意图只会污染任务树。
+# 4. 其余每一行都与 `standard` 逐字相同：工具集不变，这个 preset 换的是身份与意图
+#    能力，不是一个更小的 Agent。
+#
+# 其余纪律与 `standard` 一致：这是 AGENT-PLANE 组合，roster 在 standing scope 下挂载
+# 一次，每个命名它的会话按作用域父子关系加入。需要 provide 服务的行必须落在带
+# `isolate` realm 的 group 里，否则它会发布到根 realm（进程级）并被挂载审计拒绝 ——
+# 本文件里唯一这样的行是 delegation 那个 group。
+
+# ── identity ────────────────────────────────────────────────────────────────
+
+# The preset's own persona, shadowing the deployment default for this agent.
+# `{{model}}` and `{{cwd}}` resolve from the agent's own route and workspace.
+- id: persona
+  name: '@deepseek-ai/dsh-persona'
+  config:
+    # `{{model}}` / `{{cwd}}` 从本会话自己的路由与工作区解析。
+    suffix: Your working directory is {{cwd}}.
+    # ↓↓↓ Tita Data Agent 的提示词。**要改的就是这一段。** ↓↓↓
+    prefix: |
+      You are Tita Data Agent, the data analyst on this deployment. You are powered
+      by the {{model}} model. Your job is to answer questions about this
+      organisation's business with evidence, and to write down what you learn so the
+      next answer is cheaper than this one.
+
+      ## What you work with
+
+      - **The knowledge store.** Each knowledge base on disk is a tree.
+        `01_source_truth` holds SourceRaw: the original business documents, never
+        rewritten, kept so that a conclusion can point back at where it came from.
+        `02_wiki` holds the cards people read — one per entity, each with mandatory
+        `source_refs`, each following a type (`concept`, `metric`, `table`, `event`,
+        `playbook`, `case`). `03_compiled` is what `recall_knowledge` actually
+        searches: wikiNotes, a vector index and a knowledge graph. `05_diagnosis` is
+        the attribution graph the panel renders. There can be several bases.
+
+      - **The data surfaces, which are a different thing.** `workbook_*` reads and
+        writes the workbooks the user has open; `notebook_*` runs SQL in the user's
+        own browser. That is where a number is *computed*. The knowledge store is
+        where a number is *defined* — a `metric` card says how it must be calculated,
+        a `table` card says what one row means. Computing without the definition gets
+        you a number nobody agreed to.
+
+      - **Conversion.** `convert_source` turns a file, a web page or a video into
+        knowledge-ready Markdown and says where it landed. It is not knowledge until
+        `convert_ingest` has run, so keep saying which of the two steps you are on.
+
+      ## Working rules
+
+      1. **Record the intent before the work.** Call `intent_record` with what the
+         user wants and what you are about to do, before you do it. The vocabulary,
+         and the rule for continuing existing work instead of starting a new task,
+         live in that tool's own description. The record never gates the request:
+         record, then continue.
+      2. **Recall before you answer about this organisation.** For its own rules,
+         definitions, metrics, processes or history, call `recall_knowledge` first and
+         answer from what comes back. An empty result is a finding, not a licence to
+         improvise — say the store has nothing on it. The wiki layer answers "what does
+         this organisation mean by X"; when you need "where exactly does it say that",
+         or a card and the document behind it disagree, go read the source.
+      3. **Never invent a number.** Every figure you report must come from a tool
+         result in this turn, and you must say which one. "I don't have that data" is
+         a correct answer; a made-up number is not.
+      4. **Put it in writing while you still have the evidence.** A definition, a
+         metric's exact calculation, a process, a diagnosed cause — if it will be
+         asked again, write it as a card with `write_knowledge_card`, with
+         `source_refs` that already exist in the store. A card whose refs point
+         nowhere is rejected, and rightly so.
+      5. **Correct a card, do not quietly overwrite it.** When something in the store
+         is wrong, set its status to `problem` and say what is wrong with it. Cards
+         marked `problem` drop out of recall by default — that is how a wrong
+         definition stops propagating.
+      6. **Never invent a knowledge base id.** Call `list_knowledge_bases` before
+         addressing one you have not seen this session; omit `kb` only when the
+         default base is genuinely the right one.
+      7. **Be concrete about uncertainty.** When sources disagree, present both and
+         name them. When a conclusion rests on a single document, say so.
+      8. **Ask only what the user owns.** Which of two readings they meant, what
+         matters more, what to do next — ask. Anything you can look up, look up
+         instead of asking.
+      9. **Deliverables belong to the project, not to your home.** Write every file
+         you produce for the user under `output/<YYYY-MM-DD>/` in your working
+         directory, never into the Agent root: the Agent root accumulates across
+         projects, a deliverable belongs to one. Name it `<topic>-<kind>.<ext>` where
+         `<kind>` is exactly one of `报告`, `图表`, `数据集`, `清洗脚本`, `SQL` — the
+         name alone has to say what the file is a year later. Append `-2`, `-3` rather
+         than overwriting, and say the path in your answer.
+
+      ## How to write
+
+      Answer in the user's language. Lead with the answer, then the evidence. Prefer
+      a table when you are comparing things and a short paragraph when you are not.
+      Skip preamble and praise; the user is here for the analysis.
+    # ↑↑↑ Tita Data Agent 的提示词结束 ↑↑↑
+
+- id: agent-instructions
+  name: '@deepseek-ai/dsh-agent-instructions'
+  config:
+    maxBytes: 65536
+
+# ── 意图记录（agent 平面）────────────────────────────────────────────────────
+#
+# 0029 之前这三个工具由知识插件在**宿主平面**全局注册，于是每个 agent、每个 preset
+# 的会话都会在提示词里看到 `intent_record`。现在它们只在这里挂载：这个模块只往
+# `tools` 注册表里塞定义、不 provide 任何服务，所以和随包的 `tool-fs` 一样不需要
+# realm，而它的注册只覆盖加入本 preset 的会话。
+#
+# 引擎本身（设置、轮次记账、`/intent/**` 只读面）仍在宿主平面，通过
+# `titaDataAgentIntent` 服务把操作面暴露给这一行 —— 引擎只有一份。
+- id: intent-tools
+  name: 'dsh-data-domain-knowledge/intent-tools'
+
+# ── shell ───────────────────────────────────────────────────────────────────
+
+# `shell-env` stays in the HOST composition: `apps/cli/src/web.ts` injects it to
+# publish `DSH_WEB_URL`/`DSH_WEB_MODE`, and a host row that injects a service is
+# the criterion for host-plane ownership — injection resolves before any session
+# exists, so there is no agent to key by. Behind a preset realm those variables
+# never reached the model's shell at all. Both shell tools consume the host
+# registry from here; their executors (`bash-sandbox`/`pwsh-sandbox`) are
+# host-plane too.
+- id: tool-bash
+  name: '@deepseek-ai/dsh-tool-bash'
+  disabled: !!js process.platform === 'win32'
+
+- id: tool-pwsh
+  name: '@deepseek-ai/dsh-tool-pwsh'
+  disabled: !!js process.platform !== 'win32'
+
+# ── filesystem ──────────────────────────────────────────────────────────────
+
+# Both register into the host `tools` registry and provide nothing, so
+# they need no realm. The `fs` service and its policy stay in the host.
+- id: tool-fs
+  name: '@deepseek-ai/dsh-tool-fs'
+
+- id: tool-fs-search
+  name: '@deepseek-ai/dsh-tool-fs-search'
+  config:
+    sampleOverCapGlobResults: false
+
+# ── background jobs ────────────────────────────────────────────────────────
+
+# Only the model-facing controls. The task REGISTRY stays on the host plane:
+# its producers sit outside any realm this file could put it in — `tool-bash`
+# above resolves it with `ctx.get`, and an entry-local realm here is invisible
+# to every sibling row, so `run_in_background` would answer "background jobs
+# unavailable" while these controls sat in the catalog. The registry is keyed by
+# owning agent anyway, so one host instance serves every session. What a preset
+# chooses is whether its agent can collect and stop background work at all.
+- id: tool-jobs
+  name: '@deepseek-ai/dsh-tool-jobs'
+
+# ── skills ──────────────────────────────────────────────────────────────────
+
+# The skill REGISTRY lives in the host composition and is layered per scope:
+# these rows register into THIS preset's layer of it, so they need no realm.
+# `skill-filesystem` contributes local-root discovery for agents on this preset, and
+# `tool-skill` gives them the catalog and loader; the merged catalog also
+# carries whatever the deployment registered globally (repository plugins).
+- id: skill-filesystem
+  name: '@deepseek-ai/dsh-skill-filesystem'
+
+- id: tool-skill
+  name: '@deepseek-ai/dsh-tool-skill'
+
+# ── goals ───────────────────────────────────────────────────────────────────
+
+# The goal service and session driver stay on the host plane, where the Gateway
+# can resolve them. The human command and model-facing tool register into this
+# preset's scoped layers.
+- id: command-goal
+  name: '@deepseek-ai/dsh-command-goal'
+
+- id: tool-goal
+  name: '@deepseek-ai/dsh-tool-goal'
+
+# ── plan mode ───────────────────────────────────────────────────────────────
+
+# Plan state is per-agent by nature, so an entry-local realm is not a
+# workaround here — it is the correct lifetime.
+- id: planning
+  name: cordis:group
+  group: true
+  isolate:
+    planMode: true
+  config:
+    - id: plan-mode
+      name: '@deepseek-ai/dsh-plan-mode'
+      config:
+        section: |
+              You are in plan mode. Stay in plan mode until exit_plan_mode succeeds or the user switches the session mode. Imperative language to implement changes means plan the implementation, not execute it. A user's conversational agreement — including an answer confirming something you asked — approves nothing and does not end plan mode; fold the confirmed decision into the plan and submit it through exit_plan_mode.
+
+              Explore first. Use non-mutating reads, searches, static analysis, and checks to ground the plan in the actual repository. Do not edit or write files, change configuration, run formatters or code generation that rewrites tracked files, commit, or otherwise carry out the plan. Prefer existing functions and patterns over new machinery.
+
+              The tool catalog stays the same across modes for request-cache stability. These plan-mode rules override any later tool description or guidance that suggests using mutation tools; those tools remain listed to keep the tool catalog unchanged. Do not use todo_write to track this planning phase: it tracks implementation after an approved plan, while the plan itself belongs in exit_plan_mode.
+
+              Resolve discoverable facts by inspection. Use ask_user_question only for user-owned choices or material ambiguity that inspection cannot answer. Do not ask the user where code lives or how current behavior works when you can find out.
+
+              Make the plan decision-complete: state the goal and success criteria; group implementation changes by subsystem; identify public API, schema, and data-flow changes; cover edge cases, failure modes, tests, acceptance criteria, and explicit assumptions. Keep it concise enough to review but detailed enough that another engineer can implement it without making design decisions.
+
+              When ready, call exit_plan_mode with the complete plan markdown, starting with a # title. Make exit_plan_mode the only and final tool call in that assistant response: it presents the plan for approval, and implementation begins only in a later step after approval. Do not paste the final plan as a plain reply or ask "should I proceed?" through prose or ask_user_question. If review rejects it, incorporate the feedback and present again. If the review channel is unavailable or aborted, stay in plan mode and ask the user to switch modes manually; do not proceed with implementation.
+
+# ── compaction ──────────────────────────────────────────────────────────────
+
+# `compaction-basic` reads `toolResultPrune` through `ctx.get`, so the pruner must
+# share this realm rather than sit outside it.
+#
+# `tokenMeter` is deliberately NOT in this realm: the meter stays on the HOST
+# plane, and the rows here resolve that one instance. It takes no configuration,
+# keys every fold by Session, and owns the context-meter projection units the
+# browser reads for every session — behind a realm those units would come and go
+# with whichever presets happen to be mounted. What a preset chooses is whether
+# its agent compacts at all, which is `compaction-basic` below.
+- id: compaction
+  name: cordis:group
+  group: true
+  isolate:
+    compaction: true
+    toolResultPruner: true
+  config:
+    - id: compaction-basic
+      name: '@deepseek-ai/dsh-compaction-basic'
+
+    - id: command-compact
+      name: '@deepseek-ai/dsh-command-compact'
+
+    - id: tool-result-pruner
+      name: '@deepseek-ai/dsh-compaction-tool-result-pruner'
+      config:
+        thresholdChars: 8192
+        headChars: 4096
+        tailChars: 1024
+
+# ── delegation and workflows ────────────────────────────────────────────────
+
+# The `subagents` registry and its spawn/fork backends live in the HOST
+# composition: the registry is a process singleton whose cross-session queries
+# the api-proxy serves to the browser, and a provider name may only be
+# registered once. This preset contributes the delegation TOOLS, which resolve
+# that host registry.
+#
+# `workflows` is different — nothing outside an agent reads it — so every row
+# that reaches it shares one entry-local realm here, and a consumer left
+# outside would resolve a host registry this preset does not populate.
+- id: delegation
+  name: cordis:group
+  group: true
+  isolate:
+    workflowEngine: true
+  config:
+    - id: tool-subagent-control
+      name: '@deepseek-ai/dsh-tool-subagent-control'
+
+    - id: tool-subagent-list-agents
+      name: '@deepseek-ai/dsh-tool-subagent-control/list-agents'
+
+    - id: tool-subagent
+      name: '@deepseek-ai/dsh-tool-subagent'
+      config:
+        provider: spawn
+        toolName: subagent
+        modelSelectionSettings: true
+        backgroundMode: continuable
+        toolFilter:
+          deny:
+            - intent_record
+            - intent_store_view
+            - intent_registry_list
+
+    # Fork omits model selection so provider/model stay equal to the parent and
+    # the inherited history remains eligible for KV Cache reuse. This preset
+    # keeps fork continuable; parent and child inherit the same messaging tool,
+    # while the parent id and return guidance follow the inherited history.
+    - id: tool-subagent-fork
+      name: '@deepseek-ai/dsh-tool-subagent'
+      config:
+        provider: fork
+        toolName: subagent_fork
+        backgroundMode: continuable
+        toolFilter:
+          deny:
+            - intent_record
+            - intent_store_view
+            - intent_registry_list
+
+    # Production dsh does not install these optional providers. Install the
+    # matching Bundle in this Profile and restart the Host, then copy this
+    # preset and remove `disabled` from the matching tool row. Host availability
+    # alone grants no tool.
+    - id: tool-subagent-codex
+      name: '@deepseek-ai/dsh-tool-subagent'
+      disabled: true
+      config:
+        provider: codex
+        toolName: subagent_codex
+        backgroundMode: one-shot
+        maxDepth: provider-managed
+        toolFilter:
+          deny:
+            - intent_record
+            - intent_store_view
+            - intent_registry_list
+
+    - id: tool-subagent-claude-code
+      name: '@deepseek-ai/dsh-tool-subagent'
+      disabled: true
+      config:
+        provider: claude-code
+        toolName: subagent_claude_code
+        backgroundMode: one-shot
+        maxDepth: provider-managed
+        toolFilter:
+          deny:
+            - intent_record
+            - intent_store_view
+            - intent_registry_list
+
+    # 0.1.7 把这行换掉了：旧包名 `@deepseek-ai/dsh-workflow-worker-thread` 在
+    # 0.1.7 的安装里已经不存在，对应物是 `@deepseek-ai/dsh-workflow-ptc`
+    # （见 `dsh-web-app` 的 `presets/cordis.patch.yml`）。留着旧名不会报「找不到
+    # 包」，而是让**整个 preset 挂载失败**，症状是它出现在 Agent 预设名单里、
+    # 却带一条 `workflow-worker-thread (…): never started` 诊断 —— 这也是
+    # 「自定义 agent 没生效」最像「配置没生效」的一种失败。
+    - id: workflow-ptc
+      name: '@deepseek-ai/dsh-workflow-ptc'
+      config:
+        provider: spawn
+
+    - id: tool-workflow
+      name: '@deepseek-ai/dsh-tool-workflow'
+
+    - id: tool-ralph
+      name: '@deepseek-ai/dsh-tool-ralph'
+      config:
+        subagentProvider: spawn
+        maxRounds: 64
+
+# ── remaining model-facing rows ─────────────────────────────────────────────
+
+- id: tool-ask-user
+  name: '@deepseek-ai/dsh-tool-ask-user'
+
+- id: tool-todo
+  name: '@deepseek-ai/dsh-tool-todo'
+  config:
+    allowParallelInProgress: true
+
+# The `web` service and its search provider stay in the host composition; only
+# the model-facing tool is per-session.
+- id: tool-web
+  name: '@deepseek-ai/dsh-tool-web'
+  config:
+    fetch: true
+    searchTimeoutMs: 60000
+
+- id: present
+  name: '@deepseek-ai/dsh-tool-present'

package/preset/tita-data-agent/preset.yml

@@ -0,0 +1,3 @@
+name: Tita Data Agent
+description: TitaAgent 自己的组合：数据 Agent 的提示词，加上只有这个 preset 才有的意图记录工具。
+order: 10

package/python/requirements.txt

@@ -0,0 +1,8 @@
+torch
+torchvision
+pillow
+transformers==5.2.0
+sentence-transformers>=5.7.0
+accelerate>=1.1.0
+qwen-vl-utils==0.0.14
+huggingface_hub[cli]

package/scripts/doctor.mjs

@@ -0,0 +1,70 @@
+import assert from 'node:assert/strict'
+import { spawnSync } from 'node:child_process'
+import { existsSync } from 'node:fs'
+import { mkdtemp, rm } from 'node:fs/promises'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { loadConvertCore } from '../lib/types/convert/core-loader.js'
+import { IntentEngine } from '../lib/types/intent/engine.js'
+import { defaultKnowledgeRoot, KnowledgeRuntime } from '../lib/types/runtime.js'
+
+const root = defaultKnowledgeRoot()
+const runtime = new KnowledgeRuntime(root)
+assert.ok(root.length > 0)
+assert.ok(runtime.config.embeddingDimension > 0)
+assert.equal(existsSync(new URL('../lib/client.js', import.meta.url)), true)
+
+// The 意图 module came from @tita-data-agent/data-domain-intent (0019). Its
+// registry is the contract behind `intent_registry_list` and the injected
+// routing prompt, so the doctor keeps the same assertion the old package had.
+const registry = new IntentEngine().registry()
+assert.equal(registry.categories.length, 12)
+// v0.7 起 acts 是对象（IntentActDef：id / label / description / enabled），
+// 断言相应地按 id 判成员 —— 这条固定的是「unknown 这个动作还在」，不是它的形状。
+assert.ok(registry.acts.some(act => act.id === 'unknown'))
+
+// The conversion kernel is optional (0018): report it, never fail on it.
+let convert
+let scratch
+try {
+  const core = await loadConvertCore()
+  scratch = await mkdtemp(join(tmpdir(), 'tita-convert-doctor-'))
+  const service = await core.createConvertService({ rootDir: scratch, maxConcurrent: 1 })
+  const capabilities = await service.capabilities()
+  convert = {
+    ok: true,
+    adapters: capabilities.adapters.map(adapter => adapter.id),
+    extensions: capabilities.extensions.length,
+  }
+  await service.close()
+} catch (error) {
+  convert = { ok: false, code: error?.code ?? 'UNKNOWN', message: error?.message ?? String(error) }
+} finally {
+  if (scratch) await rm(scratch, { recursive: true, force: true }).catch(() => undefined)
+}
+
+// preset 声明必须与 `preset/` 真源同步（0036）。DSH 只读 `cordis.patch.yml` 里
+// 那条 `@deepseek-ai/dsh-agent-preset` 声明，真源改了没编译的话，症状是「我改了
+// 提示词但没生效」—— 最难自己发现的一类失败。所以这一项不同步即 doctor 失败，
+// 不像 convert 那样只报告。
+const presetCheck = spawnSync(
+  process.execPath,
+  [fileURLToPath(new URL('./sync-agent-preset.mjs', import.meta.url)), '--check'],
+  { encoding: 'utf8' },
+)
+const preset = {
+  synced: presetCheck.status === 0,
+  message: `${presetCheck.stdout ?? ''}${presetCheck.stderr ?? ''}`.trim() || undefined,
+}
+if (!preset.synced) process.exitCode = 1
+
+console.log(JSON.stringify({
+  ok: true,
+  package: 'dsh-data-domain-knowledge',
+  root,
+  maxSourceBytes: runtime.config.maxSourceBytes,
+  convert,
+  intent: { categories: registry.categories.length, acts: registry.acts.length },
+  preset,
+}))