@biaoo/tiangong-wiki 0.3.7 → 0.3.9

package/README.md

@@ -86,7 +86,7 @@ tiangong-wiki sync                                    # index Markdown pages
 
 That means commands still work best from inside a workspace, but they can also run from outside the workspace after setup, or target a specific workspace explicitly with `--env-file`.
 
-For automatic vault processing, new setup runs default to `WIKI_AGENT_AUTH_MODE=codex-login`, a dedicated Codex home under the current user's home directory, and `WIKI_AGENT_MODEL=gpt-5.5`. Before enabling that mode, run `CODEX_HOME="$HOME/.codex-tiangong-wiki" codex login` on macOS/Linux, or set `$env:CODEX_HOME = "$env:USERPROFILE\.codex-tiangong-wiki"` before `codex login` on Windows PowerShell.
+For automatic vault processing, new setup runs default to `WIKI_AGENT_AUTH_MODE=codex-login`, the current user's standard Codex home (`~/.codex`, or the user profile `.codex` directory on Windows), and `WIKI_AGENT_MODEL=gpt-5.5`. If you want an isolated Codex home, set `WIKI_AGENT_CODEX_HOME=/absolute/path/to/.codex-tiangong-wiki` and first run `CODEX_HOME=/absolute/path/to/.codex-tiangong-wiki codex login`.
 
 ```bash
 tiangong-wiki find --type concept --status active     # structured query
@@ -117,6 +117,8 @@ tiangong-wiki dashboard                               # open dashboard in browse
 
 > Environment variables are managed via `.wiki.env` (created by `tiangong-wiki setup`). The CLI prefers the nearest local `.wiki.env`, then falls back to the global default workspace config. See [references/troubleshooting.md](./references/troubleshooting.md) for the full reference. For a centralized Linux + `systemd` + Nginx deployment, see [references/centralized-service-deployment.md](./references/centralized-service-deployment.md). That deployment guide now also includes Git repository / GitHub remote setup for daemon-side commit and optional auto-push.
 
+For document-heavy vaults, set `WIKI_PARSER_SKILLS=document-granular-decompose` and configure `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` to prefer the TianGong Unstructure parser. The wiki workflow uses `return_txt=true` and consumes the plain `txt` text as the agent input; `UNSTRUCTURED_PROVIDER` and `UNSTRUCTURED_MODEL` are optional overrides.
+
 ## MCP Server
 
 Tiangong Wiki ships a separate MCP adapter that talks to the daemon over HTTP. It uses the MCP Streamable HTTP transport, not stdio.

package/README.zh-CN.md

@@ -86,7 +86,7 @@ tiangong-wiki sync                                    # 索引 Markdown 文件
 
 这意味着命令仍然最适合在 workspace 内执行；但 setup 之后，即使在 workspace 外运行，也可以通过默认配置正常工作，或者通过 `--env-file` 显式指定目标工作区。
 
-如果启用自动 vault 处理，新的 setup 默认使用 `WIKI_AGENT_AUTH_MODE=codex-login`、当前用户 home 目录下的专用 Codex home 和 `WIKI_AGENT_MODEL=gpt-5.5`。启用前，macOS/Linux 执行 `CODEX_HOME="$HOME/.codex-tiangong-wiki" codex login`；Windows PowerShell 先设置 `$env:CODEX_HOME = "$env:USERPROFILE\.codex-tiangong-wiki"`，再执行 `codex login`。
+如果启用自动 vault 处理，新的 setup 默认使用 `WIKI_AGENT_AUTH_MODE=codex-login`、当前用户标准 Codex home（Linux/macOS 为 `~/.codex`，Windows 为用户目录下的 `.codex`）和 `WIKI_AGENT_MODEL=gpt-5.5`。如果需要隔离目录，可手动设置 `WIKI_AGENT_CODEX_HOME=/absolute/path/to/.codex-tiangong-wiki`，并先执行 `CODEX_HOME=/absolute/path/to/.codex-tiangong-wiki codex login`。
 
 ```bash
 tiangong-wiki find --type concept --status active     # 结构化查询
@@ -117,6 +117,8 @@ tiangong-wiki dashboard                               # 在浏览器中打开仪
 
 > 环境变量通过 `.wiki.env` 管理（由 `tiangong-wiki setup` 创建）。CLI 会优先使用最近的本地 `.wiki.env`，找不到时再 fallback 到全局默认工作区配置。完整参考见 [references/troubleshooting.md](./references/troubleshooting.md)。如需部署中心化服务（Linux + `systemd` + Nginx），见 [references/centralized-service-deployment.md](./references/centralized-service-deployment.md)。该部署文档现在也包含了 Git 仓库初始化、GitHub remote 配置和 daemon 自动 push 的 Git 配置说明。
 
+如果 vault 里以文档解析为主，可设置 `WIKI_PARSER_SKILLS=document-granular-decompose`，并配置 `UNSTRUCTURED_API_BASE_URL` 与 `UNSTRUCTURED_AUTH_TOKEN`，让 workflow 优先使用 TianGong Unstructure parser。wiki workflow 会使用 `return_txt=true`，并把返回的纯 `txt` 文本作为 agent 主输入；`UNSTRUCTURED_PROVIDER` 和 `UNSTRUCTURED_MODEL` 只是可选覆盖项。
+
 ## MCP Server
 
 Tiangong Wiki 提供了独立的 MCP 适配层，通过 HTTP 调用 daemon。它使用的是 MCP 的 Streamable HTTP 传输，不是 stdio。

package/assets/config.example.env

@@ -11,12 +11,11 @@ EMBEDDING_MODEL=text-embedding-3-small
 EMBEDDING_DIMENSIONS=1536
 
 WIKI_AGENT_ENABLED=false
-# For local Codex login auth, run on macOS/Linux:
-# CODEX_HOME="$HOME/.codex-tiangong-wiki" codex login
-# On Windows PowerShell:
-# $env:CODEX_HOME = "$env:USERPROFILE\.codex-tiangong-wiki"; codex login
+# For local Codex login auth, run `codex login` once as the same user.
 WIKI_AGENT_AUTH_MODE=codex-login
-# Optional. Leave unset to use the current user's default: <home>/.codex-tiangong-wiki
+# Optional. Leave unset to use the current user's standard Codex home: <home>/.codex
+# For an isolated directory, set this to an absolute path and first run:
+# CODEX_HOME=/absolute/path/to/.codex-tiangong-wiki codex login
 # WIKI_AGENT_CODEX_HOME=/absolute/path/to/.codex-tiangong-wiki
 #
 # To use an API key instead of Codex login:

package/dist/commands/check-config.js

@@ -1,4 +1,5 @@
 import path from "node:path";
+import { inspectWikiAgentCodexLogin } from "../core/agent-auth.js";
 import { loadRuntimeConfig } from "../core/runtime.js";
 import { EmbeddingClient } from "../core/embedding.js";
 import { getWikiAgentStatus } from "../core/vault-processing.js";
@@ -20,6 +21,17 @@ export function registerCheckConfigCommand(program) {
         if (wikiAgent.enabled && wikiAgent.missing.length > 0) {
             throw new AppError(`WIKI_AGENT_ENABLED=true but missing required settings: ${wikiAgent.missing.join(", ")}`, "config");
         }
+        const agentCodexLogin = inspectWikiAgentCodexLogin(wikiAgent);
+        if (!agentCodexLogin.ready) {
+            throw new AppError([
+                agentCodexLogin.summary,
+                agentCodexLogin.recommendation,
+            ].filter(Boolean).join(" "), "config");
+        }
+        const safeWikiAgent = {
+            ...wikiAgent,
+            apiKey: wikiAgent.apiKey ? "<redacted>" : null,
+        };
         const templateChecks = Object.keys(config.templates).map((pageType) => {
             const templatePath = resolveTemplateFilePath(config, paths.wikiRoot, pageType);
             return {
@@ -44,7 +56,8 @@ export function registerCheckConfigCommand(program) {
             templatesPath: paths.templatesPath,
             configVersion: config.configVersion,
             embeddingConfigured: embeddingClient !== null,
-            agentProcessing: wikiAgent,
+            agentProcessing: safeWikiAgent,
+            agentCodexLogin,
             probe,
             templateChecks,
         };
@@ -70,6 +83,8 @@ export function registerCheckConfigCommand(program) {
                 agentBatchSize: wikiAgent.batchSize,
                 agentWorkflowTimeoutSeconds: wikiAgent.workflowTimeoutSeconds,
                 agentMissing: wikiAgent.missing.join(", "),
+                agentCodexLoginReady: agentCodexLogin.checked ? agentCodexLogin.ready : "",
+                agentCodexLoginAuthJson: agentCodexLogin.authJsonPath ?? "",
                 probe,
             }),
             "",

package/dist/core/agent-auth.js

@@ -0,0 +1,101 @@
+import { statSync } from "node:fs";
+import path from "node:path";
+function inspectPathKind(targetPath) {
+    try {
+        const stats = statSync(targetPath);
+        if (stats.isDirectory()) {
+            return { kind: "directory" };
+        }
+        if (stats.isFile()) {
+            return { kind: "file" };
+        }
+        return { kind: "other" };
+    }
+    catch (error) {
+        const code = error.code;
+        if (code === "ENOENT" || code === "ENOTDIR") {
+            return { kind: "missing" };
+        }
+        const message = error instanceof Error ? error.message : String(error);
+        return { kind: "inaccessible", errorMessage: message };
+    }
+}
+function codexLoginRecommendation(codexHome) {
+    return `Run \`CODEX_HOME=${codexHome} codex login\` before starting automatic vault processing.`;
+}
+export function inspectWikiAgentCodexLogin(settings) {
+    if (!settings.enabled || settings.authMode !== "codex-login") {
+        return {
+            checked: false,
+            ready: true,
+            codexHome: settings.codexHome,
+            authJsonPath: null,
+            summary: "Codex login auth is not enabled.",
+        };
+    }
+    if (!settings.codexHome) {
+        return {
+            checked: true,
+            ready: false,
+            codexHome: null,
+            authJsonPath: null,
+            summary: "WIKI_AGENT_CODEX_HOME is not configured for codex-login auth.",
+            recommendation: "Set WIKI_AGENT_CODEX_HOME or rerun `tiangong-wiki setup`.",
+        };
+    }
+    const codexHome = settings.codexHome;
+    const authJsonPath = path.join(codexHome, "auth.json");
+    const home = inspectPathKind(codexHome);
+    if (home.kind === "missing") {
+        return {
+            checked: true,
+            ready: false,
+            codexHome,
+            authJsonPath,
+            summary: `WIKI_AGENT_CODEX_HOME does not exist: ${codexHome}`,
+            recommendation: codexLoginRecommendation(codexHome),
+        };
+    }
+    if (home.kind !== "directory") {
+        return {
+            checked: true,
+            ready: false,
+            codexHome,
+            authJsonPath,
+            summary: home.kind === "inaccessible"
+                ? `WIKI_AGENT_CODEX_HOME cannot be inspected: ${codexHome} (${home.errorMessage})`
+                : `WIKI_AGENT_CODEX_HOME is not a directory: ${codexHome}`,
+            recommendation: codexLoginRecommendation(codexHome),
+        };
+    }
+    const authJson = inspectPathKind(authJsonPath);
+    if (authJson.kind === "missing") {
+        return {
+            checked: true,
+            ready: false,
+            codexHome,
+            authJsonPath,
+            summary: `Codex login auth.json was not found under WIKI_AGENT_CODEX_HOME: ${authJsonPath}`,
+            recommendation: codexLoginRecommendation(codexHome),
+        };
+    }
+    if (authJson.kind !== "file") {
+        return {
+            checked: true,
+            ready: false,
+            codexHome,
+            authJsonPath,
+            summary: authJson.kind === "inaccessible"
+                ? `Codex login auth.json cannot be inspected: ${authJsonPath} (${authJson.errorMessage})`
+                : `Codex login auth.json is not a file: ${authJsonPath}`,
+            recommendation: codexLoginRecommendation(codexHome),
+        };
+    }
+    return {
+        checked: true,
+        ready: true,
+        codexHome,
+        authJsonPath,
+        summary: `Codex login auth is available at ${codexHome}.`,
+    };
+}

package/dist/core/codex-workflow.js

@@ -1,5 +1,4 @@
 import path from "node:path";
-import { mkdirSync } from "node:fs";
 import childProcess from "node:child_process";
 import { syncBuiltinESMExports } from "node:module";
 import { readWorkflowResult } from "./workflow-result.js";
@@ -54,9 +53,6 @@ async function loadCodexSdk() {
 }
 function normalizeEnv(input) {
     const agentSettings = resolveAgentSettings(input.env);
-    if (agentSettings.authMode === "codex-login" && agentSettings.codexHome) {
-        mkdirSync(agentSettings.codexHome, { recursive: true });
-    }
     const normalized = {};
     for (const [key, value] of Object.entries({
         ...process.env,

package/dist/core/onboarding.js

@@ -2,6 +2,7 @@ import { accessSync, constants, readFileSync } from "node:fs";
 import path from "node:path";
 import { fileURLToPath } from "node:url";
 import { confirm, input, password, select } from "@inquirer/prompts";
+import { inspectWikiAgentCodexLogin } from "./agent-auth.js";
 import { DEFAULT_WIKI_ENV_FILE, getCliEnvironmentInfo, parseEnvFile, serializeEnvEntries } from "./cli-env.js";
 import { resolveTemplateFilePath, loadConfig } from "./config.js";
 import { DEFAULT_EMBEDDING_DIMENSIONS, EmbeddingClient } from "./embedding.js";
@@ -496,7 +497,7 @@ async function collectAgentSettings(driver, ctx, defaults) {
             {
                 value: "codex-login",
                 label: "codex-login",
-                description: "Use a local Codex ChatGPT login stored under WIKI_AGENT_CODEX_HOME.",
+                description: "Use the current user's Codex login by default (~/.codex); set WIKI_AGENT_CODEX_HOME for isolation.",
             },
             {
                 value: "api-key",
@@ -792,6 +793,11 @@ export async function runSetupWizard(env = process.env, options = {}) {
             `- Example: cd ${JSON.stringify(workspaceRoot)} && tiangong-wiki init`,
             "- Run `tiangong-wiki doctor` to validate the generated configuration.",
             "- Run `tiangong-wiki init` to create index.db and perform the first sync.",
+            ...(values.agentEnabled && values.agentAuthMode === "codex-login" && values.agentCodexHome
+                ? [
+                    `- Codex login auth uses ${values.agentCodexHome}; if needed, run \`CODEX_HOME=${values.agentCodexHome} codex login\` before starting the daemon.`,
+                ]
+                : []),
             ...(values.vaultSource === "synology"
                 ? ["- Protect `.wiki.env` carefully because it now stores Synology credentials."]
                 : []),
@@ -969,6 +975,13 @@ function inspectAgent(checks, env) {
             collectDoctorCheck(checks, "error", "agent", `Automatic vault processing is enabled but missing: ${settings.missing.join(", ")}`, "Set the missing WIKI_AGENT_* values in `.wiki.env` or rerun `tiangong-wiki setup`.");
             return;
         }
+        if (settings.authMode === "codex-login") {
+            const codexLogin = inspectWikiAgentCodexLogin(settings);
+            if (!codexLogin.ready) {
+                collectDoctorCheck(checks, "error", "agent", codexLogin.summary, codexLogin.recommendation);
+                return;
+            }
+        }
         collectDoctorCheck(checks, "ok", "agent", settings.authMode === "codex-login"
             ? `Automatic vault processing is enabled with model ${settings.model} via Codex login at ${settings.codexHome}.`
             : `Automatic vault processing is enabled with model ${settings.model} via API key auth.`);

package/dist/core/paths.js

@@ -88,7 +88,7 @@ function normalizeOptionalAbsolutePath(label, rawValue) {
     return path.resolve(value);
 }
 export function defaultWikiAgentCodexHome() {
-    return path.join(os.homedir(), ".codex-tiangong-wiki");
+    return path.join(os.homedir(), ".codex");
 }
 export function parseVaultHashMode(raw) {
     const value = (raw ?? "content").trim().toLowerCase();

package/dist/core/workflow-context.js

@@ -83,6 +83,8 @@ export function buildVaultWorkflowPrompt(input) {
         "",
         "1. Read queue-item.json next to RESULT_JSON_PATH.",
         "2. Read the target vault file at VAULT_FILE_PATH. Refer to `references/vault-to-wiki-instruction.md` (Phase 1) in the wiki package for file-type-specific reading strategies, parser skill discovery, image handling, and metadata utilization.",
+        "   - If `WIKI_PARSER_SKILLS` includes `document-granular-decompose` and `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` are set, prefer that skill for supported document/image formats before the legacy type-specific parser skills.",
+        "   - When using `document-granular-decompose`, request `return_txt=true`, treat the pure text extracted from `response.txt`/`txt` as the main input, and keep raw JSON only for debugging or page-number evidence.",
         "3. Discover the current page type ontology via `tiangong-wiki type list` and `tiangong-wiki type show <type>`. Do not assume any type, template, or default target type.",
         "4. Search the existing wiki for overlapping or related content:",
         "   - Use `tiangong-wiki fts` and `tiangong-wiki search` with key terms from the source.",

package/dist/core/workspace-skills.js

@@ -8,14 +8,21 @@ import { copyDirectoryContentsSync, ensureDirSync, pathExistsSync, readTextFileS
 import { resolveRuntimePaths } from "./paths.js";
 import { toOffsetIso } from "../utils/time.js";
 export const PARSER_SKILL_SOURCE = "https://github.com/anthropics/skills";
+export const TIANGONG_PARSER_SKILL_SOURCE = "https://github.com/tiangong-ai/skills";
 export const MANAGED_SKILL_STATE_FILE = ".tiangong-wiki-skill.json";
 export const OPTIONAL_PARSER_SKILLS = [
-    { name: "pdf", summary: "Process PDF files" },
-    { name: "docx", summary: "Process DOCX files" },
-    { name: "pptx", summary: "Process PPTX files" },
-    { name: "xlsx", summary: "Process XLSX/CSV files" },
+    { name: "pdf", summary: "Process PDF files", source: PARSER_SKILL_SOURCE },
+    { name: "docx", summary: "Process DOCX files", source: PARSER_SKILL_SOURCE },
+    { name: "pptx", summary: "Process PPTX files", source: PARSER_SKILL_SOURCE },
+    { name: "xlsx", summary: "Process XLSX/CSV files", source: PARSER_SKILL_SOURCE },
+    {
+        name: "document-granular-decompose",
+        summary: "Process documents through TianGong Unstructure and return plain fulltext",
+        source: TIANGONG_PARSER_SKILL_SOURCE,
+    },
 ];
 const OPTIONAL_PARSER_SKILL_NAMES = new Set(OPTIONAL_PARSER_SKILLS.map((skill) => skill.name));
+const OPTIONAL_PARSER_SKILL_BY_NAME = new Map(OPTIONAL_PARSER_SKILLS.map((skill) => [skill.name, skill]));
 const MANAGED_SKILL_SOURCE_KINDS = new Set([
     "workspace-package",
     "curated-parser",
@@ -228,11 +235,15 @@ function normalizeManagedSource(rawSource) {
     return source;
 }
 function createParserDescriptor(workspaceRoot, name, configured) {
+    const definition = OPTIONAL_PARSER_SKILL_BY_NAME.get(name);
+    if (!definition) {
+        throw new AppError(`Unsupported parser skill: ${name}`, "config");
+    }
     return {
         name,
         sourceKind: "curated-parser",
         configured,
-        source: PARSER_SKILL_SOURCE,
+        source: definition.source,
         skillPath: resolveWorkspaceSkillPath(workspaceRoot, name),
     };
 }
@@ -656,7 +667,11 @@ export function ensureWikiSkillInstall(wikiPath, packageRoot) {
     };
 }
 export function buildParserSkillInstallInvocation(skillName) {
-    return buildExternalSkillInstallInvocation(PARSER_SKILL_SOURCE, skillName);
+    const definition = OPTIONAL_PARSER_SKILL_BY_NAME.get(skillName);
+    if (!definition) {
+        throw new AppError(`Unsupported parser skill: ${skillName}`, "config");
+    }
+    return buildExternalSkillInstallInvocation(definition.source, skillName);
 }
 export function installParserSkill(skillName, workspaceRoot, options = {}) {
     const installed = installManagedExternalSkill(createParserDescriptor(workspaceRoot, skillName, true), {