@biaoo/tiangong-wiki 0.3.11 → 0.3.12

package/README.md

@@ -117,7 +117,7 @@ tiangong-wiki dashboard                               # open dashboard in browse
 
 > Environment variables are managed via `.wiki.env` (created by `tiangong-wiki setup`). The CLI prefers the nearest local `.wiki.env`, then falls back to the global default workspace config. See [references/troubleshooting.md](./references/troubleshooting.md) for the full reference. For a centralized Linux + `systemd` + Nginx deployment, see [references/centralized-service-deployment.md](./references/centralized-service-deployment.md). That deployment guide now also includes Git repository / GitHub remote setup for daemon-side commit and optional auto-push.
 
-For document-heavy vaults, set `WIKI_PARSER_SKILLS=document-granular-decompose` and configure `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` to prefer the TianGong Unstructure parser. `document-granular-decompose` is a broad document/image parser for PDF, Word, PowerPoint, Excel, Markdown, and common image formats; the skill's `SKILL.md` remains the source of truth for the exact extension allowlist. The wiki workflow uses `return_txt=true`, consumes the plain `txt` text as the agent input, and stores the extracted plain-text snapshot under `.queue-artifacts/<file-artifact>/extracted-fulltext.txt` with metadata in the queue result; `UNSTRUCTURED_PROVIDER` and `UNSTRUCTURED_MODEL` are optional overrides.
+For document-heavy vaults, set `WIKI_PARSER_SKILLS=document-granular-decompose` and configure `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` to prefer the TianGong Unstructure parser for supported non-text documents and images. `document-granular-decompose` is a broad document/image parser for PDF, Word, PowerPoint, Excel, and common image formats; Markdown and other plain text-like files are read locally by the wiki workflow. The skill's `SKILL.md` remains the source of truth for the exact extension allowlist. The wiki workflow uses `return_txt=true`, consumes the plain `txt` text as the agent input, and stores the extracted plain-text snapshot under `.queue-artifacts/<file-artifact>/extracted-fulltext.txt` with metadata in the queue result; `UNSTRUCTURED_PROVIDER` and `UNSTRUCTURED_MODEL` are optional overrides.
 
 ## MCP Server
 

package/README.zh-CN.md

@@ -117,7 +117,7 @@ tiangong-wiki dashboard                               # 在浏览器中打开仪
 
 > 环境变量通过 `.wiki.env` 管理（由 `tiangong-wiki setup` 创建）。CLI 会优先使用最近的本地 `.wiki.env`，找不到时再 fallback 到全局默认工作区配置。完整参考见 [references/troubleshooting.md](./references/troubleshooting.md)。如需部署中心化服务（Linux + `systemd` + Nginx），见 [references/centralized-service-deployment.md](./references/centralized-service-deployment.md)。该部署文档现在也包含了 Git 仓库初始化、GitHub remote 配置和 daemon 自动 push 的 Git 配置说明。
 
-如果 vault 里以文档解析为主，可设置 `WIKI_PARSER_SKILLS=document-granular-decompose`，并配置 `UNSTRUCTURED_API_BASE_URL` 与 `UNSTRUCTURED_AUTH_TOKEN`，让 workflow 优先使用 TianGong Unstructure parser。`document-granular-decompose` 是覆盖范围更宽的文档/图片解析器，适用于 PDF、Word、PowerPoint、Excel、Markdown 和常见图片格式；精确扩展名 allowlist 以该 skill 自身的 `SKILL.md` 为准。wiki workflow 会使用 `return_txt=true`，把返回的纯 `txt` 文本作为 agent 主输入，并把解析纯文本快照保存到 `.queue-artifacts/<file-artifact>/extracted-fulltext.txt`，同时在 queue result 里保留元数据；`UNSTRUCTURED_PROVIDER` 和 `UNSTRUCTURED_MODEL` 只是可选覆盖项。
+如果 vault 里以文档解析为主，可设置 `WIKI_PARSER_SKILLS=document-granular-decompose`，并配置 `UNSTRUCTURED_API_BASE_URL` 与 `UNSTRUCTURED_AUTH_TOKEN`，让 workflow 对支持的非纯文本文档和图片优先使用 TianGong Unstructure parser。`document-granular-decompose` 是覆盖范围更宽的文档/图片解析器，适用于 PDF、Word、PowerPoint、Excel 和常见图片格式；Markdown 与其他纯文本类文件由 wiki workflow 本地直接读取。精确扩展名 allowlist 以该 skill 自身的 `SKILL.md` 为准。wiki workflow 会使用 `return_txt=true`，把返回的纯 `txt` 文本作为 agent 主输入，并把解析纯文本快照保存到 `.queue-artifacts/<file-artifact>/extracted-fulltext.txt`，同时在 queue result 里保留元数据；`UNSTRUCTURED_PROVIDER` 和 `UNSTRUCTURED_MODEL` 只是可选覆盖项。
 
 ## MCP Server
 

package/dist/core/db.js

@@ -76,9 +76,15 @@ function ensureBaseTables(db, embeddingDimensions) {
       file_path TEXT NOT NULL,
       content_hash TEXT,
       file_mtime REAL,
+      source_timestamp TEXT,
+      source_timestamp_source TEXT,
+      source_timestamp_confidence REAL,
+      source_timestamp_candidates TEXT,
       indexed_at TEXT
     );
 
+    CREATE INDEX IF NOT EXISTS idx_vfiles_source_timestamp ON vault_files(source_timestamp);
+
     CREATE TABLE IF NOT EXISTS vault_changelog (
       id INTEGER PRIMARY KEY AUTOINCREMENT,
       file_id TEXT NOT NULL,
@@ -164,6 +170,13 @@ function ensureBaseTables(db, embeddingDimensions) {
         proposed_type_names: "TEXT",
         skills_used: "TEXT",
     });
+    ensureTableColumns(db, "vault_files", {
+        source_timestamp: "TEXT",
+        source_timestamp_source: "TEXT",
+        source_timestamp_confidence: "REAL",
+        source_timestamp_candidates: "TEXT",
+    });
+    db.exec("CREATE INDEX IF NOT EXISTS idx_vfiles_source_timestamp ON vault_files(source_timestamp)");
     if (!tableExists(db, "vec_pages")) {
         db.exec(`
       CREATE VIRTUAL TABLE vec_pages USING vec0(

package/dist/core/vault-processing.js

@@ -61,6 +61,38 @@ function parseOptionalStringArray(value) {
         return [];
     }
 }
+function parseSourceTimestampCandidates(value) {
+    if (Array.isArray(value)) {
+        return value;
+    }
+    if (typeof value !== "string" || !value.trim()) {
+        return [];
+    }
+    try {
+        const parsed = JSON.parse(value);
+        return Array.isArray(parsed) ? parsed : [];
+    }
+    catch {
+        return [];
+    }
+}
+function mapVaultFileRow(row) {
+    return {
+        id: String(row.id),
+        fileName: String(row.fileName),
+        fileExt: typeof row.fileExt === "string" ? row.fileExt : null,
+        sourceType: typeof row.sourceType === "string" ? row.sourceType : null,
+        fileSize: Number(row.fileSize ?? 0),
+        filePath: String(row.filePath),
+        contentHash: typeof row.contentHash === "string" ? row.contentHash : null,
+        fileMtime: typeof row.fileMtime === "number" ? row.fileMtime : null,
+        sourceTimestamp: typeof row.sourceTimestamp === "string" ? row.sourceTimestamp : null,
+        sourceTimestampSource: typeof row.sourceTimestampSource === "string" ? row.sourceTimestampSource : null,
+        sourceTimestampConfidence: typeof row.sourceTimestampConfidence === "number" ? row.sourceTimestampConfidence : null,
+        sourceTimestampCandidates: parseSourceTimestampCandidates(row.sourceTimestampCandidates),
+        indexedAt: String(row.indexedAt),
+    };
+}
 function mapQueueRow(row) {
     const attempts = Number(row.attempts ?? 0);
     const status = row.status;
@@ -99,6 +131,9 @@ function mapQueueRow(row) {
         sourceType: typeof row.sourceType === "string" ? row.sourceType : null,
         fileSize: typeof row.fileSize === "number" ? row.fileSize : undefined,
         filePath: typeof row.filePath === "string" ? row.filePath : undefined,
+        sourceTimestamp: typeof row.sourceTimestamp === "string" ? row.sourceTimestamp : null,
+        sourceTimestampSource: typeof row.sourceTimestampSource === "string" ? row.sourceTimestampSource : null,
+        sourceTimestampConfidence: typeof row.sourceTimestampConfidence === "number" ? row.sourceTimestampConfidence : null,
     };
 }
 function claimQueueItems(db, limit, options) {
@@ -146,6 +181,9 @@ function claimQueueItems(db, limit, options) {
         vault_files.source_type AS sourceType,
         vault_files.file_size AS fileSize,
         vault_files.file_path AS filePath,
+        vault_files.source_timestamp AS sourceTimestamp,
+        vault_files.source_timestamp_source AS sourceTimestampSource,
+        vault_files.source_timestamp_confidence AS sourceTimestampConfidence,
         vault_extractions.artifact_path AS extractedTextPath,
         vault_extractions.artifact_sha256 AS extractedTextSha256,
         vault_extractions.parser_skill AS extractedTextParserSkill,
@@ -242,6 +280,9 @@ function fetchQueueItemsByStatus(db, status) {
         vault_files.source_type AS sourceType,
         vault_files.file_size AS fileSize,
         vault_files.file_path AS filePath,
+        vault_files.source_timestamp AS sourceTimestamp,
+        vault_files.source_timestamp_source AS sourceTimestampSource,
+        vault_files.source_timestamp_confidence AS sourceTimestampConfidence,
         vault_extractions.artifact_path AS extractedTextPath,
         vault_extractions.artifact_sha256 AS extractedTextSha256,
         vault_extractions.parser_skill AS extractedTextParserSkill,
@@ -288,6 +329,9 @@ function fetchQueueItemByFileId(db, fileId) {
         vault_files.source_type AS sourceType,
         vault_files.file_size AS fileSize,
         vault_files.file_path AS filePath,
+        vault_files.source_timestamp AS sourceTimestamp,
+        vault_files.source_timestamp_source AS sourceTimestampSource,
+        vault_files.source_timestamp_confidence AS sourceTimestampConfidence,
         vault_extractions.artifact_path AS extractedTextPath,
         vault_extractions.artifact_sha256 AS extractedTextSha256,
         vault_extractions.parser_skill AS extractedTextParserSkill,
@@ -312,11 +356,15 @@ function fetchVaultFile(db, fileId) {
         file_path AS filePath,
         content_hash AS contentHash,
         file_mtime AS fileMtime,
+        source_timestamp AS sourceTimestamp,
+        source_timestamp_source AS sourceTimestampSource,
+        source_timestamp_confidence AS sourceTimestampConfidence,
+        source_timestamp_candidates AS sourceTimestampCandidates,
         indexed_at AS indexedAt
       FROM vault_files
       WHERE id = ?
     `).get(fileId);
-    return row ?? null;
+    return row ? mapVaultFileRow(row) : null;
 }
 function buildProcessingOwnerId() {
     return `${os.hostname()}:${process.pid}:${Date.now()}:${randomUUID().slice(0, 8)}`;
@@ -386,6 +434,9 @@ function fetchStaleProcessingQueueItems(db, processingOwnerId) {
         vault_files.source_type AS sourceType,
         vault_files.file_size AS fileSize,
         vault_files.file_path AS filePath,
+        vault_files.source_timestamp AS sourceTimestamp,
+        vault_files.source_timestamp_source AS sourceTimestampSource,
+        vault_files.source_timestamp_confidence AS sourceTimestampConfidence,
         vault_extractions.artifact_path AS extractedTextPath,
         vault_extractions.artifact_sha256 AS extractedTextSha256,
         vault_extractions.parser_skill AS extractedTextParserSkill,

package/dist/core/vault.js

@@ -19,6 +19,164 @@ function normalizeVaultFileExtension(filePath) {
     const fileExt = path.extname(filePath).replace(/^\./, "").toLowerCase();
     return fileExt || null;
 }
+function normalizeFileMtimeMs(fileMtime) {
+    if (typeof fileMtime !== "number" || !Number.isFinite(fileMtime) || fileMtime <= 0) {
+        return null;
+    }
+    return fileMtime < 1_000_000_000_000 ? fileMtime * 1000 : fileMtime;
+}
+function isValidDateParts(year, month, day, hour, minute, second) {
+    if (year < 1900 || year > 2099 || month < 1 || month > 12 || day < 1 || day > 31) {
+        return false;
+    }
+    if (hour < 0 || hour > 23 || minute < 0 || minute > 59 || second < 0 || second > 59) {
+        return false;
+    }
+    const date = new Date(year, month - 1, day, hour, minute, second);
+    return (date.getFullYear() === year &&
+        date.getMonth() === month - 1 &&
+        date.getDate() === day &&
+        date.getHours() === hour &&
+        date.getMinutes() === minute &&
+        date.getSeconds() === second);
+}
+function buildTimestampCandidate(input) {
+    const year = Number.parseInt(input.year, 10);
+    const month = Number.parseInt(input.month, 10);
+    const day = Number.parseInt(input.day, 10);
+    const hour = input.hour ? Number.parseInt(input.hour, 10) : 0;
+    const minute = input.minute ? Number.parseInt(input.minute, 10) : 0;
+    const second = input.second ? Number.parseInt(input.second, 10) : 0;
+    if (!isValidDateParts(year, month, day, hour, minute, second)) {
+        return null;
+    }
+    const precision = input.hour && input.minute ? "datetime" : "date";
+    const baseConfidence = input.source === "file_name" ? 0.9 : 0.8;
+    return {
+        timestamp: toOffsetIso(new Date(year, month - 1, day, hour, minute, second)),
+        source: input.source,
+        confidence: precision === "datetime" ? baseConfidence + 0.05 : baseConfidence,
+        precision,
+        raw: input.raw,
+    };
+}
+function collectDateCandidatesFromText(text, source) {
+    const candidates = [];
+    const separated = /(^|[^0-9])((?:19|20)\d{2})[-_.年/]([01]?\d)[-_.月/]([0-3]?\d)(?:[日\sT_@-]+([0-2]?\d)[:：]?([0-5]\d)(?:[:：]?([0-5]\d))?)?(?=$|[^0-9])/g;
+    const compact = /(^|[^0-9])((?:19|20)\d{2})([01]\d)([0-3]\d)(?:[T_\s@-]?([0-2]\d)([0-5]\d)([0-5]\d)?)?(?=$|[^0-9])/g;
+    for (const match of text.matchAll(separated)) {
+        const raw = match[0].slice(match[1].length);
+        const candidate = buildTimestampCandidate({
+            year: match[2],
+            month: match[3],
+            day: match[4],
+            hour: match[5],
+            minute: match[6],
+            second: match[7],
+            raw,
+            source,
+        });
+        if (candidate) {
+            candidates.push(candidate);
+        }
+    }
+    for (const match of text.matchAll(compact)) {
+        const raw = match[0].slice(match[1].length);
+        const candidate = buildTimestampCandidate({
+            year: match[2],
+            month: match[3],
+            day: match[4],
+            hour: match[5],
+            minute: match[6],
+            second: match[7],
+            raw,
+            source,
+        });
+        if (candidate) {
+            candidates.push(candidate);
+        }
+    }
+    return candidates;
+}
+function dedupeTimestampCandidates(candidates) {
+    const seen = new Set();
+    const result = [];
+    for (const candidate of candidates) {
+        const key = `${candidate.timestamp}:${candidate.source}:${candidate.raw}`;
+        if (seen.has(key)) {
+            continue;
+        }
+        seen.add(key);
+        result.push(candidate);
+    }
+    return result;
+}
+function inferVaultSourceTimestamp(input) {
+    const fileNameWithoutExt = input.fileName.replace(/\.[^.]+$/, "");
+    const directory = path.posix.dirname(input.id);
+    const pathText = directory === "." ? "" : directory;
+    const candidates = dedupeTimestampCandidates([
+        ...collectDateCandidatesFromText(fileNameWithoutExt, "file_name"),
+        ...collectDateCandidatesFromText(pathText, "path"),
+    ]);
+    const mtimeMs = normalizeFileMtimeMs(input.fileMtime);
+    if (mtimeMs !== null) {
+        candidates.push({
+            timestamp: toOffsetIso(new Date(mtimeMs)),
+            source: "file_mtime",
+            confidence: 0.5,
+            precision: "datetime",
+            raw: String(input.fileMtime),
+        });
+    }
+    const preferred = candidates
+        .slice()
+        .sort((left, right) => right.confidence - left.confidence || left.timestamp.localeCompare(right.timestamp))[0];
+    return {
+        sourceTimestamp: preferred?.timestamp ?? null,
+        sourceTimestampSource: preferred?.source ?? null,
+        sourceTimestampConfidence: preferred?.confidence ?? null,
+        sourceTimestampCandidates: candidates,
+    };
+}
+function serializeSourceTimestampCandidates(candidates) {
+    if (!candidates || candidates.length === 0) {
+        return null;
+    }
+    return JSON.stringify(candidates);
+}
+function parseSourceTimestampCandidates(value) {
+    if (Array.isArray(value)) {
+        return value;
+    }
+    if (typeof value !== "string" || value.trim().length === 0) {
+        return [];
+    }
+    try {
+        const parsed = JSON.parse(value);
+        return Array.isArray(parsed) ? parsed : [];
+    }
+    catch {
+        return [];
+    }
+}
+function mapVaultFileRow(row) {
+    return {
+        id: String(row.id),
+        fileName: String(row.fileName),
+        fileExt: typeof row.fileExt === "string" ? row.fileExt : null,
+        sourceType: typeof row.sourceType === "string" ? row.sourceType : null,
+        fileSize: Number(row.fileSize ?? 0),
+        filePath: String(row.filePath),
+        contentHash: typeof row.contentHash === "string" ? row.contentHash : null,
+        fileMtime: typeof row.fileMtime === "number" ? row.fileMtime : null,
+        sourceTimestamp: typeof row.sourceTimestamp === "string" ? row.sourceTimestamp : null,
+        sourceTimestampSource: typeof row.sourceTimestampSource === "string" ? row.sourceTimestampSource : null,
+        sourceTimestampConfidence: typeof row.sourceTimestampConfidence === "number" ? row.sourceTimestampConfidence : null,
+        sourceTimestampCandidates: parseSourceTimestampCandidates(row.sourceTimestampCandidates),
+        indexedAt: String(row.indexedAt),
+    };
+}
 function createAllowedVaultFileTypeSet(vaultFileTypes) {
     return new Set(vaultFileTypes.map((item) => item.trim().replace(/^\./, "").toLowerCase()).filter(Boolean));
 }
@@ -42,6 +200,11 @@ function localVaultFiles(vaultPath, hashMode, vaultFileTypes) {
             filePath,
             contentHash: computeVaultHash(hashMode, id, filePath, stats.size, stats.mtimeMs),
             fileMtime: stats.mtimeMs,
+            ...inferVaultSourceTimestamp({
+                id,
+                fileName: path.basename(filePath),
+                fileMtime: stats.mtimeMs,
+            }),
             indexedAt,
         };
     });
@@ -162,6 +325,11 @@ async function scanSynologyFolder(client, remoteRoot, currentFolder, results, al
             filePath,
             contentHash: sha256Text(`${relativeId}:${filePath}:${fileSize}:${fileMtime}`),
             fileMtime,
+            ...inferVaultSourceTimestamp({
+                id: relativeId,
+                fileName: item.name ?? path.basename(filePath),
+                fileMtime,
+            }),
             indexedAt,
         });
     }
@@ -198,10 +366,17 @@ function getExistingVaultFiles(db) {
         file_path AS filePath,
         content_hash AS contentHash,
         file_mtime AS fileMtime,
+        source_timestamp AS sourceTimestamp,
+        source_timestamp_source AS sourceTimestampSource,
+        source_timestamp_confidence AS sourceTimestampConfidence,
+        source_timestamp_candidates AS sourceTimestampCandidates,
         indexed_at AS indexedAt
       FROM vault_files
     `).all();
-    return new Map(rows.map((row) => [row.id, row]));
+    return new Map(rows.map((row) => {
+        const file = mapVaultFileRow(row);
+        return [file.id, file];
+    }));
 }
 export function getVaultQueuePriority(fileExt) {
     const normalized = (fileExt ?? "").toLowerCase();
@@ -406,9 +581,13 @@ export function syncVaultIndex(db, currentFiles, syncId) {
     }
     const upsertStatement = db.prepare(`
       INSERT INTO vault_files(
-        id, file_name, file_ext, source_type, file_size, file_path, content_hash, file_mtime, indexed_at
+        id, file_name, file_ext, source_type, file_size, file_path, content_hash, file_mtime,
+        source_timestamp, source_timestamp_source, source_timestamp_confidence, source_timestamp_candidates,
+        indexed_at
       ) VALUES (
-        @id, @file_name, @file_ext, @source_type, @file_size, @file_path, @content_hash, @file_mtime, @indexed_at
+        @id, @file_name, @file_ext, @source_type, @file_size, @file_path, @content_hash, @file_mtime,
+        @source_timestamp, @source_timestamp_source, @source_timestamp_confidence, @source_timestamp_candidates,
+        @indexed_at
       )
       ON CONFLICT(id) DO UPDATE SET
         file_name = excluded.file_name,
@@ -418,6 +597,10 @@ export function syncVaultIndex(db, currentFiles, syncId) {
         file_path = excluded.file_path,
         content_hash = excluded.content_hash,
         file_mtime = excluded.file_mtime,
+        source_timestamp = excluded.source_timestamp,
+        source_timestamp_source = excluded.source_timestamp_source,
+        source_timestamp_confidence = excluded.source_timestamp_confidence,
+        source_timestamp_candidates = excluded.source_timestamp_candidates,
         indexed_at = excluded.indexed_at
     `);
     const insertChange = db.prepare(`
@@ -573,6 +756,10 @@ export function syncVaultIndex(db, currentFiles, syncId) {
                 file_path: file.filePath,
                 content_hash: file.contentHash,
                 file_mtime: file.fileMtime,
+                source_timestamp: file.sourceTimestamp ?? null,
+                source_timestamp_source: file.sourceTimestampSource ?? null,
+                source_timestamp_confidence: file.sourceTimestampConfidence ?? null,
+                source_timestamp_candidates: serializeSourceTimestampCandidates(file.sourceTimestampCandidates),
                 indexed_at: file.indexedAt,
             });
         }

package/dist/core/workflow-context.js

@@ -85,9 +85,11 @@ export function buildVaultWorkflowPrompt(input) {
         "",
         "1. Read queue-item.json next to RESULT_JSON_PATH.",
         "2. Read the target vault file at VAULT_FILE_PATH. Refer to `references/vault-to-wiki-instruction.md` (Phase 1) in the wiki package for file-type-specific reading strategies, parser skill discovery, image handling, and metadata utilization.",
-        "   - If `WIKI_PARSER_SKILLS` includes `document-granular-decompose` and `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` are set, prefer that skill for supported document/image files before the legacy type-specific parser skills. This includes PDF, Word, PowerPoint, Excel, Markdown, and common image formats; use the skill's `SKILL.md` for the exact extension allowlist.",
+        "   - Plain text-like files (`.txt`, `.md`, `.markdown`, `.json`, `.csv`, `.tsv`, `.yaml`, `.yml`) must be read directly from VAULT_FILE_PATH. Do not send them to parser skills or remote unstructure APIs; write the direct plain-text snapshot to EXTRACTED_TEXT_PATH when it becomes the analysis input.",
+        "   - If `WIKI_PARSER_SKILLS` includes `document-granular-decompose` and `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` are set, prefer that skill for supported non-text document/image files before the legacy type-specific parser skills. This includes PDF, Word, PowerPoint, Excel, and common image formats; use the skill's `SKILL.md` for the exact extension allowlist.",
         "   - When using `document-granular-decompose`, request `return_txt=true`, treat the pure text extracted from `response.txt`/`txt` as the main input, and keep raw JSON only for debugging or page-number evidence.",
         "   - If you extract plain text through any parser skill, write that canonical plain text snapshot to EXTRACTED_TEXT_PATH. For `document-granular-decompose`, write the same pure text from `response.txt`/`txt`. Leave EXTRACTED_TEXT_PATH empty only when no extractable text exists.",
+        "   - queue-item.json may include `file.sourceTimestamp`, `file.sourceTimestampSource`, `file.sourceTimestampConfidence`, and timestamp candidates inferred from the source file name, path, or mtime. Use this as evidence about source recency, but do not blindly copy it into page `createdAt` or `updatedAt`; those fields are normalized by the wiki system.",
         "3. Discover the current page type ontology via `tiangong-wiki type list` and `tiangong-wiki type show <type>`. Do not assume any type, template, or default target type.",
         "4. Search the existing wiki for overlapping or related content:",
         "   - Use `tiangong-wiki fts` and `tiangong-wiki search` with key terms from the source.",

package/dist/operations/dashboard.js

@@ -287,6 +287,9 @@ function buildQueueListItem(item) {
         sourceType: item.sourceType ?? null,
         fileSize: item.fileSize ?? null,
         filePath: item.filePath ?? null,
+        sourceTimestamp: item.sourceTimestamp ?? null,
+        sourceTimestampSource: item.sourceTimestampSource ?? null,
+        sourceTimestampConfidence: item.sourceTimestampConfidence ?? null,
         createdPageIds: item.createdPageIds ?? [],
         updatedPageIds: item.updatedPageIds ?? [],
         appliedTypeNames: item.appliedTypeNames ?? [],
@@ -430,6 +433,10 @@ async function resolvePageVaultSource(db, config, env, page, rawData) {
         file_path AS filePath,
         content_hash AS contentHash,
         file_mtime AS fileMtime,
+        source_timestamp AS sourceTimestamp,
+        source_timestamp_source AS sourceTimestampSource,
+        source_timestamp_confidence AS sourceTimestampConfidence,
+        source_timestamp_candidates AS sourceTimestampCandidates,
         indexed_at AS indexedAt
       FROM vault_files
       WHERE id = ?
@@ -451,6 +458,9 @@ async function resolvePageVaultSource(db, config, env, page, rawData) {
         sourceType: row.sourceType,
         fileSize: row.fileSize,
         remotePath: row.filePath,
+        sourceTimestamp: row.sourceTimestamp ?? null,
+        sourceTimestampSource: row.sourceTimestampSource ?? null,
+        sourceTimestampConfidence: row.sourceTimestampConfidence ?? null,
         indexedAt: row.indexedAt,
         ...preview,
     };
@@ -922,6 +932,10 @@ export async function openDashboardPageSource(env = process.env, inputPageId, ta
           file_path AS filePath,
           content_hash AS contentHash,
           file_mtime AS fileMtime,
+          source_timestamp AS sourceTimestamp,
+          source_timestamp_source AS sourceTimestampSource,
+          source_timestamp_confidence AS sourceTimestampConfidence,
+          source_timestamp_candidates AS sourceTimestampCandidates,
           indexed_at AS indexedAt
         FROM vault_files
         WHERE id = ?
@@ -955,6 +969,10 @@ export function getDashboardVaultSummary(env = process.env) {
           file_path AS filePath,
           content_hash AS contentHash,
           file_mtime AS fileMtime,
+          source_timestamp AS sourceTimestamp,
+          source_timestamp_source AS sourceTimestampSource,
+          source_timestamp_confidence AS sourceTimestampConfidence,
+          source_timestamp_candidates AS sourceTimestampCandidates,
           indexed_at AS indexedAt
         FROM vault_files
         ORDER BY id
@@ -1027,6 +1045,10 @@ export function listDashboardVaultFiles(env = process.env, options = {}) {
           file_path AS filePath,
           content_hash AS contentHash,
           file_mtime AS fileMtime,
+          source_timestamp AS sourceTimestamp,
+          source_timestamp_source AS sourceTimestampSource,
+          source_timestamp_confidence AS sourceTimestampConfidence,
+          source_timestamp_candidates AS sourceTimestampCandidates,
           indexed_at AS indexedAt
         FROM vault_files
         ORDER BY id
@@ -1053,6 +1075,9 @@ export function listDashboardVaultFiles(env = process.env, options = {}) {
                 sourceType: file.sourceType,
                 fileSize: file.fileSize,
                 filePath: file.filePath,
+                sourceTimestamp: file.sourceTimestamp ?? null,
+                sourceTimestampSource: file.sourceTimestampSource ?? null,
+                sourceTimestampConfidence: file.sourceTimestampConfidence ?? null,
                 indexedAt: file.indexedAt,
                 queueStatus: queueItem?.status ?? "not-queued",
                 queueItem: queueItem ? buildQueueListItem(queueItem) : null,
@@ -1097,6 +1122,10 @@ export async function getDashboardVaultFileDetail(env = process.env, fileId) {
           file_path AS filePath,
           content_hash AS contentHash,
           file_mtime AS fileMtime,
+          source_timestamp AS sourceTimestamp,
+          source_timestamp_source AS sourceTimestampSource,
+          source_timestamp_confidence AS sourceTimestampConfidence,
+          source_timestamp_candidates AS sourceTimestampCandidates,
           indexed_at AS indexedAt
         FROM vault_files
         WHERE id = ?
@@ -1134,6 +1163,10 @@ export async function openDashboardVaultFile(env = process.env, fileId) {
           file_path AS filePath,
           content_hash AS contentHash,
           file_mtime AS fileMtime,
+          source_timestamp AS sourceTimestamp,
+          source_timestamp_source AS sourceTimestampSource,
+          source_timestamp_confidence AS sourceTimestampConfidence,
+          source_timestamp_candidates AS sourceTimestampCandidates,
           indexed_at AS indexedAt
         FROM vault_files
         WHERE id = ?

package/dist/operations/query.js

@@ -67,6 +67,21 @@ function normalizeOptionalString(value) {
     const normalized = value.trim();
     return normalized ? normalized : null;
 }
+function parseOptionalJsonArray(value) {
+    if (Array.isArray(value)) {
+        return value;
+    }
+    if (typeof value !== "string" || !value.trim()) {
+        return [];
+    }
+    try {
+        const parsed = JSON.parse(value);
+        return Array.isArray(parsed) ? parsed : [];
+    }
+    catch {
+        return [];
+    }
+}
 function isAbsoluteLikePath(value) {
     return path.isAbsolute(value) || /^[A-Za-z]:[\\/]/.test(value);
 }
@@ -469,7 +484,7 @@ export function listVaultFiles(env = process.env, options = {}) {
             clauses.push("file_ext = ?");
             params.push(String(options.ext).replace(/^\./, ""));
         }
-        return db
+        const rows = db
             .prepare(`
           SELECT
             id,
@@ -480,12 +495,20 @@ export function listVaultFiles(env = process.env, options = {}) {
             file_path AS filePath,
             content_hash AS contentHash,
             file_mtime AS fileMtime,
+            source_timestamp AS sourceTimestamp,
+            source_timestamp_source AS sourceTimestampSource,
+            source_timestamp_confidence AS sourceTimestampConfidence,
+            source_timestamp_candidates AS sourceTimestampCandidates,
             indexed_at AS indexedAt
           FROM vault_files
           ${clauses.length > 0 ? `WHERE ${clauses.join(" AND ")}` : ""}
           ORDER BY id
         `)
             .all(...params);
+        return rows.map((row) => ({
+            ...row,
+            sourceTimestampCandidates: parseOptionalJsonArray(row.sourceTimestampCandidates),
+        }));
     }
     finally {
         db.close();

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@biaoo/tiangong-wiki",
-  "version": "0.3.11",
+  "version": "0.3.12",
   "description": "Local-first wiki index and query engine for Markdown knowledge pages (Tiangong Wiki).",
   "type": "module",
   "publishConfig": {

package/references/cli-interface.md

@@ -293,7 +293,7 @@ tiangong-wiki vault diff [--since <date>] [--path <prefix>]
 tiangong-wiki vault queue [--status pending|processing|done|skipped|error]
 ```
 
-- `list` — List indexed vault files; `--path` does prefix matching on relative paths
+- `list` — List indexed vault files; `--path` does prefix matching on relative paths. JSON output includes source timestamp inference fields when available (`sourceTimestamp`, `sourceTimestampSource`, `sourceTimestampConfidence`, `sourceTimestampCandidates`).
 - `diff` — Show changes since the last sync (or since a given date with `--since`)
 - `queue` — Show processing queue status and item details, including extracted plain-text artifact metadata when a parser snapshot exists
 

package/references/troubleshooting.md

@@ -91,13 +91,13 @@ The agent uses [Codex SDK](https://www.npmjs.com/package/@openai/codex-sdk) to p
 | `WIKI_AGENT_MODEL` | No | Model name (default: `gpt-5.5`; e.g. `Qwen/Qwen3.5-397B-A17B-GPTQ-Int4`) |
 | `WIKI_AGENT_BATCH_SIZE` | No | Max concurrent vault queue workers per cycle (default: `5`) |
 | `WIKI_AGENT_SANDBOX_MODE` | No | Codex sandbox mode: `danger-full-access` (default) or `workspace-write` |
-| `WIKI_PARSER_SKILLS` | No | Comma-separated parser skill list (e.g. `pdf,docx,pptx,xlsx,document-granular-decompose`). `document-granular-decompose` covers PDF, Office, Markdown, and common image formats; use its own `SKILL.md` for the exact extension allowlist |
+| `WIKI_PARSER_SKILLS` | No | Comma-separated parser skill list (e.g. `pdf,docx,pptx,xlsx,document-granular-decompose`). `document-granular-decompose` covers PDF, Office, and common image formats; Markdown and other plain text-like files are read locally by the wiki workflow. Use the skill's own `SKILL.md` for the exact extension allowlist |
 | `UNSTRUCTURED_API_BASE_URL` | For `document-granular-decompose` | TianGong Unstructure API base URL |
 | `UNSTRUCTURED_AUTH_TOKEN` | For `document-granular-decompose` | Bearer token for TianGong Unstructure |
 | `UNSTRUCTURED_PROVIDER` | No | Optional provider override passed to TianGong Unstructure |
 | `UNSTRUCTURED_MODEL` | No | Optional model override passed to TianGong Unstructure |
 
-When `document-granular-decompose` is configured with `UNSTRUCTURED_API_BASE_URL` and `UNSTRUCTURED_AUTH_TOKEN`, the wiki agent prefers it before the type-specific `pdf`, `docx`, `pptx`, and `xlsx` skills for supported document/image files. Keep the type-specific skills configured only when you want them available as fallback tools.
+When `document-granular-decompose` is configured with `UNSTRUCTURED_API_BASE_URL` and `UNSTRUCTURED_AUTH_TOKEN`, the wiki agent prefers it before the type-specific `pdf`, `docx`, `pptx`, and `xlsx` skills for supported non-text document/image files. Keep the type-specific skills configured only when you want them available as fallback tools.
 
 For successful parser runs, the workflow keeps the exact plain-text extraction used by the agent at `.queue-artifacts/<file-artifact>/extracted-fulltext.txt`. `tiangong-wiki vault queue` exposes `extractedTextPath`, `extractedTextSha256`, `extractedTextParserSkill`, and `extractedTextCharCount` when a snapshot exists.
 

package/references/vault-to-wiki-instruction.md

@@ -28,9 +28,9 @@ Parser skills are installed under `<workspace-root>/.agents/skills/`. Do not ass
 | `docx` | Extract text and structure from DOCX files |
 | `pptx` | Extract text, slide structure, and speaker notes from PPTX files |
 | `xlsx` | Extract tables and data from XLSX/CSV files |
-| `document-granular-decompose` | Extract fulltext from PDF, Office documents, Markdown, and common image formats through TianGong Unstructure |
+| `document-granular-decompose` | Extract fulltext from PDF, Office documents, and common image formats through TianGong Unstructure |
 
-When `document-granular-decompose` is available, `WIKI_PARSER_SKILLS` includes it, and `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` are set, prefer it for supported document/image formats before the type-specific parser skills below. This includes PDF, Word, PowerPoint, Excel, Markdown, and common image formats; read the skill's own `SKILL.md` for the exact extension allowlist. The client should request JSON with `return_txt=true`, then use the plain text from `response.txt` / `txt` as the wiki agent's primary input. Write that same plain text to `EXTRACTED_TEXT_PATH` so the queue artifact retains the exact text snapshot used for analysis. Keep JSON chunks and page numbers only for debugging or provenance evidence.
+When `document-granular-decompose` is available, `WIKI_PARSER_SKILLS` includes it, and `UNSTRUCTURED_API_BASE_URL` plus `UNSTRUCTURED_AUTH_TOKEN` are set, prefer it for supported non-text document/image formats before the type-specific parser skills below. This includes PDF, Word, PowerPoint, Excel, and common image formats; read the skill's own `SKILL.md` for the exact extension allowlist. The client should request JSON with `return_txt=true`, then use the plain text from `response.txt` / `txt` as the wiki agent's primary input. Write that same plain text to `EXTRACTED_TEXT_PATH` so the queue artifact retains the exact text snapshot used for analysis. Keep JSON chunks and page numbers only for debugging or provenance evidence.
 
 When any other parser skill is available and the vault file matches its type, use the skill. Read the skill's SKILL.md for interface details before invoking.
 
@@ -39,7 +39,7 @@ If a parser skill fails due to missing runtime dependencies, attempt to install
 ### File Type Strategies
 
 **Markdown / Plain Text (md, txt)**
-Read directly. For large files (>5000 lines), read in sections. Parse YAML frontmatter separately if present.
+Read directly. Do not send Markdown or other plain text-like files to parser skills or remote unstructure APIs. For large files (>5000 lines), read in sections. Parse YAML frontmatter separately if present.
 
 **PDF**
 Prefer the `pdf` parser skill. Without it: attempt direct read; if unreadable, skip. Use PDF metadata (title, author, date, subject) to inform decisions.
@@ -88,6 +88,7 @@ Use vision to understand each image in context. Extract only high-value images v
 6. `sourceRefs` may only contain existing wiki page ids. Raw file provenance belongs in the page body or a field like `vaultPath`.
 7. Only write frontmatter fields declared by the chosen type (`tiangong-wiki type show <type>`). Do not invent ad-hoc fields.
 8. If the type system cannot represent the knowledge cleanly, prefer `propose_only` unless template evolution is explicitly allowed.
+9. queue-item.json may include a source timestamp inferred from file name, path, or mtime. Use it only as source-date evidence; do not copy it blindly into page `createdAt` / `updatedAt`, which are system-normalized.
 
 ### Runtime Discovery