@aigne/doc-smith 0.8.15-beta.9 → 0.8.16-beta

package/prompts/structure/update/system-prompt.md

@@ -90,17 +90,4 @@ Analyze the user feedback to determine the intended operation:
 </operation_output_constraints>
 </operation_execution_rules>
 
-<file_tool_usage>
-1. glob: Find files matching specific patterns with advanced filtering and sorting.
-
-2. grep: Search file contents using regular expressions with multiple strategies (git grep → system grep → JavaScript fallback).
-
-3. readFile: Read file contents with intelligent binary detection, pagination, and metadata extraction.
-
-When to use Tools:
-- During document structure update, if the given context is missing or lacks referenced content, use glob/grep/readFile to obtain more context
-- When sourceIds or file content from <file_list> is needed but not provided in DataSources, use readFile to read the file content
-</file_tool_usage>
-
-
 {% include "../../common/document-structure/output-constraints.md" %}

package/prompts/structure/update/user-prompt.md

@@ -6,10 +6,11 @@
 {{allFilesPaths}}
 </file_list>
 
-<datasources>
-{{ datasources }}
-</datasources>
-
+{% if needDataSources %}
+<data_sources>
+{{ dataSourceChunk }}
+</data_sources>
+{% endif %}
 
 Initial Documentation Structure:
 <initial_document_structure>
@@ -38,5 +39,5 @@ Processing workflow:
 
 Rules:
 ** All changes must be made using Tools. **
-** Carefully check if the latest version of documentStructure data meets user requirements, must avoid duplicate Tool calls. **
+** Carefully check if the latest version of `<document_structure>` data meets user requirements, must avoid duplicate Tool calls. **
 </instructions>

package/prompts/translate/translate-document.md

@@ -5,10 +5,10 @@ You are an **Elite Polyglot Localization and Translation Specialist** with exten
 Core Mandates:
 
 1. Semantic Fidelity (Accuracy): The translation must perfectly and comprehensively convey the **entire meaning, tone, and nuance** of the source text. **No omission, addition, or distortion of the original content** is permitted.
-2. Native Fluency and Style: The resulting text must adhere strictly to the target language's **grammar, syntax, and idiomatic expressions**. The translation must **sound like it was originally written by a native speaker**, completely **free of grammatical errors** or "translationese" (literal, stiff, or unnatural phrasing).
+2. Native Fluency and Style: The resulting text must adhere strictly to the target language's **grammar, syntax, and idiomatic expressions**. The translation must **sound like it was originally written by a native speaker**, completely **free of grammatical errors**, avoid "translationese" (literal, stiff, or unnatural phrasing).
 3. Readability and Flow: The final output must be **smooth, logical, and highly readable**. Sentences must flow naturally, ensuring a pleasant and coherent reading experience for the target audience.
 4. Localization and Clarity: Where a **literal (word-for-word) translation** of a term, phrase, or idiom would be **uncommon, confusing, or ambiguous** in the target language, you must apply **localization best practices**. This means translating the **concept** into the most **idiomatic, common, and easily understandable expression** in the target language.
-5. Versatility and Scope: You are proficient in handling **any pair of requested languages** (e.g., Chinese $\leftrightarrow$ English, English $\leftrightarrow$ Japanese) and are adept at translating diverse **document types**, including but not limited to: **Technical Manuals, Business Reports, Marketing Copy/Ads, Legal Documents, Academic Papers, and General Correspondence.**
+5. Versatility and Scope: You are proficient in handling **any pair of requested languages** (e.g., Chinese <--> English, English <--> Japanese) and are adept at translating diverse **document types**, including but not limited to: **Technical Manuals, Business Reports, Marketing Copy/Ads, Legal Documents, Academic Papers, and General Correspondence.**
 
 </role_and_goal>
 
@@ -299,5 +299,5 @@ Original text as follows:
 </content>
 
 <output_constraints>
-Please **accurately** translate the content within <content> tags (excluding the outermost <content> tags) into **{{ language }}**, strictly following the translation requirements.
+Please **accurately** translate the content within `<content>` tags (excluding the outermost `<content>` tags) into **{{ language }}**, strictly following the translation requirements.
 </output_constraints>

package/prompts/utils/analyze-feedback-intent.md

@@ -0,0 +1,55 @@
+<role>
+You are a feedback intent analyzer. Your task is to determine whether data sources are needed to fulfill the user's feedback about content modifications.
+</role>
+
+<input>
+- feedback: {{feedback}}
+</input>
+
+<analysis_rules>
+**Determining Data Source Necessity:**
+
+You need to analyze the user's feedback and categorize it into different intent types. Based on the intent type, determine if data sources are required.
+
+This analyzer is generic and can be used for any content modification scenarios (documentation structure, document content, translations, etc.).
+
+**Intent Types:**
+
+1. **add** - Adding new items, sections, or content
+   - Requires data sources: **YES**
+   - Reason: Need sufficient context from codebase or related materials to generate accurate new content
+
+2. **edit** - Modifying existing content, descriptions, titles, or properties
+   - Requires data sources: **YES**
+   - Reason: Need context to ensure modifications are accurate and contextually appropriate
+
+3. **delete** - Removing items, sections, or content
+   - Requires data sources: **NO**
+   - Reason: Deletion only needs to identify what to remove, no new content generation needed
+
+4. **move** - Moving items to different positions or parent sections
+   - Requires data sources: **NO**
+   - Reason: Only changing item location in the structure, no content changes needed
+
+5. **reorder** - Changing the order of items at the same level
+   - Requires data sources: **NO**
+   - Reason: Only rearranging sequence, no content generation needed
+
+6. **mixed** - Combination of multiple intent types
+   - Requires data sources: **Depends on whether add/edit operations are included**
+   - Reason: If the feedback includes any add or edit operations, data sources are needed
+
+**Decision Logic:**
+- If the feedback contains ANY add or edit operations → `needDataSources = true`
+- If the feedback ONLY contains delete, move, or reorder operations → `needDataSources = false`
+- When in doubt, default to `needDataSources = true` to ensure sufficient context
+</analysis_rules>
+
+<output_rules>
+Return a JSON object with:
+- `needDataSources`: boolean indicating if data sources are required
+- `intentType`: the primary intent type (add, edit, delete, move, reorder, or mixed)
+- `reason`: clear explanation of why data sources are or aren't needed
+
+Analyze the feedback carefully and be conservative - when uncertain, prefer `needDataSources: true` to ensure sufficient context is available.
+</output_rules>

package/utils/constants/index.mjs

@@ -549,3 +549,41 @@ export const DOC_SMITH_DIR = ".aigne/doc-smith";
 export const TMP_DIR = ".tmp";
 export const TMP_DOCS_DIR = "docs";
 export const TMP_ASSETS_DIR = "assets";
+
+export const DOC_ACTION = {
+  translate: "translate",
+  update: "update",
+  clear: "clear",
+};
+
+// Default thinking effort level, available options: 'lite', 'standard', 'pro'
+// This level can be defined by the user in the config file to influence reasoning effort mapping
+export const DEFAULT_THINKING_EFFORT_LEVEL = "standard";
+
+// Default reasoning effort level, available options: 'minimal', 'low', 'medium', 'high'
+export const DEFAULT_REASONING_EFFORT_LEVEL = "low";
+
+export const DEFAULT_REASONING_EFFORT_VALUE = 500;
+
+export const REASONING_EFFORT_LEVELS = {
+  minimal: {
+    lite: 100,
+    standard: 300,
+    pro: 500,
+  },
+  low: {
+    lite: 200,
+    standard: 500,
+    pro: 1000,
+  },
+  medium: {
+    lite: 300,
+    standard: 800,
+    pro: 1500,
+  },
+  high: {
+    lite: 500,
+    standard: 1000,
+    pro: 2000,
+  },
+};

package/utils/deploy.mjs

@@ -31,6 +31,7 @@ export async function deploy(id, cachedUrl) {
   const result = await client.deploy({
     cachedCheckoutId: id,
     cachedPaymentUrl: cachedUrl,
+    needShortUrl: true,
     pageInfo: { successMessage: SUCCESS_MESSAGE },
     hooks: {
       [STEPS.PAYMENT_PENDING]: async ({ sessionId, paymentUrl, isResuming }) => {
@@ -71,7 +72,7 @@ export async function deploy(id, cachedUrl) {
     },
   });
 
-  const { appUrl, homeUrl, subscriptionUrl, dashboardUrl, vendors } = result;
+  const { appUrl, homeUrl, subscriptionUrl, dashboardUrl, vendors, sessionId } = result;
   const token = vendors?.[0]?.token;
 
   return {
@@ -80,5 +81,6 @@ export async function deploy(id, cachedUrl) {
     dashboardUrl,
     subscriptionUrl,
     token,
+    sessionId,
   };
 }

package/utils/docs-finder-utils.mjs

@@ -4,12 +4,11 @@ import { pathExists } from "./file-utils.mjs";
 
 /**
  * Get action-specific text based on isTranslate flag
- * @param {boolean} isTranslate - Whether this is a translation action
  * @param {string} baseText - Base text template with {action} placeholder
+ * @param {string} action - doc action type
  * @returns {string} Text with action replaced
  */
-export function getActionText(isTranslate, baseText) {
-  const action = isTranslate ? "translate" : "update";
+export function getActionText(baseText, action) {
   return baseText.replace("{action}", action);
 }
 
@@ -324,3 +323,38 @@ export async function loadDocumentStructure(outputDir) {
     return null;
   }
 }
+
+/**
+ * Build a tree structure from a flat document structure array using parentId
+ * @param {Array} documentStructure - Flat array of document structure items with path and parentId
+ * @returns {Object} Object containing rootNodes (array of root nodes) and nodeMap (Map for lookups)
+ */
+export function buildDocumentTree(documentStructure) {
+  // Create a map of nodes for easy lookup
+  const nodeMap = new Map();
+  const rootNodes = [];
+
+  // First pass: create node map
+  documentStructure.forEach((node) => {
+    nodeMap.set(node.path, {
+      ...node,
+      children: [],
+    });
+  });
+
+  // Build the tree structure using parentId
+  documentStructure.forEach((node) => {
+    if (node.parentId) {
+      const parent = nodeMap.get(node.parentId);
+      if (parent) {
+        parent.children.push(nodeMap.get(node.path));
+      } else {
+        rootNodes.push(nodeMap.get(node.path));
+      }
+    } else {
+      rootNodes.push(nodeMap.get(node.path));
+    }
+  });
+
+  return { rootNodes, nodeMap };
+}

package/utils/file-utils.mjs

@@ -13,6 +13,7 @@ import { debug } from "./debug.mjs";
 import { isGlobPattern } from "./utils.mjs";
 import { uploadFiles } from "./upload-files.mjs";
 import { extractApi } from "./extract-api.mjs";
+import { minimatch } from "minimatch";
 
 /**
  * Check if a directory is inside a git repository using git command
@@ -859,3 +860,99 @@ export async function downloadAndUploadImage(imageUrl, docsDir, appUrl, accessTo
     return { url: imageUrl, downloadFinalPath: null };
   }
 }
+
+/**
+ * Extract the path prefix from a glob pattern until the first glob character
+ */
+export function getPathPrefix(pattern) {
+  const segments = pattern.split("/");
+  const result = [];
+
+  for (const segment of segments) {
+    if (isGlobPattern(segment)) {
+      break;
+    }
+    result.push(segment);
+  }
+
+  return result.join("/") || ".";
+}
+
+/**
+ * Check if a dir matches any exclude pattern
+ */
+export function isDirExcluded(dir, excludePatterns) {
+  if (!dir || typeof dir !== "string") {
+    return false;
+  }
+
+  let normalizedDir = dir.replace(/\\/g, "/").replace(/^\.\/+/, "");
+  normalizedDir = normalizedDir.endsWith("/") ? normalizedDir : `${normalizedDir}/`;
+
+  for (const excludePattern of excludePatterns) {
+    if (minimatch(normalizedDir, excludePattern, { dot: true })) {
+      return true;
+    }
+  }
+
+  return false;
+}
+
+/**
+ * Return source paths that would be excluded by exclude patterns (files are skipped, directories use minimatch, glob patterns use path prefix heuristic)
+ */
+export async function findInvalidSourcePaths(sourcePaths, excludePatterns) {
+  if (!Array.isArray(sourcePaths) || sourcePaths.length === 0) {
+    return [];
+  }
+
+  if (!Array.isArray(excludePatterns) || excludePatterns.length === 0) {
+    return [];
+  }
+
+  const invalidPaths = [];
+
+  for (const sourcePath of sourcePaths) {
+    if (typeof sourcePath !== "string" || !sourcePath) {
+      continue;
+    }
+
+    // Skip paths starting with "!" (exclusion patterns)
+    if (sourcePath.startsWith("!")) {
+      continue;
+    }
+
+    // Skip remote URLs
+    if (isRemoteFile(sourcePath)) {
+      continue;
+    }
+
+    // Check glob pattern: use heuristic algorithm
+    if (isGlobPattern(sourcePath)) {
+      const representativePath = getPathPrefix(sourcePath);
+      if (isDirExcluded(representativePath, excludePatterns)) {
+        invalidPaths.push(sourcePath);
+      }
+      continue;
+    }
+
+    try {
+      const stats = await stat(sourcePath);
+      // Skip file
+      if (stats.isFile()) {
+        continue;
+      }
+      // Check dir with minimatch
+      if (stats.isDirectory()) {
+        if (isDirExcluded(sourcePath, excludePatterns)) {
+          invalidPaths.push(sourcePath);
+        }
+      }
+    } catch {
+      // Path doesn't exist
+      invalidPaths.push(sourcePath);
+    }
+  }
+
+  return invalidPaths;
+}

package/utils/load-config.mjs

@@ -1,7 +1,10 @@
 import fs from "node:fs/promises";
+import chalk from "chalk";
 import path from "node:path";
 import { parse } from "yaml";
 import { processConfigFields, resolveFileReferences } from "./utils.mjs";
+import { DEFAULT_EXCLUDE_PATTERNS } from "./constants/index.mjs";
+import { findInvalidSourcePaths, toDisplayPath } from "./file-utils.mjs";
 
 export default async function loadConfig({ config, appUrl }) {
   const configPath = path.isAbsolute(config) ? config : path.join(process.cwd(), config);
@@ -30,6 +33,22 @@ export default async function loadConfig({ config, appUrl }) {
     // Parse new configuration fields and convert keys to actual content
     const processedConfig = await processConfigFields(parsedConfig);
 
+    // Validate sourcePaths against exclude patterns
+    const sourcesPath = processedConfig.sourcesPath || parsedConfig.sourcesPath;
+    if (sourcesPath) {
+      const excludePatterns = [
+        ...DEFAULT_EXCLUDE_PATTERNS,
+        ...(processedConfig.excludePatterns || parsedConfig.excludePatterns || []),
+      ];
+
+      const invalidPaths = await findInvalidSourcePaths(sourcesPath, excludePatterns);
+      if (invalidPaths.length > 0) {
+        console.warn(
+          `⚠️  Some source paths have been excluded and will not be processed:\n${invalidPaths.map((p) => `  - ${chalk.yellow(p)}`).join("\n")}\n💡 Tip: You can remove these paths in ${toDisplayPath(configPath)}\n`,
+        );
+      }
+    }
+
     return {
       lastGitHead: parsedConfig.lastGitHead || "",
       ...parsedConfig,

package/agents/utils/docs-fs-actor.yaml

@@ -1,27 +0,0 @@
-type: team
-name: docs_fs_actor
-description: File system operations for documentation management
-skills:
-  - url: ../init/index.mjs
-    default_input:
-      skipIfExists: true
-  - url: ./fs.mjs
-    default_input:
-      rootDir:
-        $get: docsDir
-input_schema:
-  type: object
-  properties:
-    action:
-      type: "string"
-      enum: ["read_file", "write_file", "delete_file", "list_directory"]
-      description:
-        "The file system action to perform, available actions are: read_file, write_file, delete_file, list_directory"
-    path:
-      type: "string"
-      description: "The path to the file or directory to operate on"
-    content:
-      type: "string"
-      description: "The content to write to the file, required for write_file action"
-  required: ["action", "path"]
-task_render_mode: hide

package/agents/utils/fs.mjs

@@ -1,60 +0,0 @@
-import { mkdir, readdir, readFile, rm, writeFile } from "node:fs/promises";
-import { dirname, join } from "node:path";
-
-export default async function fs({ rootDir, action, path, content }) {
-  if (!rootDir) throw new Error("Root directory is not specified");
-
-  path = join(rootDir, path);
-
-  switch (action) {
-    case "read_file":
-      return {
-        status: "ok",
-        path,
-        content: await readFile(path, "utf-8"),
-      };
-    case "write_file": {
-      await mkdir(dirname(path), { recursive: true });
-      await writeFile(path, content || "");
-      return {
-        status: "ok",
-        path,
-        content,
-      };
-    }
-    case "delete_file":
-      await rm(path, { recursive: true, force: true });
-      return {
-        status: "ok",
-        path,
-      };
-    case "list_directory":
-      return {
-        status: "ok",
-        entries: await readdir(path, { withFileTypes: true }).then((list) =>
-          list.map((entry) => ({
-            path: join(entry.parentPath, entry.name),
-            isDirectory: entry.isDirectory(),
-          })),
-        ),
-      };
-  }
-}
-
-fs.input_schema = {
-  type: "object",
-  properties: {
-    action: {
-      type: "string",
-      enum: ["read_file", "write_file", "delete_file", "list_directory"],
-      description:
-        "The file system action to perform, available actions are: read_file, write_file, delete_file, list_directory",
-    },
-    path: { type: "string", description: "The path to the file or directory to operate on" },
-    content: {
-      type: "string",
-      description: "The content to write to the file, required for write_file action",
-    },
-  },
-  required: ["action", "path"],
-};