@aigne/doc-smith 0.9.10 → 0.9.11-beta

package/agents/publish/ai/intent.md

@@ -0,0 +1,182 @@
+# Publish Agent 功能意图
+
+## 功能概述
+
+文档发布工具，将 DocSmith 生成的文档发布到在线平台。
+
+## 功能意图
+
+DocSmith 生成的文档存储在本地 workspace 目录中，使用特定的目录结构（文件夹 + .meta.yaml + 语言文件）。用户需要将这些文档发布到在线平台供他人访问。Publish Agent 负责完成发布前检查、格式转换、图片处理和上传等全流程。
+
+## 工作流程
+
+```
+用户调用 publish
+    ↓
+1. check.mjs - 发布前检查
+   ├─ 检查配置文件存在性
+   ├─ 检查项目元数据（projectName、projectDesc、projectLogo）
+   │   └─ 如果为空，尝试自动更新
+   ├─ 检查文档结构（checkStructure）
+   └─ 检查文档内容（checkContent）
+    ↓
+2. translate-meta.mjs - 翻译元数据（可选）
+   ├─ 如果只有一种语言，跳过翻译，直接保存到缓存
+   └─ 如果有多种语言，将项目名称和描述翻译为目标语言
+    ↓
+3. publish-docs.mjs - 执行发布
+   ├─ 加载文档结构
+   ├─ docs-converter 转换文档格式
+   │   ├─ 扫描文档目录
+   │   ├─ 替换 AFS image slots 为真实图片
+   │   ├─ 处理 /sources/... 绝对路径图片 ← 本次新增
+   │   ├─ 添加 .md 后缀到内部链接
+   │   └─ 调整图片相对路径
+   ├─ 生成 _sidebar.md
+   ├─ 选择发布平台
+   └─ 上传文档到平台
+```
+
+## 核心能力
+
+1. **发布前检查**：验证配置、结构、内容的完整性和正确性
+   - 检查项目元数据（projectName、projectDesc、projectLogo）
+   - 若元数据为空，尝试自动填充更新
+2. **格式转换**：将 workspace 目录结构转换为发布格式
+3. **图片处理**：
+   - 替换 AFS image slots 为真实图片引用
+   - 处理 `/sources/...` 绝对路径图片，支持两种格式：
+     - Markdown 格式：`![alt](/sources/path/to/image.png)`
+     - HTML 格式：`<img src="/sources/path/to/image.png" ... />`
+   - 调整相对路径以适应发布后的目录结构
+4. **多平台支持**：DocSmith Cloud、自有网站、新建网站
+5. **元数据翻译**：支持将项目信息翻译为多语言
+   - 单语言时跳过翻译，直接缓存原始内容
+   - 多语言时才调用 AI 进行翻译
+
+## 输入输出
+
+### 输入
+
+- 可选输入：
+  - `appUrl`: 发布目标网站 URL
+  - `boardId`: 文档集合 ID
+  - `projectName`, `projectDesc`, `projectLogo`: 项目信息
+  - `with-branding`: 是否更新网站品牌信息
+  - `translatedMetadata`: 翻译后的元数据
+
+- 自动获取：
+  - 从 `config.yaml` 读取配置信息
+  - 从 `planning/document-structure.yaml` 读取文档结构
+  - 从 `docs/` 目录读取文档内容
+
+### 输出
+
+- 输出内容：发布结果消息
+  - 成功：文档访问 URL
+  - 失败：错误信息和建议
+
+## 约束条件
+
+### 必须遵循的规范
+
+1. **发布前必须通过检查**：元数据检查、结构检查和内容检查都必须通过
+   - 元数据检查：projectName、projectDesc、projectLogo 不能为空
+   - 若元数据为空，尝试自动更新后重新检查
+2. **图片路径转换规则**：
+   - AFS image slots → 真实图片路径
+   - `/sources/...` → 复制图片并转换为相对路径
+   - 相对路径根据文档深度调整
+3. **链接格式**：内部链接添加 `.md` 后缀
+
+### Sources 绝对路径处理
+
+文档中使用 `/sources/...` 格式引用数据源中的图片时：
+
+1. **识别**：匹配 `/sources/` 开头的图片路径，支持两种格式：
+   - **Markdown 格式**：`![alt](/sources/path/to/image.png)`
+   - **HTML 格式**：`<img src="/sources/path/to/image.png" ... />`
+2. **解析**：从 config.yaml 获取 sources 配置，依次在每个 source 中查找图片
+3. **复制**：将源图片复制到临时目录的 `sources/` 子目录（保持相同路径结构）
+4. **转换**：将绝对路径替换为相对路径（根据文档深度计算）
+
+**Markdown 格式转换示例**：
+```
+原始：![screenshot](/sources/assets/screenshot.png)
+转换后：![screenshot](../sources/assets/screenshot.png)（depth=1）
+       ![screenshot](../../sources/assets/screenshot.png)（depth=2）
+```
+
+**HTML 格式转换示例**：
+```
+原始：<img src="/sources/blocklets/core/public/logo/doubao.png" alt="Doubao" style="width: 48px;" />
+转换后：<img src="../sources/blocklets/core/public/logo/doubao.png" alt="Doubao" style="width: 48px;" />（depth=1）
+       <img src="../../sources/blocklets/core/public/logo/doubao.png" alt="Doubao" style="width: 48px;" />（depth=2）
+```
+
+### 职责边界
+
+- **必须执行**：
+  - 发布前检查配置、元数据和文档
+  - 元数据为空时尝试自动更新
+  - 转换文档格式到发布格式
+  - 处理所有图片路径（包括 sources 绝对路径）
+  - 上传文档到指定平台
+
+- **不应执行**：
+  - 不修改原始文档文件
+  - 不修改文档结构文件
+  - 不生成新文档内容
+
+- **协作方式**：
+  - 依赖 `structure-checker` 检查文档结构
+  - 依赖 `content-checker` 检查文档内容
+  - 依赖 `docs-converter.mjs` 转换文档格式
+  - 依赖 `sources-path-resolver.mjs` 解析 sources 路径
+  - 依赖 `@aigne/publish-docs` 执行实际上传
+
+## 预期结果
+
+### 成功标准
+
+1. 文档成功发布到在线平台
+2. 所有图片正确显示（包括 sources 中的图片）
+3. 所有内部链接正常工作
+4. 返回可访问的文档 URL
+
+## 错误处理
+
+### 常见错误
+
+1. **检查失败**：配置缺失、元数据不完整、结构错误、内容问题
+2. **认证失败**：无效的访问令牌
+3. **网络错误**：上传失败
+4. **图片处理失败**：图片不存在、路径无法解析
+
+### 处理策略
+
+1. **检查失败**：
+   - 元数据为空时，尝试自动更新后继续
+   - 自动更新失败或其他检查失败，抛出错误提示用户修复
+2. **认证失败**：提示用户重新认证
+3. **网络错误**：返回错误消息
+4. **图片处理**：跳过无法处理的图片并警告
+
+## 实现方式
+
+### 文件组成
+
+- `index.yaml`: Agent 配置，定义 skills 列表
+- `check.mjs`: 发布前检查
+- `translate-meta.mjs`: 元数据翻译
+- `publish-docs.mjs`: 发布主流程
+
+### 依赖工具
+
+- `utils/docs-converter.mjs`: 文档格式转换
+- `utils/sources-path-resolver.mjs`: sources 路径解析
+- `utils/config.mjs`: 配置文件操作
+- `@aigne/publish-docs`: 发布上传
+
+---
+**注意**：本文档描述功能意图，不包含具体实现细节。

package/agents/publish/check.mjs

@@ -0,0 +1,107 @@
+import { readFile } from "node:fs/promises";
+import { join } from "node:path";
+import { loadConfigFromFile, saveValueToConfig } from "../../utils/config.mjs";
+import { PATHS } from "../../utils/agent-constants.mjs";
+import checkStructure from "../structure-checker/index.mjs";
+import checkContent from "../content-checker/index.mjs";
+import { getProjectInfo } from "../../utils/project.mjs";
+
+/**
+ * Check configuration and documents before publishing
+ * @param {Object} params
+ * @param {string} params.fileName - Config file name (relative to workspace)
+ * @returns {Promise<Object>} - Result object with valid flag and message
+ */
+export default async function check({ fileName = "config.yaml" } = {}, _options) {
+  // 1. Check configuration
+  // Use PATHS.WORKSPACE_BASE to support both project and standalone modes
+  const filePath = join(PATHS.WORKSPACE_BASE, fileName);
+  const configContent = await readFile(filePath, "utf8").catch(() => null);
+
+  if (!configContent || configContent.trim() === "") {
+    throw new Error(
+      "Configuration file not found or is empty. Please ensure the workspace is properly initialized with a valid configuration file.",
+    );
+  }
+
+  const config = await loadConfigFromFile();
+
+  // 2. Check and populate project metadata if not exists
+  const missingFields = [];
+  if (!config.projectName || config.projectName.trim() === "") missingFields.push("projectName");
+  if (!config.projectDesc || config.projectDesc.trim() === "") missingFields.push("projectDesc");
+  if (!config.projectLogo || config.projectLogo.trim() === "") missingFields.push("projectLogo");
+
+  if (missingFields.length > 0) {
+    try {
+      const projectInfo = await getProjectInfo();
+
+      if (
+        missingFields.includes("projectName") &&
+        projectInfo.name &&
+        projectInfo.name.trim() !== ""
+      ) {
+        await saveValueToConfig("projectName", projectInfo.name, "Project name");
+        config.projectName = projectInfo.name;
+      }
+
+      if (
+        missingFields.includes("projectDesc") &&
+        projectInfo.description &&
+        projectInfo.description.trim() !== ""
+      ) {
+        await saveValueToConfig("projectDesc", projectInfo.description, "Project description");
+        config.projectDesc = projectInfo.description;
+      }
+
+      if (
+        missingFields.includes("projectLogo") &&
+        projectInfo.icon &&
+        projectInfo.icon.trim() !== ""
+      ) {
+        await saveValueToConfig("projectLogo", projectInfo.icon, "Project logo or icon");
+        config.projectLogo = projectInfo.icon;
+      }
+    } catch (error) {
+      console.warn("Failed to get project info:", error.message);
+    }
+  }
+
+  // 3. Check document structure
+  const structureResult = await checkStructure();
+
+  // If structure check failed and not fixed, throw error
+  if (!structureResult.valid && !structureResult.fixed) {
+    throw new Error(
+      `Document structure validation failed:\n${structureResult.message}\n\nPlease fix the structure issues before publishing.`,
+    );
+  }
+
+  // 4. Check document content
+  const contentResult = await checkContent();
+
+  // If content check failed and not fixed, throw error
+  if (!contentResult.valid && !contentResult.fixed) {
+    throw new Error(
+      `Document content validation failed:\n${contentResult.message}\n\nPlease fix the content issues before publishing.`,
+    );
+  }
+
+  return {
+    valid: true,
+    config,
+    message: "✅ Configuration and document checks passed.",
+  };
+}
+
+check.description = "Check configuration and documents before publishing";
+
+check.input_schema = {
+  type: "object",
+  properties: {
+    fileName: {
+      type: "string",
+      description: "Config file name",
+    },
+  },
+};

package/agents/publish/index.yaml

@@ -3,23 +3,18 @@ name: publish
 alias:
   - pub
   - p
-description: Publish your documentation online
+description: Publish DocSmith generated documentation to the online platform
 skills:
-  - url: ../init/index.mjs
-    default_input:
-      skipIfExists: true
-      checkOnly: true
-  - url: ../init/check.mjs
-  - url: ../utils/load-sources.mjs
-  - ../utils/ensure-document-icons.mjs
-  - translate-meta.mjs
-  - publish-docs.mjs
+  - url: ./check.mjs
+  - url: ./translate-meta.mjs
+  - url: ./publish-docs.mjs
+
 input_schema:
   type: object
   properties:
     appUrl:
       type: string
-      description: Website URL where docs will be published (optional - leave empty for interactive setup)
-    with-branding:
-      type: boolean
-      description: Update your website branding (title, description, logo)
+      description: |
+        Website URL where docs will be published. 
+        The URL from the last publish will be saved in config.yaml `appUrl` and used as the default for the next publish.
+        Optional: Only used if a specific website URL is provided by the user.

package/agents/publish/publish-docs.mjs

@@ -1,68 +1,80 @@
-import { basename, join } from "node:path";
+import { basename, join, relative } from "node:path";
 import { publishDocs as publishDocsFn } from "@aigne/publish-docs";
 import { BrokerClient } from "@blocklet/payment-broker-client/node";
 import chalk from "chalk";
 import fs from "fs-extra";
+import { joinURL } from "ufo";
 
 import {
   getAccessToken,
   getCachedAccessToken,
   getDiscussKitMountPoint,
-} from "../../utils/auth-utils.mjs";
-import {
-  CLOUD_SERVICE_URL_PROD,
-  DISCUSS_KIT_STORE_URL,
-  DOC_SMITH_DIR,
-  TMP_DIR,
-  TMP_DOCS_DIR,
-} from "../../utils/constants/index.mjs";
-import { ensureTmpDir } from "../../utils/d2-utils.mjs";
+} from "../../utils/auth.mjs";
+import { CLOUD_SERVICE_URL_PROD, DISCUSS_KIT_STORE_URL } from "../../utils/constants.mjs";
+import { PATHS } from "../../utils/agent-constants.mjs";
 import { deploy } from "../../utils/deploy.mjs";
-import { getGithubRepoUrl, loadConfigFromFile, saveValueToConfig } from "../../utils/utils.mjs";
-import updateBranding from "../utils/update-branding.mjs";
-import { isRemoteFile, downloadAndUploadImage } from "../../utils/file-utils.mjs";
-import { joinURL } from "ufo";
+import { loadConfigFromFile, saveValueToConfig } from "../../utils/config.mjs";
+import { ensureTmpDir } from "../../utils/files.mjs";
+import { getGithubRepoUrl, isValidGithubUrl } from "../../utils/git.mjs";
+import updateBranding from "../../utils/branding.mjs";
+import { generateSidebar, loadDocumentStructure } from "../../utils/docs.mjs";
+import { copyDocumentsToTemp } from "../../utils/docs-converter.mjs";
 
 const BASE_URL = process.env.DOC_SMITH_BASE_URL || CLOUD_SERVICE_URL_PROD;
 
 export default async function publishDocs(
   {
-    docsDir: rawDocsDir,
     appUrl,
-    boardId,
-    projectName,
-    projectDesc,
-    projectLogo,
-    translatedMetadata,
-    originalDocumentStructure,
+    outputDir = PATHS.PLANNING_DIR,
     "with-branding": withBrandingOption,
+    config,
+    translatedMetadata,
   },
   options,
 ) {
+  // Note: Document validation is now done in check.mjs which throws errors on failure
+
+  // Absolute path for file operations (reading docs)
+  const docsAbsolutePath = PATHS.DOCS_DIR;
+  // Relative path for mediaFolder (relative to cwd for publish-docs library)
+  const docsRelativePath = relative(process.cwd(), PATHS.DOCS_DIR) || "./docs";
+  // Relative path for tmp directory
+  const tmpDirRelative = relative(process.cwd(), PATHS.TMP_DIR) || ".tmp";
+  const docsDir = join(tmpDirRelative, "docs");
   let message;
   let shouldWithBranding = withBrandingOption || false;
 
   try {
+    // Load document structure from output directory
+    const documentStructure = await loadDocumentStructure(outputDir);
+    if (!documentStructure || documentStructure.length === 0) {
+      console.warn("⚠️  No document structure found. Sidebar generation may be limited.");
+    }
+
     // move work dir to tmp-dir
     await ensureTmpDir();
-
-    const docsDir = join(DOC_SMITH_DIR, TMP_DIR, TMP_DOCS_DIR);
     await fs.rm(docsDir, { recursive: true, force: true });
     await fs.mkdir(docsDir, {
       recursive: true,
     });
-    await fs.cp(rawDocsDir, docsDir, { recursive: true });
+
+    // Convert documents from new directory format to publish format
+    await copyDocumentsToTemp(docsAbsolutePath, docsDir);
+
+    // Generate _sidebar.md in tmp directory
+    const sidebar = generateSidebar(documentStructure || []);
+    const tmpSidebarPath = join(docsDir, "_sidebar.md");
+    await fs.writeFile(tmpSidebarPath, sidebar, "utf8");
 
     // ----------------- main publish process flow -----------------------------
     // Check if DOC_DISCUSS_KIT_URL is set in environment variables
-    const useEnvAppUrl = !!(
-      process.env.DOC_SMITH_PUBLISH_URL ||
-      process.env.DOC_DISCUSS_KIT_URL ||
-      appUrl
-    );
+    const useEnvAppUrl = !!(process.env.DOC_SMITH_PUBLISH_URL || process.env.DOC_DISCUSS_KIT_URL);
 
-    // Check if appUrl is default and not saved in config (only when not using env variable)
-    const config = await loadConfigFromFile();
+    // Use config from parameters or load from file as fallback
+    if (!config) {
+      config = await loadConfigFromFile();
+    }
+    const { projectName, projectDesc, projectLogo, boardId } = config || {};
     appUrl =
       process.env.DOC_SMITH_PUBLISH_URL ||
       process.env.DOC_DISCUSS_KIT_URL ||
@@ -81,7 +93,10 @@ export default async function publishDocs(
 
       sessionId = "";
       if (officialAccessToken) {
-        client = new BrokerClient({ baseUrl: BASE_URL, authToken: officialAccessToken });
+        client = new BrokerClient({
+          baseUrl: BASE_URL,
+          authToken: officialAccessToken,
+        });
         const info = await client.checkCacheSession({
           needShortUrl: true,
           sessionId: config?.checkoutId,
@@ -185,7 +200,9 @@ export default async function publishDocs(
           locale = data?.preferredLocale || locale;
         } catch (error) {
           const errorMsg = error?.message || "Unknown error occurred";
-          return { message: `${chalk.red("❌ Failed to create website:")} ${errorMsg}` };
+          return {
+            message: `${chalk.red("❌ Failed to create website:")} ${errorMsg}`,
+          };
         }
       }
     }
@@ -204,7 +221,7 @@ export default async function publishDocs(
     process.env.DOC_ROOT_DIR = docsDir;
 
     const sidebarPath = join(docsDir, "_sidebar.md");
-    const publishCacheFilePath = join(DOC_SMITH_DIR, "upload-cache.yaml");
+    const publishCacheFilePath = join(PATHS.CACHE, "upload-cache.yaml");
 
     // Get project info from config
     const projectInfo = {
@@ -212,46 +229,39 @@ export default async function publishDocs(
       description: projectDesc || config?.projectDesc || "",
       icon: projectLogo || config?.projectLogo || "",
     };
-    let finalPath = null;
 
     console.log(`Publishing docs collection: ${chalk.cyan(projectInfo.name || boardId)}\n`);
 
-    // Handle project logo download if it's a URL
-    if (projectInfo.icon && isRemoteFile(projectInfo.icon)) {
-      const { url: uploadedImageUrl, downloadFinalPath } = await downloadAndUploadImage(
-        projectInfo.icon,
-        docsDir,
-        discussKitUrl,
-        accessToken,
-      );
-      projectInfo.icon = uploadedImageUrl;
-      finalPath = downloadFinalPath;
-    }
-
+    // Skip image download - use icon URL directly
     if (shouldWithBranding) {
-      updateBranding({ appUrl: discussKitUrl, projectInfo, accessToken, finalPath });
-    }
-
-    const iconMap = {};
-    for (const item of originalDocumentStructure) {
-      if (item.icon) {
-        iconMap[item.title] = item.icon;
-      }
+      updateBranding({ appUrl: discussKitUrl, projectInfo, accessToken });
     }
 
     // Construct boardMeta object
+    // In standalone mode, get GitHub URL from git-clone type source in sources array
+    // In project mode, use current git repo URL (even if git-clone sources exist as supplements)
+    let githubRepoUrl = getGithubRepoUrl();
+    if (config?.mode === "standalone") {
+      const gitCloneSource = config?.sources?.find((s) => s.type === "git-clone");
+      const configUrl = gitCloneSource?.url;
+      // Only use config URL if it's a valid GitHub URL
+      githubRepoUrl = isValidGithubUrl(configUrl) ? configUrl : "";
+    }
     const boardMeta = {
       category: config?.documentPurpose || [],
-      githubRepoUrl: getGithubRepoUrl(),
+      githubRepoUrl,
       commitSha: config?.lastGitHead || "",
       languages: [
         ...(config?.locale ? [config.locale] : []),
         ...(config?.translateLanguages || []),
       ].filter((lang, index, arr) => arr.indexOf(lang) === index), // Remove duplicates
     };
+
+    // Add translatedMetadata if available
     if (translatedMetadata) {
       boardMeta.translation = translatedMetadata;
     }
+
     const {
       success,
       boardId: newBoardId,
@@ -267,10 +277,9 @@ export default async function publishDocs(
       boardName: projectInfo.name,
       boardDesc: projectInfo.description,
       boardCover: projectInfo.icon,
-      mediaFolder: rawDocsDir,
+      mediaFolder: docsRelativePath,
       cacheFilePath: publishCacheFilePath,
       boardMeta,
-      iconMap,
     });
 
     // Save values to config.yaml if publish was successful
@@ -303,14 +312,13 @@ export default async function publishDocs(
     }
 
     // clean up tmp work dir
-    await fs.rm(docsDir, { recursive: true, force: true });
+    await fs.rm(tmpDirRelative, { recursive: true, force: true });
   } catch (error) {
     message = `❌ Sorry, I encountered an error while publishing your documentation: \n\n${error.message}`;
 
     // clean up tmp work dir in case of error
     try {
-      const docsDir = join(DOC_SMITH_DIR, TMP_DIR, TMP_DOCS_DIR);
-      await fs.rm(docsDir, { recursive: true, force: true });
+      await fs.rm(tmpDirRelative, { recursive: true, force: true });
     } catch {
       // Ignore cleanup errors
     }
@@ -322,10 +330,18 @@ export default async function publishDocs(
 publishDocs.input_schema = {
   type: "object",
   properties: {
+    config: {
+      type: "object",
+      description: "Configuration object from check step.",
+    },
     docsDir: {
       type: "string",
       description: "The directory of the documentation.",
     },
+    outputDir: {
+      type: "string",
+      description: "Output directory containing document structure file (default: ./planning).",
+    },
     appUrl: {
       type: "string",
       description: "The URL of the app.",
@@ -350,6 +366,10 @@ publishDocs.input_schema = {
       type: "string",
       description: "The logo or icon of the project.",
     },
+    translatedMetadata: {
+      type: "object",
+      description: "Translated metadata (title and description) for multiple languages.",
+    },
   },
 };