@michengai/dsh-skills-manager 1.2.33 → 1.2.34

package/CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## [1.2.34] - 2026-11-06
+
+### Improvements
+
+- **Load large skill repositories without size-limit errors** (Issue #79):
+  - GitHub repositories are now fetched on demand: only directories containing `SKILL.md` are downloaded, so README screenshots and other unrelated assets no longer count against the size limit (the reported 235 MiB repository now transfers about 20 MiB while listing all 177 skills);
+  - Repository scanning and installation are no longer capped by the old 32 MiB archive limit; when no usable local Git is available, the plugin falls back to the previous full-archive download;
+  - The snapshot version badge now shows the Git commit of GitHub repositories.
+
+[中文文档](CHANGELOG.zh-CN.md)
+
 ## [1.2.33] - 2026-11-05
 
 ### Improvements

package/CHANGELOG.zh-CN.md

@@ -1,5 +1,16 @@
 # 更新日志
 
+## [1.2.34] - 2026-11-06
+
+### 体验优化
+
+- **大体积技能仓库不再报体积超限**（Issue #79）：
+  - GitHub 仓库改为按需检出：只下载包含 `SKILL.md` 的技能目录，仓库首页截图等无关文件不再计入体积限制（反馈中的 235 MiB 仓库现在只传输约 20 MiB，即可列出全部 177 个技能）；
+  - 仓库扫描与安装不再受旧的 32 MiB 归档上限约束；本机没有可用 Git 时自动回落为整库下载；
+  - 扫描版本标识改为显示 GitHub 仓库的 Git 提交。
+
+[English](CHANGELOG.md)
+
 ## [1.2.33] - 2026-11-05
 
 ### 交互体验优化

package/README.en.md

@@ -73,7 +73,7 @@ dsh plugin --profile web add @michengai/dsh-skills-manager@latest --registry=htt
 
 Restart DSH, refresh the page, and open **Settings → Skills**. To update, select **Check for updates** or run the installation command again.
 
-- Plugin `1.2.33` is tested with DeepSeek Harness `0.2.0-rc.1`, `0.2.0-rc.2`, `0.2.1-rc.1`, `0.2.1-rc.2`, `0.2.1-rc.3`, `0.2.1-rc.4`, `0.2.1-rc.5`, `0.2.2-rc.1`.
+- Plugin `1.2.34` is tested with DeepSeek Harness `0.2.0-rc.1`, `0.2.0-rc.2`, `0.2.1-rc.1`, `0.2.1-rc.2`, `0.2.1-rc.3`, `0.2.1-rc.4`, `0.2.1-rc.5`, `0.2.2-rc.1`.
 - Host compatibility: Full DeepSeek Harness `0.2.x` line (`>=0.2.0-rc.1 <0.3.0-0`), forwards-compatible with subsequent `0.2.x` RC releases without requiring routine plugin version bumps. Legacy versions and breaking changes in the `0.3.0` generation are rejected by the version gate. If you still use a `0.1.x` host, install plugin `1.1.23` instead of `@latest`.
 - **Reverse proxy deployment**: Supports hosting DSH behind prefix-stripping reverse proxies (e.g. `/app/dsh/`). API and update endpoints resolve against the page base URI (`document.baseURI`).
 
@@ -159,7 +159,7 @@ The DeepSeek Harness loader enforces strict identifier policies:
 
 - The header **Check for updates** updates the plugin itself; repository **Refresh** checks subscribed skill contents without replacing local files.
 - Repository installation can target user DSH skills, Shared Agent (`~/.agents/skills`, overridable via `DSH_AGENTS_HOME`), or `<project>/.dsh/skills` for registered workspaces. Unsafe paths and external agent roots (Codex, Claude) are locked against writes.
-- 32 MiB archive limit; downloading subdirectories still fetches the full archive.
+- GitHub repositories are fetched on demand: only directories containing `SKILL.md` are downloaded, so README images and other unrelated assets are skipped (a subdirectory narrows the scope further). Without a usable local Git the plugin falls back to the full archive, capped at 32 MiB.
 - Local edits are detected and require explicit confirmation before overwriting; backup history is preserved for rollback.
 
 ## Supported Agent directories

package/README.md

@@ -73,7 +73,7 @@ dsh plugin --profile web add @michengai/dsh-skills-manager@latest --registry=htt
 
 安装后重启 DSH 并刷新页面，打开「设置 → 技能」即可使用。更新时可点击「检查更新」，或重新执行安装命令。
 
-- `1.2.33` 已验证兼容 DeepSeek Harness `0.2.0-rc.1`、`0.2.0-rc.2`、`0.2.1-rc.1`、`0.2.1-rc.2`、`0.2.1-rc.3`、`0.2.1-rc.4`、`0.2.1-rc.5`、`0.2.2-rc.1`。
+- `1.2.34` 已验证兼容 DeepSeek Harness `0.2.0-rc.1`、`0.2.0-rc.2`、`0.2.1-rc.1`、`0.2.1-rc.2`、`0.2.1-rc.3`、`0.2.1-rc.4`、`0.2.1-rc.5`、`0.2.2-rc.1`。
 - 宿主支持范围：DeepSeek Harness `0.2.x` 全系列（`>=0.2.0-rc.1 <0.3.0-0`），自动支持后续全部 0.2.x RC 版本，同代宿主升级无需重复等待发版。早期版本与 `0.3.0` 代际破坏性更新由版本门拦截。仍在使用 `0.1.x` 的，请安装插件 `1.1.23`，不要装 `@latest`。
 - **反向代理部署**：支持将 DSH 部署在带路径前缀的反向代理后面（如 `/app/dsh/`），插件 API 统一自动按页面基址（`document.baseURI`）相对解析。
 
@@ -159,7 +159,7 @@ DeepSeek Harness 宿主加载机制对技能标识符遵循规范命名约束：
 
 - 顶栏「检查更新」用于更新技能管理器插件自身；仓库区「刷新」仅检查技能仓库内容更新，不自动覆盖本地。
 - 仓库安装可以写入用户目录下的 DSH skills 或公共 Agent（`~/.agents/skills`，可用 `DSH_AGENTS_HOME` 覆盖），也可以写入已登记工作区的 `<项目根>/.dsh/skills`。选择器显示实际配置路径，并禁用不安全路径；缺少的目录仅在确认安装后创建。不能安装到 Codex、Claude 等其他 Agent 目录。
-- 归档上限 32 MiB，指定子目录也需下载整库。
+- GitHub 仓库按需检出：只下载包含 `SKILL.md` 的技能目录，仓库首页配图等无关内容不参与下载（指定子目录可进一步缩小范围）；本机没有可用 Git 时回落整库归档，归档上限 32 MiB。
 - 更新前检测本地修改，覆盖必须明确确认；历史备份不自动清理，可随时恢复上一版。
 
 ## 支持的 Agent 目录

package/README.zh-CN.md

@@ -73,7 +73,7 @@ dsh plugin --profile web add @michengai/dsh-skills-manager@latest --registry=htt
 
 安装后重启 DSH 并刷新页面，打开「设置 → 技能」即可使用。更新时可点击「检查更新」，或重新执行安装命令。
 
-- `1.2.33` 已验证兼容 DeepSeek Harness `0.2.0-rc.1`、`0.2.0-rc.2`、`0.2.1-rc.1`、`0.2.1-rc.2`、`0.2.1-rc.3`、`0.2.1-rc.4`、`0.2.1-rc.5`、`0.2.2-rc.1`。
+- `1.2.34` 已验证兼容 DeepSeek Harness `0.2.0-rc.1`、`0.2.0-rc.2`、`0.2.1-rc.1`、`0.2.1-rc.2`、`0.2.1-rc.3`、`0.2.1-rc.4`、`0.2.1-rc.5`、`0.2.2-rc.1`。
 - 宿主支持范围：DeepSeek Harness `0.2.x` 全系列（`>=0.2.0-rc.1 <0.3.0-0`），自动支持后续全部 0.2.x RC 版本，同代宿主升级无需重复等待发版。早期版本与 `0.3.0` 代际破坏性更新由版本门拦截。仍在使用 `0.1.x` 的，请安装插件 `1.1.23`，不要装 `@latest`。
 - **反向代理部署**：支持将 DSH 部署在带路径前缀的反向代理后面（如 `/app/dsh/`），插件 API 统一自动按页面基址（`document.baseURI`）相对解析。
 
@@ -159,7 +159,7 @@ DeepSeek Harness 宿主加载机制对技能标识符遵循规范命名约束：
 
 - 顶栏「检查更新」用于更新技能管理器插件自身；仓库区「刷新」仅检查技能仓库内容更新，不自动覆盖本地。
 - 仓库安装可以写入用户目录下的 DSH skills 或公共 Agent（`~/.agents/skills`，可用 `DSH_AGENTS_HOME` 覆盖），也可以写入已登记工作区的 `<项目根>/.dsh/skills`。选择器显示实际配置路径，并禁用不安全路径；缺少的目录仅在确认安装后创建。不能安装到 Codex、Claude 等其他 Agent 目录。
-- 归档上限 32 MiB，指定子目录也需下载整库。
+- GitHub 仓库按需检出：只下载包含 `SKILL.md` 的技能目录，仓库首页配图等无关内容不参与下载（指定子目录可进一步缩小范围）；本机没有可用 Git 时回落整库归档，归档上限 32 MiB。
 - 更新前检测本地修改，覆盖必须明确确认；历史备份不自动清理，可随时恢复上一版。
 
 ## 支持的 Agent 目录

package/lib/repositories.js

@@ -4,7 +4,7 @@ import { createHash, randomUUID } from "node:crypto";
 import { unzipSync } from "fflate";
 import { managerHomePath, resolveWorkspaceTarget, repositoryDestinations, trashRootPath, listTrash, restoreTrash as restoreSkillTrash, permanentlyDeleteTrash, parseSkillDoc, importUploadedSkill, deleteSkill, setSkillEnabled, readManagerState, state, createWorkspaceIndex } from "./core.js";
 import { createRepositoryUpdater, fileIndex, signature, readSkillTree, repositoryInstallRoot, recordInstallRoot, selectInstallRecord } from "./repository-updates.js";
-import { parseGitUrl, fetchGitRepository, probeGitRemote, probeGitHubHttp } from "./repository-git.js";
+import { parseGitUrl, fetchGitRepository, sparseFetchGitRepository, probeGitRemote, probeGitHubHttp } from "./repository-git.js";
 const LIMIT = 32 << 20;
 const STATE_LIMIT = 16 << 20;
 const STATE_READ_LIMIT = 64 << 20;
@@ -128,6 +128,7 @@ async function readResponse(response, limit) {
 function createRepositoryManager({
   fetchImpl = globalThis.fetch,
   gitFetch,
+  sparseFetch = sparseFetchGitRepository,
   probeRemote,
   maxRepositoryBytes: rawMaxBytes = 2 * 1024 * 1024 * 1024,
   maxTransientBytes: rawMaxTransientBytes,
@@ -793,22 +794,35 @@ function createRepositoryManager({
           commit = res.commit;
           skills = await scanSkillsFromDirectory(stageDir, repo.subdirectory, repo.name);
         } else {
-          const bytes = await fetchBranch(repo);
-          const entries = decodeRepositoryArchive(bytes);
-          for (const [path, b] of Object.entries(entries)) {
-            if (path !== "SKILL.md" && !path.endsWith("/SKILL.md")) continue;
-            const directory = path === "SKILL.md" ? "" : path.slice(0, -9);
-            if (repo.subdirectory && directory !== repo.subdirectory && !directory.startsWith(`${repo.subdirectory}/`)) continue;
-            if (b.length > 256 << 10 || skills.length >= 500 || skills.reduce((n, s) => n + Buffer.byteLength(s.body, "utf8"), 0) + b.length > 4 << 20) throw failure("\u4ED3\u5E93\u6280\u80FD\u6570\u91CF\u6216\u8BF4\u660E\u957F\u5EA6\u8D85\u9650");
-            const parsed = parseSkillDoc(Buffer.from(b).toString("utf8"));
-            const name = String(parsed.map.name || "");
-            const description = String(parsed.map.description || "");
-            const prefix = directory ? directory + "/" : "";
-            const files = fileIndex(Object.fromEntries(Object.entries(entries).filter(([p]) => p.startsWith(prefix)).map(([p, b2]) => [p.slice(prefix.length), b2])));
-            if (files.length > 1e3) throw failure("\u6280\u80FD\u6587\u4EF6\u6570\u91CF\u8D85\u9650");
-            skills.push({ path: directory, files, name: name || directory.split("/").pop() || repo.name, description, body: parsed.body.trim(), documentHash: hash(b), valid: parsed.hasFrontmatter && /^[a-z0-9]+(?:-[a-z0-9]+)*$/.test(name) && !/^(con|prn|aux|nul|com[1-9]|lpt[1-9])$/i.test(name) && name.length <= 100 && !!description });
+          let useArchive = false;
+          try {
+            const res = await sparseFetch(repo, stageDir, { maxBytes: maxRepositoryBytes, maxTransientBytes, timeoutMs: gitTimeoutMs });
+            commit = res.commit;
+            skills = await scanSkillsFromDirectory(stageDir, repo.subdirectory, repo.name);
+          } catch (caught) {
+            const error = caught;
+            if (error.code !== "error.repo.noGit" && error.code !== "error.repo.network") throw error;
+            await fs.rm(stageDir, { recursive: true, force: true }).catch(() => void 0);
+            useArchive = true;
+          }
+          if (useArchive) {
+            const bytes = await fetchBranch(repo);
+            const entries = decodeRepositoryArchive(bytes);
+            for (const [path, b] of Object.entries(entries)) {
+              if (path !== "SKILL.md" && !path.endsWith("/SKILL.md")) continue;
+              const directory = path === "SKILL.md" ? "" : path.slice(0, -9);
+              if (repo.subdirectory && directory !== repo.subdirectory && !directory.startsWith(`${repo.subdirectory}/`)) continue;
+              if (b.length > 256 << 10 || skills.length >= 500 || skills.reduce((n, s) => n + Buffer.byteLength(s.body, "utf8"), 0) + b.length > 4 << 20) throw failure("\u4ED3\u5E93\u6280\u80FD\u6570\u91CF\u6216\u8BF4\u660E\u957F\u5EA6\u8D85\u9650");
+              const parsed = parseSkillDoc(Buffer.from(b).toString("utf8"));
+              const name = String(parsed.map.name || "");
+              const description = String(parsed.map.description || "");
+              const prefix = directory ? directory + "/" : "";
+              const files = fileIndex(Object.fromEntries(Object.entries(entries).filter(([p]) => p.startsWith(prefix)).map(([p, b2]) => [p.slice(prefix.length), b2])));
+              if (files.length > 1e3) throw failure("\u6280\u80FD\u6587\u4EF6\u6570\u91CF\u8D85\u9650");
+              skills.push({ path: directory, files, name: name || directory.split("/").pop() || repo.name, description, body: parsed.body.trim(), documentHash: hash(b), valid: parsed.hasFrontmatter && /^[a-z0-9]+(?:-[a-z0-9]+)*$/.test(name) && !/^(con|prn|aux|nul|com[1-9]|lpt[1-9])$/i.test(name) && name.length <= 100 && !!description });
+            }
+            commit = await cacheArchive(repo, bytes);
           }
-          commit = await cacheArchive(repo, bytes);
         }
       } catch (caught) {
         const error = caught;

package/lib/repository-git.js

@@ -195,8 +195,14 @@ function runGit(args, cwd, optionsOrTimeout, legacyGitBinary = "git") {
         "core.symlinks=false",
         "-c",
         "core.autocrlf=false",
+        // 技能目录层级深时，Windows 默认 260 字符上限会让检出直接失败。
+        "-c",
+        "core.longpaths=true",
         "-c",
         "advice.detachedHead=false",
+        // 只支持公开仓库：禁用凭据助手，避免 Git Credential Manager 弹出账号选择窗口并卡住插件进程。
+        "-c",
+        "credential.helper=",
         "-c",
         "http.lowSpeedLimit=1000",
         "-c",
@@ -208,16 +214,23 @@ function runGit(args, cwd, optionsOrTimeout, legacyGitBinary = "git") {
         env: {
           ...process.env,
           GIT_TERMINAL_PROMPT: "0",
+          GCM_INTERACTIVE: "never",
+          GIT_ASKPASS: "echo",
           LC_ALL: "C",
           LANGUAGE: "C"
         },
-        stdio: ["ignore", "pipe", "pipe"],
+        // 只在确实要写规则时才接管 stdin，其余调用保持 ignore。
+        stdio: options.input === void 0 ? ["ignore", "pipe", "pipe"] : ["pipe", "pipe", "pipe"],
         detached: process.platform !== "win32"
       }
     );
     let stdout = "";
     let stderr = "";
     let killed = false;
+    if (options.input !== void 0 && child.stdin) {
+      child.stdin.on("error", () => void 0);
+      child.stdin.end(options.input);
+    }
     child.stdout.on("data", (chunk) => {
       stdout += chunk;
     });
@@ -343,10 +356,117 @@ async function fetchGitRepository(repo, destinationDir, options = {}) {
     throw err;
   }
 }
+const SKILL_DOC_SUFFIX = "/SKILL.md";
+const EMPTY_CHECKOUT_PATTERN = "/.dsh-skills-manager-empty/**";
+function checkoutPatternFor(directory) {
+  return `/${directory.replace(/[[\]]/g, "\\$&")}/**`;
+}
+function skillCheckoutPatterns(paths, subdirectory = "") {
+  const collected = [];
+  for (const path of paths) {
+    if (typeof path !== "string" || !path) continue;
+    if (path === "SKILL.md") return null;
+    if (!path.endsWith(SKILL_DOC_SUFFIX)) continue;
+    const directory = path.slice(0, -SKILL_DOC_SUFFIX.length);
+    if (!directory) continue;
+    if (subdirectory && directory !== subdirectory && !directory.startsWith(`${subdirectory}/`)) continue;
+    safePath(directory);
+    collected.push(directory);
+  }
+  if (!collected.length) return [EMPTY_CHECKOUT_PATTERN];
+  const directories = [...new Set(collected)].sort();
+  return directories.filter((directory) => !directories.some((other) => other !== directory && directory.startsWith(`${other}/`))).map(checkoutPatternFor);
+}
+async function listTreePaths(destinationDir, revision, options) {
+  const { stdout } = await runGit(["ls-tree", "-r", "-z", "--name-only", "--", revision], destinationDir, options);
+  return stdout.split("\0").filter(Boolean);
+}
+async function clearSparseCheckout(destinationDir, options) {
+  for (const key of ["core.sparseCheckout", "core.sparseCheckoutCone"]) {
+    await runGit(["config", "--unset-all", key], destinationDir, options).catch(() => void 0);
+  }
+  await fs.rm(join(destinationDir, ".git", "info", "sparse-checkout"), { force: true }).catch(() => void 0);
+}
+async function sparseFetchGitRepository(repo, destinationDir, options = {}) {
+  const maxBytes = options.maxBytes ?? 2 * 1024 * 1024 * 1024;
+  const maxTransientBytes = options.maxTransientBytes ?? maxBytes * 2.5;
+  const timeoutMs = options.timeoutMs ?? 9e5;
+  const deadline = Date.now() + timeoutMs;
+  const gitBinary = options.gitBinary || "git";
+  const remainingTimeout = () => {
+    const remain = deadline - Date.now();
+    if (remain <= 0) throw failure("Git \u8BF7\u6C42\u8D85\u65F6\uFF0C\u8BF7\u68C0\u67E5\u7F51\u7EDC\u540E\u91CD\u8BD5", "error.repo.network");
+    return { timeoutMs: remain, gitBinary };
+  };
+  const repoUrl = repo.url || `https://github.com/${repo.owner}/${repo.name}.git`;
+  const ref = repo.ref && repo.ref !== "HEAD" ? repo.ref : "HEAD";
+  await fs.mkdir(destinationDir, { recursive: true });
+  try {
+    await runGit(["init", "-q"], destinationDir, remainingTimeout());
+    await runGit(["remote", "add", "origin", repoUrl], destinationDir, remainingTimeout());
+    const refsToTry = ref === "HEAD" ? ["HEAD"] : [`refs/heads/${ref}`, `refs/tags/${ref}`, ref];
+    let fetched = false;
+    let lastError;
+    for (const r of refsToTry) {
+      try {
+        await runGit(["fetch", "--progress", "--depth", "1", "--no-tags", "--filter=blob:none", "--", "origin", r], destinationDir, remainingTimeout());
+        fetched = true;
+        break;
+      } catch (err) {
+        lastError = err;
+        try {
+          await runGit(["fetch", "--progress", "--depth", "1", "--no-tags", "--", "origin", r], destinationDir, remainingTimeout());
+          fetched = true;
+          break;
+        } catch (err2) {
+          lastError = err2;
+          if (r === refsToTry[refsToTry.length - 1]) {
+            try {
+              await runGit(["fetch", "--progress", "--no-tags", "--", "origin", r], destinationDir, remainingTimeout());
+              fetched = true;
+            } catch {
+              throw err2;
+            }
+          }
+        }
+      }
+    }
+    if (!fetched) throw lastError || failure("Git \u5206\u652F\u6216\u6807\u7B7E\u4E0D\u5B58\u5728", "error.repo.network");
+    const { stdout: commitOut } = await runGit(["rev-parse", "FETCH_HEAD^{commit}"], destinationDir, remainingTimeout());
+    const commit = commitOut.trim();
+    const patterns = skillCheckoutPatterns(await listTreePaths(destinationDir, "FETCH_HEAD", remainingTimeout()), repo.subdirectory);
+    if (patterns) {
+      try {
+        await runGit(["sparse-checkout", "set", "--no-cone", "--stdin"], destinationDir, { ...remainingTimeout(), input: `${patterns.join("\n")}
+` });
+      } catch (err) {
+        const code = err?.code;
+        if (code === "error.repo.noGit" || code === "error.repo.network") throw err;
+        await clearSparseCheckout(destinationDir, remainingTimeout());
+      }
+    }
+    await runGit(["checkout", "-q", "FETCH_HEAD"], destinationDir, remainingTimeout());
+    const rawSize = await getDirectorySize(destinationDir);
+    if (rawSize > maxTransientBytes) {
+      throw failure("\u4ED3\u5E93\u4E34\u65F6\u4E0B\u8F7D\u4F53\u79EF\u8FC7\u5927\uFF08\u5305\u542B Git \u5386\u53F2\u5BF9\u8C61\uFF09", "error.repo.transientTooLarge", { limit: maxTransientBytes, actual: rawSize });
+    }
+    await rmWithRetry(join(destinationDir, ".git"));
+    const totalSize = await getDirectorySize(destinationDir);
+    if (totalSize > maxBytes) {
+      throw failure("\u4ED3\u5E93\u8D85\u8FC7\u5927\u5C0F\u9650\u5236", "error.repo.tooLarge", { limit: maxBytes, actual: totalSize });
+    }
+    return { commit };
+  } catch (err) {
+    await rmWithRetry(destinationDir).catch(() => void 0);
+    throw err;
+  }
+}
 export {
   fetchGitRepository,
   getDirectorySize,
   parseGitUrl,
   probeGitHubHttp,
-  probeGitRemote
+  probeGitRemote,
+  skillCheckoutPatterns,
+  sparseFetchGitRepository
 };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@michengai/dsh-skills-manager",
-  "version": "1.2.33",
+  "version": "1.2.34",
   "description": "DSH Skills Manager — 在 DeepSeek Harness 中统一加载并安全管理本机 Agent Skills · Load and safely manage local Agent Skills in DSH",
   "license": "Apache-2.0",
   "repository": {