agentsmesh 0.19.1 → 0.21.0

package/CHANGELOG.md

@@ -1,5 +1,92 @@
 # Changelog
 
+## 0.21.0
+
+### Minor Changes
+
+- b1efca1: Harden install pipeline against third-party supply-chain attacks.
+  - **Strip elevated artifacts from non-local sources by default.** `hooks.yaml`, `permissions.yaml`, and `mcp.json` are now removed from any pack installed from a `github:`, `gitlab:`, or `git+...` source unless you opt in. These three files control your agent's tool settings (shell hooks, granted permissions, MCP launch specs) and a malicious pack shipping any of them could otherwise execute arbitrary local commands the next time the matching event fires. Opt in per-artifact with `--accept-hooks`, `--accept-permissions`, `--accept-mcp`, or all three with `--accept-elevated`. Local sources remain trusted as before.
+  - **Skill supporting-file traversal no longer follows symlinks.** A pack containing `skills/foo/keys -> /Users/victim/.ssh` previously pulled external bytes (private keys, etc.) into the canonical skill content. Skill traversal now uses the existing `readDirRecursiveNoSymlinks` helper, mirroring the hardening already applied to install-manifest hashing.
+  - **Redact credentials from remote-fetch error output.** `oauth2:<token>@`, `x-access-token:<token>@`, and any other userinfo-bearing URLs are now masked (`https://***@host/...`) in console warnings and thrown error messages so GitHub PATs and GitLab tokens never leak into CI logs, terminal scrollback, or log shippers.
+  - **Gate `git+file://` sources behind `AGENTSMESH_ALLOW_LOCAL_GIT=1`.** On shared/multi-tenant hosts a `git+file:///tmp/world-writable-repo` `extends:` clause could silently consume a repo planted by another user; combined with downstream elevated-artifact emission this was a local priv-esc vector. Set `AGENTSMESH_ALLOW_LOCAL_GIT=1` to enable for closed-network development.
+  - **Allowlist tar entry types on GitHub tarball extraction.** Previously only `Link` and `SymbolicLink` were rejected (denylist). Now only `File` and `Directory` entries extract; FIFOs, devices, hardlinks, and any future/exotic tar variant are rejected by default.
+
+  These changes apply to `agentsmesh install`, `agentsmesh refresh`, and any `extends:` resolution against a non-local source. They are behavior changes for users who were silently inheriting hooks/permissions/mcp from a remote pack — re-run with the matching `--accept-*` flag (or `--accept-elevated`) to preserve previous behavior intentionally.
+
+## 0.20.0
+
+### Minor Changes
+
+- 6739c63: feat(refresh): add `agentsmesh refresh` to re-fetch and re-apply installed packs
+
+  A new top-level CLI command and MCP tool for keeping installed packs in
+  sync with their declared sources. Branch pins re-resolve to the current
+  tip; tag pins re-resolve in case the tag moved; SHA pins stay put. Per-pack
+  atomic via the existing `materializePack` swap — a failure or interruption
+  leaves the affected pack at its pre-refresh state.
+
+  ```
+  agentsmesh refresh                          # refresh every installed pack
+  agentsmesh refresh my-pack,other-pack       # refresh just these
+  agentsmesh refresh --dry-run                # preview without writing
+  agentsmesh refresh --force                  # skip the drift consent prompt
+  agentsmesh refresh --json                   # JSON output (implies --force)
+  agentsmesh refresh --global                 # global scope
+  ```
+
+  **MCP parity:** new `mcp__agentsmesh__refresh` tool with the same
+  `{ names?, dry_run?, global? }` input shape MCP install/uninstall use.
+  `force: true` is implicit (no TTY). Errors map to two new codes —
+  `REFRESH_RESOLVE_FAILED` and `REFRESH_APPLY_FAILED` — plus the existing
+  `LOCK_HELD` and `IO_ERROR`.
+
+  **Drift handling:** modified pack files trigger a consolidated consent
+  prompt with a 5-minute timeout (default no). `--force` bypasses the prompt
+  and overwrites local edits. The prompt is collapsed across packs so a bulk
+  refresh asks once, not N times.
+
+  **Schema additions** (`installs.yaml`, all optional, backwards-compatible):
+  - `original_ref?: string` — the user's original ref expression (e.g.
+    `main`, `v1.2.3`) captured at install time. Used by refresh to
+    re-resolve branch pins. Absent on installs predating this release;
+    refresh becomes a deterministic no-op for those rows.
+  - `refreshed_at?: string` — ISO-8601 timestamp of the last successful
+    refresh. Surfaces in `installs list` under the "LAST TOUCHED" column
+    (falls back to `installed_at` when absent).
+
+  **Behavior changes that could affect existing consumers:**
+  - `installs.yaml` rows written by this release include `original_ref`.
+    Pre-existing rows continue to parse and behave identically.
+  - `--json` on `agentsmesh refresh` implies `--force` (CI/MCP can't
+    answer the consent prompt). Documented on the website CLI reference.
+  - `installs list` column header was "INSTALLED AT", now "LAST TOUCHED",
+    showing `refreshed_at` when present and `installed_at` otherwise.
+
+  **Architecture notes:**
+  - `installAsPack` gains an optional `forceFreshMaterialize` flag,
+    threaded through five layers (`install-flags → run-install →
+run-install-locked → single-pack-install → run-install-execute →
+installAsPack`). Default is false; install's existing flow is
+    untouched. Refresh sets the flag to bypass the
+    merge-into-existing-pack branch and force atomic replacement via
+    `materializePack`.
+  - Source-URL parsing is now shared between install and refresh via the
+    new pure `parseSourceUrl` helper (`src/install/source/parse-source-url.ts`).
+
+  **Refresh does NOT switch refs.** To move a pack to a different ref,
+  re-run `agentsmesh install <source>@<new-ref>` — install silently
+  overwrites an existing pack of the same name.
+
+  **Refresh vs `install --sync`** are orthogonal. `--sync` replays missing
+  installs from `installs.yaml` (fresh clone). `refresh` updates existing
+  installs against their declared sources.
+
+  Verified end-to-end against 64 community packs from the install
+  compatibility log spanning Anthropic skill-packs, marketplaces (`--all`),
+  canonical mixed packs, flat collections, root SKILL.md, root CLAUDE.md,
+  manual `--path`/`--as`/`--target` combinations, and packs using `pick`
+  selectors. Full unit/integration/e2e suite green (7700+ tests).
+
 ## 0.19.1
 
 ### Patch Changes

package/README.md

@@ -233,8 +233,10 @@ agentsmesh check [--global]
 agentsmesh merge [--global]
 agentsmesh matrix [--global] [--targets <csv>] [--verbose]
 agentsmesh install <source> [--sync] [--path <dir>] [--target <id>] [--as <kind>] [--name <id>] [--extends] [--all] [--dry-run] [--global] [--force]
+                            [--accept-hooks|--accept-permissions|--accept-mcp|--accept-elevated]
 agentsmesh uninstall <name>[,<name>...] [--all] [--keep-pack] [--keep-generated] [--dry-run] [--global] [--force]
 agentsmesh installs list [--global]
+agentsmesh refresh [<name>[,<name>...]] [--dry-run] [--force] [--json] [--global]
 agentsmesh plugin add|list|remove|info [--version <v>] [--id <id>]
 agentsmesh target scaffold <id> [--name <displayName>] [--force]
 ```
@@ -314,6 +316,34 @@ agentsmesh uninstall <name> --dry-run          # preview; no writes
 
 Each install writes `.agentsmesh-install-manifest.json` next to the pack with per-file sha256 hashes; uninstall compares current contents against that manifest and prompts before deleting locally-modified files. `--force` accepts the documented defaults (bulk = accept all, broken-link = leave-with-warnings, modified = delete-anyway). `.agentsmesh/.install.lock` serialises install/uninstall so concurrent runs on the same project fail fast rather than racing on disk.
 
+### Refreshing packs
+
+`agentsmesh refresh` re-fetches every installed pack from its recorded source/ref
+and re-applies it. Branch pins (`@main`) advance to the current tip; tag pins
+re-resolve in case the tag moved; SHA pins stay put (re-fetch with the same content).
+
+```bash
+agentsmesh refresh                    # refresh every installed pack
+agentsmesh refresh my-pack,other-pack # refresh just these
+agentsmesh refresh --dry-run          # preview without writing
+agentsmesh refresh --force            # skip the drift prompt
+```
+
+Each pack is refreshed atomically — a failure or interruption leaves the
+affected pack at its pre-refresh state. Local edits to pack files trigger
+a consolidated consent prompt (5-minute timeout) unless `--force` is set.
+
+**refresh does NOT switch refs.** To move a pack to a different ref, just install
+with the new ref — install silently overwrites an existing pack of the same name:
+
+```bash
+agentsmesh install github:org/repo@v2.0.0
+```
+
+**refresh vs `install --sync`.** `--sync` replays missing installs from
+`installs.yaml` (e.g. after a fresh clone). `refresh` updates existing
+installs against their declared sources. They are orthogonal.
+
 ### What to commit and what to gitignore
 
 `agentsmesh init` writes a `.gitignore` that follows the recommended convention. The defaults are deliberate:
@@ -457,7 +487,7 @@ See the [full feature matrix docs](https://samplexbro.github.io/agentsmesh/refer
 
 - **[Getting Started](https://samplexbro.github.io/agentsmesh/getting-started/installation/)** — installation, quick start
 - **[Canonical Config](https://samplexbro.github.io/agentsmesh/canonical-config/)** — rules, commands, agents, skills, MCP, hooks, ignore, permissions
-- **[CLI Reference](https://samplexbro.github.io/agentsmesh/cli/)** — `init`, `generate`, `import`, `convert`, `install`, `uninstall`, `installs`, `diff`, `lint`, `watch`, `check`, `merge`, `matrix`, `plugin`, `target`
+- **[CLI Reference](https://samplexbro.github.io/agentsmesh/cli/)** — `init`, `generate`, `import`, `convert`, `install`, `uninstall`, `installs`, `refresh`, `diff`, `lint`, `watch`, `check`, `merge`, `matrix`, `plugin`, `target`
 - **[Configuration](https://samplexbro.github.io/agentsmesh/configuration/agentsmesh-yaml/)** — `agentsmesh.yaml`, local overrides, extends, collaboration, conversions
 - **[Guides](https://samplexbro.github.io/agentsmesh/guides/existing-project/)** — adopting in existing projects · multi-tool teams · sharing config · CI drift detection · community packs · **building plugins**
 - **[Reference](https://samplexbro.github.io/agentsmesh/reference/generation-pipeline/)** — supported tools matrix · generation pipeline · managed embedding

package/dist/canonical.js

@@ -185,6 +185,33 @@ async function readDirRecursive(dir, visited, branchSegments) {
     );
   }
 }
+async function readDirRecursiveNoSymlinks(dir, branchSegments) {
+  const currentBranchSegments = branchSegments ?? [basename(dir)];
+  try {
+    const entries = await readdir(dir, { withFileTypes: true });
+    const files = [];
+    for (const ent of entries) {
+      if (ent.isSymbolicLink()) continue;
+      const full = join(dir, ent.name);
+      if (ent.isDirectory()) {
+        const nextSegments = [...currentBranchSegments, ent.name];
+        if (shouldSkipRecursiveBranch(nextSegments)) continue;
+        files.push(...await readDirRecursiveNoSymlinks(full, nextSegments));
+      } else if (ent.isFile()) {
+        files.push(full);
+      }
+    }
+    return files;
+  } catch (err) {
+    const e = err;
+    if (e.code === "ENOENT" || e.code === "ENOTDIR" || e.code === "EACCES") return [];
+    throw new FileSystemError(
+      dir,
+      `Failed to read directory ${dir}: ${e.message}. Check permissions.`,
+      { cause: err, errnoCode: e.code }
+    );
+  }
+}
 var MAX_RECURSIVE_DEPTH, MAX_SEGMENT_REPETITIONS;
 var init_fs_traverse = __esm({
   "src/utils/filesystem/fs-traverse.ts"() {
@@ -1023,7 +1050,7 @@ ${legacy}`, "");
   }
   return result.trim();
 }
-var ROOT_INSTRUCTION_BODY_V1, ROOT_INSTRUCTION_BODY_V2, ROOT_INSTRUCTION_BODY_V3, ROOT_INSTRUCTION_BODY_V4, ROOT_INSTRUCTION_BODY_V5, ROOT_INSTRUCTION_BODY_V6, ROOT_INSTRUCTION_BODY_V7, ROOT_INSTRUCTION_BODY, LEGACY_AGENTSMESH_ROOT_INSTRUCTION_PARAGRAPH, LEGACY_AGENTSMESH_ROOT_INSTRUCTION_SECTION, AGENTSMESH_CONTRACT_WITH_V1_BODY, AGENTSMESH_CONTRACT_WITH_V2_BODY, AGENTSMESH_CONTRACT_WITH_V3_BODY, AGENTSMESH_CONTRACT_WITH_V4_BODY, AGENTSMESH_CONTRACT_WITH_V5_BODY, AGENTSMESH_CONTRACT_WITH_V6_BODY, AGENTSMESH_CONTRACT_WITH_V7_BODY, AGENTSMESH_ROOT_INSTRUCTION_PARAGRAPH, LEGACY_FORMS;
+var ROOT_INSTRUCTION_BODY_V1, ROOT_INSTRUCTION_BODY_V2, ROOT_INSTRUCTION_BODY_V3, ROOT_INSTRUCTION_BODY_V4, ROOT_INSTRUCTION_BODY_V5, ROOT_INSTRUCTION_BODY_V6, ROOT_INSTRUCTION_BODY_V7, ROOT_INSTRUCTION_BODY_V8, ROOT_INSTRUCTION_BODY, LEGACY_AGENTSMESH_ROOT_INSTRUCTION_PARAGRAPH, LEGACY_AGENTSMESH_ROOT_INSTRUCTION_SECTION, AGENTSMESH_CONTRACT_WITH_V1_BODY, AGENTSMESH_CONTRACT_WITH_V2_BODY, AGENTSMESH_CONTRACT_WITH_V3_BODY, AGENTSMESH_CONTRACT_WITH_V4_BODY, AGENTSMESH_CONTRACT_WITH_V5_BODY, AGENTSMESH_CONTRACT_WITH_V6_BODY, AGENTSMESH_CONTRACT_WITH_V7_BODY, AGENTSMESH_CONTRACT_WITH_V8_BODY, AGENTSMESH_ROOT_INSTRUCTION_PARAGRAPH, LEGACY_FORMS;
 var init_root_instruction_paragraph = __esm({
   "src/targets/projection/root-instruction-paragraph.ts"() {
     init_managed_blocks();
@@ -1034,7 +1061,8 @@ var init_root_instruction_paragraph = __esm({
     ROOT_INSTRUCTION_BODY_V5 = "Use Claude-style Markdown in `.agentsmesh`: `agents/*.md`, `commands/*.md`, and `skills/*/SKILL.md`; keep rules in `rules/*.md`, hooks in `hooks.yaml`, MCP in `mcp.json`, permissions in `permissions.yaml`, and ignore patterns in `ignore`, then run `agentsmesh generate`.";
     ROOT_INSTRUCTION_BODY_V6 = "Create canonical files in `.agentsmesh`: `rules/_root.md` and `rules/*.md` are Markdown rules; `commands/*.md`, `agents/*.md`, and `skills/*/SKILL.md` plus supporting files use Claude-style frontmatter Markdown; `mcp.json` is MCP JSON; `hooks.yaml` and `permissions.yaml` are YAML; `ignore` is gitignore-style text. Then run `agentsmesh generate`.";
     ROOT_INSTRUCTION_BODY_V7 = "`.agentsmesh` is the only folder you edit or add these files in: `rules/_root.md` and `rules/*.md` are Markdown rules; `commands/*.md`, `agents/*.md`, and `skills/*/SKILL.md` plus supporting files use Claude-style frontmatter Markdown; `mcp.json` is MCP JSON; `hooks.yaml` and `permissions.yaml` are YAML; `ignore` is gitignore-style text. Do not edit generated tool files; run `agentsmesh generate`.";
-    ROOT_INSTRUCTION_BODY = "`agentsmesh.yaml` selects targets/features (`agentsmesh.local.yaml` overrides locally), and `.agentsmesh` is the only place to add or edit canonical items: `rules/_root.md`, `rules/*.md`, `commands/*.md`, `agents/*.md`, `skills/*/SKILL.md` plus supporting files, `mcp.json`, `hooks.yaml`, `permissions.yaml`, and `ignore`; if missing run `agentsmesh init`, use `agentsmesh import --from <tool>` for native configs, `agentsmesh install <source>` or `install --sync` for reusable packs, then run `agentsmesh generate`. Use `diff`, `lint`, `check`, `watch`, `matrix`, and `merge` as needed; never edit generated tool files.";
+    ROOT_INSTRUCTION_BODY_V8 = "`agentsmesh.yaml` selects targets/features (`agentsmesh.local.yaml` overrides locally), and `.agentsmesh` is the only place to add or edit canonical items: `rules/_root.md`, `rules/*.md`, `commands/*.md`, `agents/*.md`, `skills/*/SKILL.md` plus supporting files, `mcp.json`, `hooks.yaml`, `permissions.yaml`, and `ignore`; if missing run `agentsmesh init`, use `agentsmesh import --from <tool>` for native configs, `agentsmesh install <source>` or `install --sync` for reusable packs, then run `agentsmesh generate`. Use `diff`, `lint`, `check`, `watch`, `matrix`, and `merge` as needed; never edit generated tool files.";
+    ROOT_INSTRUCTION_BODY = "`agentsmesh.yaml` selects targets/features (`agentsmesh.local.yaml` overrides locally), and `.agentsmesh` is the only place to add or edit canonical items: `rules/_root.md`, `rules/*.md`, `commands/*.md`, `agents/*.md`, `skills/*/SKILL.md` plus supporting files, `mcp.json`, `hooks.yaml`, `permissions.yaml`, and `ignore`; if missing run `agentsmesh init`, use `agentsmesh import --from <tool>` for native configs, `agentsmesh install <source>` or `install --sync` for reusable packs, then run `agentsmesh generate`. Use `diff`, `lint`, `check`, `watch`, `matrix`, `merge`, and `refresh` as needed; never edit generated tool files.";
     LEGACY_AGENTSMESH_ROOT_INSTRUCTION_PARAGRAPH = ROOT_INSTRUCTION_BODY_V1;
     LEGACY_AGENTSMESH_ROOT_INSTRUCTION_SECTION = `## Project-Specific Rules
 
@@ -1060,12 +1088,16 @@ ${ROOT_INSTRUCTION_BODY_V6}`;
     AGENTSMESH_CONTRACT_WITH_V7_BODY = `## AgentsMesh Generation Contract
 
 ${ROOT_INSTRUCTION_BODY_V7}`;
+    AGENTSMESH_CONTRACT_WITH_V8_BODY = `## AgentsMesh Generation Contract
+
+${ROOT_INSTRUCTION_BODY_V8}`;
     AGENTSMESH_ROOT_INSTRUCTION_PARAGRAPH = `${ROOT_CONTRACT_START}
 ## AgentsMesh Generation Contract
 
 ${ROOT_INSTRUCTION_BODY}
 ${ROOT_CONTRACT_END}`;
     LEGACY_FORMS = [
+      AGENTSMESH_CONTRACT_WITH_V8_BODY,
       AGENTSMESH_CONTRACT_WITH_V7_BODY,
       AGENTSMESH_CONTRACT_WITH_V6_BODY,
       AGENTSMESH_CONTRACT_WITH_V5_BODY,
@@ -18726,6 +18758,19 @@ init_fs();
 
 // src/config/remote/git-remote.ts
 init_fs();
+
+// src/utils/output/redact-url-secrets.ts
+var URL_WITH_CREDENTIALS = /([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)([^/@\s"'<>]+)@([^\s"'<>]+)/g;
+function redactUrlSecrets(message) {
+  return message.replace(
+    URL_WITH_CREDENTIALS,
+    (_full, scheme, _userinfo, rest) => {
+      return `${scheme}***@${rest}`;
+    }
+  );
+}
+
+// src/config/remote/git-remote.ts
 var execFileAsync = promisify(execFile);
 var REPO_DIRNAME = "repo";
 function ensureNotFlag(value, kind) {
@@ -18759,12 +18804,13 @@ async function fetchGitRemoteExtend(parsed, extendName, options, cacheDir, build
     await rm(stagedRoot, { recursive: true, force: true });
     const allowFallback = options.allowOfflineFallback !== false;
     if (allowFallback && await hasCachedRepo(cacheRepoDir)) {
+      const rawMsg = err instanceof Error ? err.message : String(err);
       console.warn(
-        `[agentsmesh] Remote fetch failed for ${extendName}; using cached version. Error: ${err instanceof Error ? err.message : String(err)}`
+        `[agentsmesh] Remote fetch failed for ${extendName}; using cached version. Error: ${redactUrlSecrets(rawMsg)}`
       );
       return readCachedRepo(cacheRepoDir);
     }
-    throw err;
+    throw err instanceof Error ? Object.assign(new Error(redactUrlSecrets(err.message)), { cause: err.cause }) : err;
   }
 }
 async function readCachedRepo(repoDir) {
@@ -18925,13 +18971,14 @@ async function fetchGithubRemoteExtend(parsed, extendName, options, cacheDir, bu
     if (allowFallback && await exists(extractDir)) {
       const topDir2 = await findExtractTopDir(extractDir);
       if (topDir2) {
+        const rawMsg = err instanceof Error ? err.message : String(err);
         console.warn(
-          `[agentsmesh] Network failed for ${extendName}; using cached version. Error: ${err instanceof Error ? err.message : String(err)}`
+          `[agentsmesh] Network failed for ${extendName}; using cached version. Error: ${redactUrlSecrets(rawMsg)}`
         );
         return { resolvedPath: join(extractDir, topDir2), version: tag };
       }
     }
-    throw err;
+    throw err instanceof Error ? Object.assign(new Error(redactUrlSecrets(err.message)), { cause: err.cause }) : err;
   }
   await rm(extractDir, { recursive: true, force: true });
   await mkdir(extractDir, { recursive: true });
@@ -18942,12 +18989,14 @@ async function fetchGithubRemoteExtend(parsed, extendName, options, cacheDir, bu
       file: tarPath,
       cwd: extractDir,
       strict: true,
+      // Allowlist entry types instead of denylist: only `File` and `Directory`
+      // can be extracted. Hardlinks (`Link`), symlinks (`SymbolicLink`), FIFOs,
+      // character/block devices, and any future/exotic tar entry type are
+      // rejected. A denylist would silently let an unknown variant through.
       filter: (entryPath, entry) => {
         if (isZipSlipPath(entryPath)) return false;
-        if (entry && "type" in entry && (entry.type === "Link" || entry.type === "SymbolicLink")) {
-          return false;
-        }
-        return true;
+        const type = entry && "type" in entry ? entry.type : void 0;
+        return type === "File" || type === "Directory";
       }
     });
   } finally {
@@ -19045,8 +19094,11 @@ function parseGitSource(source) {
     return null;
   }
   const allowInsecure = process.env.AGENTSMESH_ALLOW_INSECURE_GIT === "1" || process.env.AGENTSMESH_ALLOW_INSECURE_GIT === "true";
-  const allowedProtocols = allowInsecure ? ["https:", "http:", "ssh:", "file:"] : ["https:", "ssh:", "file:"];
-  if (!allowedProtocols.includes(parsedUrl.protocol)) {
+  const allowLocalGit = process.env.AGENTSMESH_ALLOW_LOCAL_GIT === "1" || process.env.AGENTSMESH_ALLOW_LOCAL_GIT === "true";
+  const allowed = ["https:", "ssh:"];
+  if (allowInsecure) allowed.push("http:");
+  if (allowLocalGit) allowed.push("file:");
+  if (!allowed.includes(parsedUrl.protocol)) {
     return null;
   }
   return { url, ref };
@@ -19571,6 +19623,7 @@ async function parseAgents(agentsDir, opts = {}) {
 
 // src/canonical/features/skills.ts
 init_fs();
+init_fs_traverse();
 init_markdown();
 init_boilerplate_filter();
 async function readContent(path) {
@@ -19589,7 +19642,7 @@ function sanitizeSkillName(raw) {
   return raw.toLowerCase().replace(/[^a-z0-9-]+/g, "-").replace(/-+/g, "-").replace(/^-|-$/g, "");
 }
 async function listSupportingFiles(skillDir) {
-  const files = await readDirRecursive(skillDir);
+  const files = await readDirRecursiveNoSymlinks(skillDir);
   const result = [];
   for (const absPath of files) {
     const raw = absPath.slice(skillDir.length + 1);