skillcellar 0.6.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/package.json +1 -1
- package/src/backup.mjs +97 -45
- package/src/cli.mjs +15 -12
- package/src/plan.mjs +3 -0
- package/src/restore.mjs +11 -4
- package/src/secrets.mjs +12 -2
package/README.md
CHANGED
|
@@ -29,14 +29,14 @@ Both read this computer as `sync` does and send that list as `sync` sends it (na
|
|
|
29
29
|
|
|
30
30
|
## Backups
|
|
31
31
|
|
|
32
|
-
`skillcellar backup --repo <owner>/<repo>` pushes this computer's Claude Code setup to a private GitHub repository you own, with your own git and its credentials, under `devices/<device>/`: `global/` holds your skills' source files, `CLAUDE.md`, `rules/`, `agents/` and `mcp.json`; `projects/<project>/` holds the same for each project, with its `AGENTS.md`, the session memory Claude Code keeps for it under `memory/`, and its MCP servers in `mcp.json` (the project's `.mcp.json`) and `mcp.local.json` (those Claude Code keeps for its folders). A `manifest.json` names the device and its projects. The repository is remembered, so `skillcellar backup` alone works from then on; `--ssh` clones and pushes over SSH, and `--dry-run` lists what would go without cloning, committing or pushing.
|
|
32
|
+
`skillcellar backup --repo <owner>/<repo>` pushes this computer's Claude Code setup to a private GitHub repository you own, with your own git and its credentials, under `devices/<device>/`: `global/` holds your skills' source files, `CLAUDE.md`, `rules/`, `agents/` and `mcp.json`; `projects/<project>/` holds the same for each project, with its `AGENTS.md`, the session memory Claude Code keeps for it under `memory/`, and its MCP servers in `mcp.json` (the project's `.mcp.json`) and `mcp.local.json` (those Claude Code keeps for its folders). A project checked out more than once on the computer is one project, as on the site: every checkout goes in its one folder, and where two hold a file or a skill of the same name, the one the sync sends first goes and the other is listed as left out. MCP servers of the same name are kept once the same way, but separately in `mcp.json` and in `mcp.local.json`, and the one left out is not listed. A `manifest.json` names the device and its projects. The repository is remembered, so `skillcellar backup` alone works from then on; `--ssh` clones and pushes over SSH, and `--dry-run` lists what would go without cloning, committing or pushing.
|
|
33
33
|
|
|
34
34
|
- In the MCP config every `env` and header value, an address's credentials and its query's values, the parts of its path that are not short words, and an argument that looks like a key are replaced by a placeholder, named after its key where one exists (`${CONTEXT7_KEY}`), else `${ARG_n}` or `${PATH_n}`. A key of a shape the check does not know, in another field, or given as a bare argument shorter than 16 characters, stays: read the backup once before you rely on it.
|
|
35
35
|
- Source files are text of a known kind (Markdown, code, JSON, YAML and the like) under 1 MB. Folders of installed packages, caches and build output (`node_modules`, `vendor`, `.venv`, `venv`, `env`, `__pycache__`, `.git`, `.cache`, `.next`, `.turbo`, `.nuxt`, `.svelte-kit`, `dist`, `build`, `target`, `coverage`, `.tox`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `.gradle` and `ms-playwright`) are left out at any depth, as are links inside a skill folder, and the command lists the folders and files it left out for those reasons, a file over 1 MB and a folder it could not read among them. Files a project's own git tracks are left out without a line: a clone brings them.
|
|
36
|
-
- Before anything is pushed, every file is read for what looks like a key, a token, a password or a private key: known shapes (GitHub, OpenAI, AWS, Stripe, Slack, Google, npm, Neon, Resend, Hugging Face, Groq, JSON Web Tokens, private key blocks), a password in an address, a bearer or basic credential, and a name such as `API_KEY`, `password` or `token` assigned a value with digits in it. Anything found stops the backup with each file and line (the first 20), and nothing is pushed. A file you have read and know holds no secret goes through with `--allow <path>` for that file. Then the repository is checked without a token: one that answers anyone is public, and is refused. The clone lives beside the config, in `skillcellar/backups/<owner>/<repo>`, readable by you only; each backup brings the clone to the remote's state, commits and pushes, and a commit a failed push left behind goes with the next run.
|
|
36
|
+
- Before anything is pushed, every file is read for what looks like a key, a token, a password or a private key: known shapes (GitHub, OpenAI, AWS, Stripe, Slack, Google, npm, Neon, Resend, Hugging Face, Groq, JSON Web Tokens, private key blocks), a password in an address, a bearer or basic credential, and a name such as `API_KEY`, `password` or `token` assigned a value with digits in it; neither of the last two counts when the value is elided (`abc123...`) or holds an example word (`dummy`, `fake`, `sample`, `example`, `placeholder`, `redacted` or `changeme`) in any case with no letter next to it; an assignment whose value starts with `example`, `placeholder`, `redacted` or `changeme` is passed whatever follows. An assignment does not count either when its value is unquoted and is a dotted name directly followed by `(`, such as `base64.b64encode(`. Anything found stops the backup with each file and line (the first 20), and nothing is pushed. A file you have read and know holds no secret goes through with `--allow <path>` for that file, and each run lists the files it lets through this way. A backup that pushes remembers each with the hash of its contents, so later runs pass it without `--allow` until it changes, and names it in `manifest.json`, so a restore copies it as it was allowed. Then the repository is checked without a token: one that answers anyone is public, and is refused. The clone lives beside the config, in `skillcellar/backups/<owner>/<repo>`, readable by you only; each backup brings the clone to the remote's state, commits and pushes, and a commit a failed push left behind goes with the next run.
|
|
37
37
|
- The site learns only the repository's name, the commit and the time, which the device's page shows. It holds no token for the repository and never reads the backup.
|
|
38
38
|
|
|
39
|
-
`skillcellar restore` carries out the copy tasks waiting for this computer ("Copy to…" on the compare page, or "Copy everything to…" on a device's page). The site names the other computer's backup repository and commit, and a name is not proof: the restore takes a repository this computer does not itself back up to only with `--repo <owner>/<repo>`, or a yes at the terminal, and refuses a public one. It fetches the backup into a bare mirror beside the config (`skillcellar/backups/<owner>/<repo>.git`, never checked out, so a link in the repository is never followed), reads the commit the site recorded, and writes its skills, memory files and agents into place: a memory file or agent only when its hash is the one the other computer reported, where it reported one, nothing that holds what looks like a secret, nothing over the skillcellar skill, nothing over a file that is a link, and nothing into a project's `.claude`, `.claude/skills`, `.claude/agents` or `.claude/rules` when that folder is a link; what is there already stays unless you pass `--force`, which replaces every item the tasks name. It prints the commands left: a plugin's install, `npx skills add` for a skill from a repository, and `claude mcp add-json` for each MCP server with the backup's config, where each `${NAME}` is yours to fill in, and a `review` note marks a `${VAR:-default}` or `$VAR` reference or a `headersHelper` to read before adding. It also names the `package.json`, `requirements.txt`, `pyproject.toml` or Playwright config a restored skill carries, so you can rebuild its environment. `--from <device>` takes one computer's tasks; `--ssh` fetches over SSH; `--dry-run` fetches the backup but writes nothing into place. The printed commands are quoted for a POSIX shell; on Windows, paste them into Git Bash or WSL. Then `skillcellar sync` closes each task this computer now satisfies.
|
|
39
|
+
`skillcellar restore` carries out the copy tasks waiting for this computer ("Copy to…" on the compare page, or "Copy everything to…" on a device's page). The site names the other computer's backup repository and commit, and a name is not proof: the restore takes a repository this computer does not itself back up to only with `--repo <owner>/<repo>`, or a yes at the terminal, and refuses a public one. It fetches the backup into a bare mirror beside the config (`skillcellar/backups/<owner>/<repo>.git`, never checked out, so a link in the repository is never followed), reads the commit the site recorded, and writes its skills, memory files and agents into place: a memory file or agent only when its hash is the one the other computer reported, where it reported one, nothing that holds what looks like a secret unless the backup's `manifest.json` names the file as allowed with those very contents, nothing over the skillcellar skill, nothing over a file that is a link, and nothing into a project's `.claude`, `.claude/skills`, `.claude/agents` or `.claude/rules` when that folder is a link; what is there already stays unless you pass `--force`, which replaces every item the tasks name. It prints the commands left: a plugin's install, `npx skills add` for a skill from a repository, and `claude mcp add-json` for each MCP server with the backup's config, where each `${NAME}` is yours to fill in, and a `review` note marks a `${VAR:-default}` or `$VAR` reference or a `headersHelper` to read before adding. It also names the `package.json`, `requirements.txt`, `pyproject.toml` or Playwright config a restored skill carries, so you can rebuild its environment. `--from <device>` takes one computer's tasks; `--ssh` fetches over SSH; `--dry-run` fetches the backup but writes nothing into place. The printed commands are quoted for a POSIX shell; on Windows, paste them into Git Bash or WSL. Then `skillcellar sync` closes each task this computer now satisfies.
|
|
40
40
|
|
|
41
41
|
## What it reads
|
|
42
42
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "skillcellar",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.1",
|
|
4
4
|
"description": "Sync what Claude Code has on this computer (skills, plugins, MCP servers, memory and agents) to your SkillCellar library, by name and hash, list the tasks you queue there for Claude Code to carry out, check which skills and plugins are outdated, and apply playbooks.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"claude-code",
|
package/src/backup.mjs
CHANGED
|
@@ -1,10 +1,11 @@
|
|
|
1
1
|
import { execFile } from "node:child_process";
|
|
2
|
+
import { createHash } from "node:crypto";
|
|
2
3
|
import { lstat, mkdir, readdir, readFile, rm, stat, writeFile } from "node:fs/promises";
|
|
3
4
|
import { dirname, extname, join, relative } from "node:path";
|
|
4
5
|
import { configPath } from "./config.mjs";
|
|
5
6
|
import { markdownFiles } from "./files.mjs";
|
|
6
7
|
import { sessionFolderName } from "./places.mjs";
|
|
7
|
-
import { projectSlug } from "./plan.mjs";
|
|
8
|
+
import { projectIdentity, projectSlug } from "./plan.mjs";
|
|
8
9
|
import { skillFolders } from "./scan.mjs";
|
|
9
10
|
import { findSecret } from "./secrets.mjs";
|
|
10
11
|
|
|
@@ -287,6 +288,7 @@ const sanitizeServers = (servers) => Object.fromEntries(Object.entries(servers ?
|
|
|
287
288
|
// ── What a device backs up ──────────────────────────────────────────────────────────────────
|
|
288
289
|
|
|
289
290
|
const text = (path, source, content) => ({ path, source, content: Buffer.from(content), executable: false });
|
|
291
|
+
const manifestFile = (base, manifest) => text(`${base}/manifest.json`, "manifest", `${JSON.stringify(manifest, null, 2)}\n`);
|
|
290
292
|
|
|
291
293
|
/** The `.md` files of a folder, for the backup, as `<into>/<path>`; a folder's absence is nothing to back up, one that cannot be read is named. */
|
|
292
294
|
async function markdownInto(dir, into, own, skipped) {
|
|
@@ -362,52 +364,80 @@ export async function collectBackup({ device, claudeDir, claudeJson, projects, t
|
|
|
362
364
|
if (Object.keys(globalServers).length) files.push(text(`${base}/global/mcp.json`, "~/.claude.json (mcpServers)", `${JSON.stringify({ mcpServers: globalServers }, null, 2)}\n`));
|
|
363
365
|
|
|
364
366
|
const places = [{ folder: "global", files: files.length }];
|
|
365
|
-
|
|
367
|
+
// Every checkout of one project, such as two clones of a repository, goes in the project's one
|
|
368
|
+
// folder, as the site merges them into one project. Another project whose name makes the same
|
|
369
|
+
// folder name is left out.
|
|
370
|
+
const groups = new Map();
|
|
366
371
|
for (const project of projects) {
|
|
367
372
|
const slug = projectSlug(project, device.slug);
|
|
368
373
|
if (!slug) continue;
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
}
|
|
373
|
-
|
|
374
|
+
const group = groups.get(slug);
|
|
375
|
+
if (!group) groups.set(slug, [project]);
|
|
376
|
+
else if (projectIdentity(group[0]) === projectIdentity(project)) group.push(project);
|
|
377
|
+
else skipped.push({ path: project.root, reason: `a folder of the same name, ${slug}, is backed up already` });
|
|
378
|
+
}
|
|
379
|
+
for (const [slug, group] of groups) {
|
|
380
|
+
// In the order the sync sends them, which the site merges them in: by name, then as found.
|
|
381
|
+
const checkouts = [...group].sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
|
|
374
382
|
const into = `${base}/projects/${slug}`;
|
|
375
|
-
const own = { root: project.root, tracked: await trackedOf(project.root) };
|
|
376
383
|
const before = files.length;
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
const
|
|
383
|
-
for (const
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
384
|
+
// The first checkout's item of a name wins, as on the site: a file by its path, a skill by its whole folder.
|
|
385
|
+
const item = (path) => (path.startsWith(`${into}/skills/`) ? path.split("/", into.split("/").length + 2).join("/") : path);
|
|
386
|
+
const taken = new Set();
|
|
387
|
+
const shared = new Map();
|
|
388
|
+
const local = new Map();
|
|
389
|
+
const sharedSources = [];
|
|
390
|
+
for (const project of checkouts) {
|
|
391
|
+
const own = { root: project.root, tracked: await trackedOf(project.root) };
|
|
392
|
+
const found = [
|
|
393
|
+
...(await skillsInto(join(project.root, ".claude", "skills"), `${into}/skills`, own, skipped, budget)),
|
|
394
|
+
...(await oneFile(join(project.root, "CLAUDE.md"), `${into}/CLAUDE.md`, own, skipped)),
|
|
395
|
+
...(await oneFile(join(project.root, "AGENTS.md"), `${into}/AGENTS.md`, own, skipped)),
|
|
396
|
+
...(await markdownInto(join(project.root, ".claude", "rules"), `${into}/rules`, own, skipped)),
|
|
397
|
+
...(await markdownInto(join(project.root, ".claude", "agents"), `${into}/agents`, own, skipped)),
|
|
398
|
+
];
|
|
399
|
+
const seen = new Set();
|
|
400
|
+
for (const folder of new Set([project.root, ...(project.folders ?? [])])) {
|
|
401
|
+
for (const file of await markdownInto(join(claudeDir, "projects", sessionFolderName(folder), "memory"), `${into}/memory`, null, skipped)) {
|
|
402
|
+
if (seen.has(file.path)) continue;
|
|
403
|
+
seen.add(file.path);
|
|
404
|
+
found.push(file);
|
|
405
|
+
}
|
|
388
406
|
}
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
servers = mcpJson === null ? null : JSON.parse(mcpJson)?.mcpServers;
|
|
395
|
-
} catch {
|
|
396
|
-
skipped.push({ path: join(project.root, ".mcp.json"), reason: "not valid JSON" });
|
|
407
|
+
const lost = new Map();
|
|
408
|
+
for (const file of found) {
|
|
409
|
+
const key = item(file.path);
|
|
410
|
+
if (!taken.has(key)) files.push(file);
|
|
411
|
+
else if (!lost.has(key)) lost.set(key, key.startsWith(`${into}/skills/`) ? join(project.root, ".claude", "skills", key.slice(`${into}/skills/`.length)) : file.source);
|
|
397
412
|
}
|
|
398
|
-
|
|
399
|
-
|
|
413
|
+
for (const path of lost.values()) skipped.push({ path, reason: `another checkout of ${project.name} has one of the same name, backed up instead, as the site keeps one` });
|
|
414
|
+
for (const file of found) taken.add(item(file.path));
|
|
415
|
+
if (!own.tracked.has(".mcp.json")) {
|
|
416
|
+
const mcpJson = await readFile(join(project.root, ".mcp.json"), "utf8").catch(() => null);
|
|
417
|
+
let servers = null;
|
|
418
|
+
try {
|
|
419
|
+
servers = mcpJson === null ? null : JSON.parse(mcpJson)?.mcpServers;
|
|
420
|
+
} catch {
|
|
421
|
+
skipped.push({ path: join(project.root, ".mcp.json"), reason: "not valid JSON" });
|
|
422
|
+
}
|
|
423
|
+
if (servers && typeof servers === "object" && Object.keys(servers).length) {
|
|
424
|
+
sharedSources.push(join(project.root, ".mcp.json"));
|
|
425
|
+
for (const [name, server] of Object.entries(sanitizeServers(servers))) if (!shared.has(name)) shared.set(name, server);
|
|
426
|
+
}
|
|
400
427
|
}
|
|
428
|
+
const ownLocal = {};
|
|
429
|
+
for (const folder of [project.root, ...(project.folders ?? [])]) Object.assign(ownLocal, sanitizeServers(claudeJson?.projects?.[folder]?.mcpServers));
|
|
430
|
+
for (const [name, server] of Object.entries(ownLocal)) if (!local.has(name)) local.set(name, server);
|
|
401
431
|
}
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
432
|
+
if (shared.size) files.push({ ...text(`${into}/mcp.json`, sharedSources.join(", "), `${JSON.stringify({ mcpServers: Object.fromEntries(shared) }, null, 2)}\n`), sources: sharedSources });
|
|
433
|
+
if (local.size) files.push(text(`${into}/mcp.local.json`, `~/.claude.json (projects, mcpServers)`, `${JSON.stringify({ mcpServers: Object.fromEntries(local) }, null, 2)}\n`));
|
|
434
|
+
const [project] = checkouts;
|
|
405
435
|
places.push({ folder: `projects/${slug}`, name: project.name, repo: project.repo ?? null, remote: Boolean(project.remote), files: files.length - before });
|
|
406
436
|
}
|
|
407
437
|
|
|
408
438
|
// No time in the manifest: the commit carries it, and an unchanged computer then commits nothing.
|
|
409
439
|
const manifest = { skillcellar: version, device: { id: device.id, slug: device.slug, name: device.name }, places };
|
|
410
|
-
files.push(
|
|
440
|
+
files.push(manifestFile(base, manifest));
|
|
411
441
|
return { files, skipped, manifest };
|
|
412
442
|
}
|
|
413
443
|
|
|
@@ -415,18 +445,27 @@ export async function collectBackup({ device, claudeDir, claudeJson, projects, t
|
|
|
415
445
|
const FINDINGS_MAX = 20;
|
|
416
446
|
|
|
417
447
|
/**
|
|
418
|
-
* The files of a backup that hold what looks like a secret, each with the line, up to FINDINGS_MAX
|
|
419
|
-
*
|
|
448
|
+
* The files of a backup that hold what looks like a secret, each with the line, up to FINDINGS_MAX,
|
|
449
|
+
* and those that pass despite one, each with the hash of what passed: a file the user allows by its
|
|
450
|
+
* path, whole or from a folder's start, and one an earlier backup let through holding these bytes.
|
|
420
451
|
*/
|
|
421
|
-
export function findSecretsIn(files, allowed = []) {
|
|
452
|
+
export function findSecretsIn(files, allowed = [], remembered = []) {
|
|
422
453
|
const findings = [];
|
|
454
|
+
const passed = [];
|
|
423
455
|
for (const file of files) {
|
|
424
|
-
if (allowed.some((path) => file.source === path || file.source.endsWith(`/${path}`))) continue;
|
|
425
456
|
const found = findSecret(file.content.toString("utf8"));
|
|
426
|
-
if (found)
|
|
457
|
+
if (!found) continue;
|
|
458
|
+
const sha256 = createHash("sha256").update(file.content).digest("hex");
|
|
459
|
+
// A merged file, such as the mcp.json of several checkouts, is allowed by any file it came from.
|
|
460
|
+
const names = file.sources ?? [file.source];
|
|
461
|
+
if (allowed.some((path) => names.some((name) => name === path || name.endsWith(`/${path}`))) || remembered.some((entry) => entry?.file === file.source && entry.sha256 === sha256)) {
|
|
462
|
+
passed.push({ file: file.source, path: file.path, sha256, line: found.line, kind: found.kind });
|
|
463
|
+
continue;
|
|
464
|
+
}
|
|
465
|
+
findings.push({ ...found, file: file.source });
|
|
427
466
|
if (findings.length >= FINDINGS_MAX) break;
|
|
428
467
|
}
|
|
429
|
-
return findings;
|
|
468
|
+
return { findings, passed };
|
|
430
469
|
}
|
|
431
470
|
|
|
432
471
|
/** Writes the backup's files into the clone, replacing the device's folder and any older folder of the same device under another slug. */
|
|
@@ -514,14 +553,27 @@ export function describeSkipped(skipped) {
|
|
|
514
553
|
|
|
515
554
|
const plural = (n, word) => `${n} ${word}${n === 1 ? "" : "s"}`;
|
|
516
555
|
|
|
556
|
+
/** The files let through despite a finding, as lines a person reads, so an allowance is never silent. */
|
|
557
|
+
function describePassed(passed) {
|
|
558
|
+
if (!passed.length) return [];
|
|
559
|
+
return ["Let through as you allowed them:", ...passed.map(({ file, line, kind }) => ` ${file}:${line} looks like ${kind}`)];
|
|
560
|
+
}
|
|
561
|
+
|
|
517
562
|
/**
|
|
518
563
|
* Backs up the device: collects its files, stops on anything that looks like a secret, refuses a
|
|
519
564
|
* public repository, then commits and pushes with the user's own git. Answers what to print and the
|
|
520
565
|
* commit to report, or null when nothing changed.
|
|
521
566
|
*/
|
|
522
|
-
export async function runBackup({ repo, device, claudeDir, claudeJson, projects, trackedOf, version, dryRun, allowed = [], print, onPushed = async () => {} }) {
|
|
523
|
-
const
|
|
524
|
-
const
|
|
567
|
+
export async function runBackup({ repo, device, claudeDir, claudeJson, projects, trackedOf, version, dryRun, allowed = [], remembered = [], print, onPushed = async () => {} }) {
|
|
568
|
+
const collected = await collectBackup({ device, claudeDir, claudeJson, projects, trackedOf, version });
|
|
569
|
+
const { skipped } = collected;
|
|
570
|
+
const { findings, passed } = findSecretsIn(collected.files, allowed, remembered);
|
|
571
|
+
// What passed despite a finding is named in the manifest with the hash of what passed, so a
|
|
572
|
+
// restore copies that and reads any other copy of the file again.
|
|
573
|
+
const base = `devices/${device.slug}`;
|
|
574
|
+
const allowedFiles = passed.map(({ path, sha256 }) => ({ path: path.slice(base.length + 1), sha256 })).sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
|
|
575
|
+
const manifest = allowedFiles.length ? { ...collected.manifest, allowed: allowedFiles } : collected.manifest;
|
|
576
|
+
const files = allowedFiles.length ? collected.files.map((file) => (file.path === `${base}/manifest.json` ? manifestFile(base, manifest) : file)) : collected.files;
|
|
525
577
|
if (findings.length) {
|
|
526
578
|
const lines = findings.map((found) => ` ${found.file}:${found.line} looks like ${found.kind}`);
|
|
527
579
|
throw new Error(
|
|
@@ -532,7 +584,7 @@ export async function runBackup({ repo, device, claudeDir, claudeJson, projects,
|
|
|
532
584
|
const summary = `${plural(files.length, "file")} (${size(bytes)}): ${manifest.places.map((place) => `${place.folder} ${place.files}`).join(", ")}`;
|
|
533
585
|
if (dryRun) {
|
|
534
586
|
print(`Would back up ${device.name} to ${repo.name}, ${summary}.`);
|
|
535
|
-
for (const line of describeSkipped(skipped)) print(line);
|
|
587
|
+
for (const line of [...describeSkipped(skipped), ...describePassed(passed)]) print(line);
|
|
536
588
|
return null;
|
|
537
589
|
}
|
|
538
590
|
await refusePublic(repo, version);
|
|
@@ -548,10 +600,10 @@ export async function runBackup({ repo, device, claudeDir, claudeJson, projects,
|
|
|
548
600
|
if (await currentBranch(clone)) {
|
|
549
601
|
await git(["push", "--quiet", "-u", "origin", "HEAD"], { cwd: clone, timeout: 600_000 });
|
|
550
602
|
commit = (await git(["rev-parse", "HEAD"], { cwd: clone })).trim();
|
|
551
|
-
await onPushed(commit);
|
|
603
|
+
await onPushed(commit, passed);
|
|
552
604
|
}
|
|
553
605
|
if (changed) print(`Backed up ${device.name} to ${repo.name} at ${commit.slice(0, 7)}, ${summary}.`);
|
|
554
606
|
else print(`Nothing changed since the last backup of ${device.name} in ${repo.name}${commit ? ` (${commit.slice(0, 7)})` : ""}.`);
|
|
555
|
-
for (const line of describeSkipped(skipped)) print(line);
|
|
607
|
+
for (const line of [...describeSkipped(skipped), ...describePassed(passed)]) print(line);
|
|
556
608
|
return commit;
|
|
557
609
|
}
|
package/src/cli.mjs
CHANGED
|
@@ -7,7 +7,7 @@ import { parseBackupRepo, runBackup } from "./backup.mjs";
|
|
|
7
7
|
import { describeCheck } from "./check.mjs";
|
|
8
8
|
import { configPath, readConfig, writeConfig } from "./config.mjs";
|
|
9
9
|
import { findProjects } from "./places.mjs";
|
|
10
|
-
import { describePlan, localPlan } from "./plan.mjs";
|
|
10
|
+
import { describePlan, localPlan, projectIdentity } from "./plan.mjs";
|
|
11
11
|
import { applyPlaybook, cautionLine, describeStatus, readPlaybook } from "./playbook.mjs";
|
|
12
12
|
import { findProject, trackedFiles } from "./project.mjs";
|
|
13
13
|
import { ask, askSecret, firstLine, interactive } from "./prompt.mjs";
|
|
@@ -68,8 +68,8 @@ Usage:
|
|
|
68
68
|
address credential and key-like argument replaced by a \${NAME} placeholder) and skills'
|
|
69
69
|
source files to a private GitHub repository you own, with your own git. It stops on anything
|
|
70
70
|
that looks like a secret and refuses a public repository. --repo is remembered; --ssh clones
|
|
71
|
-
and pushes over SSH; --allow <file> pushes a file you have read despite a finding in it
|
|
72
|
-
--dry-run lists what would go.
|
|
71
|
+
and pushes over SSH; --allow <file> pushes a file you have read despite a finding in it,
|
|
72
|
+
and once pushed is remembered while the file stays the same; --dry-run lists what would go.
|
|
73
73
|
skillcellar restore [--from <device>] [--repo <owner>/<repo>] [--ssh] [--force] [--dry-run]
|
|
74
74
|
Carry out the copy tasks waiting for this computer: fetch the other computer's backup, write
|
|
75
75
|
its skills, memory files and agents into place, and list the commands left to run. A backup
|
|
@@ -202,9 +202,6 @@ function summary(payload, result, deviceName, whole) {
|
|
|
202
202
|
*/
|
|
203
203
|
const claudeJsonPath = (home) => (process.env.CLAUDE_CONFIG_DIR?.trim() ? join(process.env.CLAUDE_CONFIG_DIR.trim(), ".claude.json") : join(home, ".claude.json"));
|
|
204
204
|
|
|
205
|
-
/** What makes checkouts one project on the site: their GitHub repository, else their name. */
|
|
206
|
-
const identity = (project) => project.repo ?? project.name.toLowerCase();
|
|
207
|
-
|
|
208
205
|
/** The tasks a sync closed, in one line. */
|
|
209
206
|
function closedLine(closed) {
|
|
210
207
|
const verb = (task) => (task.action === "remove" ? "removed" : task.action === "copy" ? "copied" : "installed");
|
|
@@ -232,7 +229,7 @@ async function sync(flags, print, warn) {
|
|
|
232
229
|
if (!here) throw new UsageError("Run sync --project inside the project's folder, not in your home folder.");
|
|
233
230
|
if (!/[A-Za-z0-9]/.test(flags.project)) throw new UsageError("--project needs a letter a–z or a digit 0–9.");
|
|
234
231
|
// Every checkout of the same repository is one project on the site, so all of them go together.
|
|
235
|
-
const same = projects.filter((known) => known.root === here.root || (here.remote && known.remote &&
|
|
232
|
+
const same = projects.filter((known) => known.root === here.root || (here.remote && known.remote && projectIdentity(known) === projectIdentity(here)));
|
|
236
233
|
projects = (same.length ? same : [{ ...here, folders: new Set([cwd]) }]).map((known) => ({ ...known, name: flags.project }));
|
|
237
234
|
}
|
|
238
235
|
|
|
@@ -252,7 +249,7 @@ async function sync(flags, print, warn) {
|
|
|
252
249
|
} catch (error) {
|
|
253
250
|
// One project's broken file stops only that project in a sync of the whole computer.
|
|
254
251
|
if (!whole) throw error;
|
|
255
|
-
unread.add(
|
|
252
|
+
unread.add(projectIdentity(project));
|
|
256
253
|
warn(`Not reported: ${project.name}, because ${error.message} The site keeps what it had there until a sync can read it.`);
|
|
257
254
|
continue;
|
|
258
255
|
}
|
|
@@ -266,7 +263,7 @@ async function sync(flags, print, warn) {
|
|
|
266
263
|
// one as it was and closes no task there; an empty place is then sent, to empty it. Another
|
|
267
264
|
// checkout of the unread project is left out too, or it would replace the project on its own.
|
|
268
265
|
const complete = whole && !unread.size;
|
|
269
|
-
if (!complete) places = [...places, ...empty].filter((place) => !unread.has(
|
|
266
|
+
if (!complete) places = [...places, ...empty].filter((place) => !unread.has(projectIdentity(place.project)));
|
|
270
267
|
if (places.length > PROJECTS_PER_SYNC) {
|
|
271
268
|
throw new Error(`This computer has ${places.length} projects to send, and one sync takes ${PROJECTS_PER_SYNC}. Sync each one you need from its folder with --project <name>.`);
|
|
272
269
|
}
|
|
@@ -415,9 +412,15 @@ async function backup(flags, print, warn) {
|
|
|
415
412
|
const claudeJson = (await readJson(claudeJsonPath(home))) ?? {};
|
|
416
413
|
const projects = await knownProjects(home, claudeDir);
|
|
417
414
|
const allowed = (Array.isArray(flags.allow) ? flags.allow : []).map((path) => path.trim()).filter(Boolean);
|
|
418
|
-
// The repository is remembered as soon as a push has gone through, whatever the site answers after
|
|
419
|
-
|
|
420
|
-
const
|
|
415
|
+
// The repository is remembered as soon as a push has gone through, whatever the site answers after,
|
|
416
|
+
// with the files the user allowed despite a finding, each by the hash of what was allowed.
|
|
417
|
+
const remember = (commit, passed = []) => {
|
|
418
|
+
if (!config) return Promise.resolve();
|
|
419
|
+
const kept = passed.length ? { allowed: passed.map(({ file, sha256 }) => ({ file, sha256 })) } : {};
|
|
420
|
+
return writeConfig({ ...config, device: { id: device.id, slug: device.slug, name: device.name }, backup: { repo: repo.name, ssh: repo.ssh, ...kept } });
|
|
421
|
+
};
|
|
422
|
+
const remembered = Array.isArray(config?.backup?.allowed) ? config.backup.allowed : [];
|
|
423
|
+
const commit = await runBackup({ repo, device, claudeDir, claudeJson, projects, trackedOf: trackedFiles, version: VERSION, dryRun, allowed, remembered, print, onPushed: remember });
|
|
421
424
|
if (dryRun || !commit) return;
|
|
422
425
|
const recorded = await call(site, config.key, "POST", "/api/v1/backup", { device: device.id, repo: repo.name, commit }, VERSION);
|
|
423
426
|
const slug = recorded.device?.slug ?? device.slug;
|
package/src/plan.mjs
CHANGED
|
@@ -15,6 +15,9 @@ export const toSlug = (name) =>
|
|
|
15
15
|
/** A project's slug as the site makes it: its name's, and for a folder with no git remote that followed by the device's. */
|
|
16
16
|
export const projectSlug = (project, deviceSlug) => (project.remote ? toSlug(project.name) : `${toSlug(project.name)}-${deviceSlug}`);
|
|
17
17
|
|
|
18
|
+
/** What makes checkouts one project on the site: their GitHub repository, else their name. */
|
|
19
|
+
export const projectIdentity = (project) => project.repo ?? project.name.toLowerCase();
|
|
20
|
+
|
|
18
21
|
/**
|
|
19
22
|
* The folder on this computer of a task's project, from the projects the sync finds: one with a
|
|
20
23
|
* GitHub repository by that repository, any other by its slug, which for a folder with no git remote
|
package/src/restore.mjs
CHANGED
|
@@ -122,7 +122,12 @@ async function checkProjectFolder(root, sub) {
|
|
|
122
122
|
* command installs, a plugin, a skill from a repository or an MCP server, is answered as that
|
|
123
123
|
* command for Claude Code or the user to run. Nothing is run here.
|
|
124
124
|
*/
|
|
125
|
-
async function restoreItem(item, { mirror, tree, budget, place, scope, root, claudeDir, force, dryRun }) {
|
|
125
|
+
async function restoreItem(item, { mirror, tree, budget, place, scope, root, claudeDir, force, dryRun, allowed }) {
|
|
126
|
+
// A file that looks like it holds a secret is copied only as the source's user allowed it, byte for byte.
|
|
127
|
+
const secretIn = (path, content) => {
|
|
128
|
+
const found = findSecret(content.toString("utf8"));
|
|
129
|
+
return found && allowed.get(path) !== sha256(content) ? found : null;
|
|
130
|
+
};
|
|
126
131
|
const { kind, name } = item;
|
|
127
132
|
if (item.tracked) return { outcome: "repository" };
|
|
128
133
|
if (kind === "plugin" || (kind === "skill" && item.install)) {
|
|
@@ -153,7 +158,7 @@ async function restoreItem(item, { mirror, tree, budget, place, scope, root, cla
|
|
|
153
158
|
if (!path.startsWith(prefix)) continue;
|
|
154
159
|
if (!regular(entry)) return { outcome: "link", path };
|
|
155
160
|
const content = await blobOf(mirror, entry, budget);
|
|
156
|
-
const found =
|
|
161
|
+
const found = secretIn(path, content);
|
|
157
162
|
if (found) return { outcome: "secret", path, line: found.line, kind: found.kind };
|
|
158
163
|
files.push({ segments: path.slice(prefix.length).split("/"), content, executable: entry.mode === "100755" });
|
|
159
164
|
}
|
|
@@ -192,7 +197,7 @@ async function restoreItem(item, { mirror, tree, budget, place, scope, root, cla
|
|
|
192
197
|
const content = await blobOf(mirror, entry, budget);
|
|
193
198
|
// The site holds the file's hash as the source reported it: a file that differs is not the source's.
|
|
194
199
|
if (item.hash && sha256(content) !== item.hash) return { outcome: "differs" };
|
|
195
|
-
const found =
|
|
200
|
+
const found = secretIn(path, content);
|
|
196
201
|
if (found) return { outcome: "secret", path, line: found.line, kind: found.kind };
|
|
197
202
|
if (dryRun) return { outcome: (await lstat(target).catch(() => null)) && !force ? "exists" : "copied", target };
|
|
198
203
|
return { outcome: await placeFile(content, target, force), target };
|
|
@@ -295,6 +300,8 @@ export async function runRestore({ plan, claudeDir, config, flags, version, prin
|
|
|
295
300
|
warn(`${repo.name} at ${commit.slice(0, 7)} holds no backup of ${source.device.name}. Run skillcellar backup there, then restore here again.`);
|
|
296
301
|
continue;
|
|
297
302
|
}
|
|
303
|
+
const listed = Array.isArray(found.manifest?.allowed) ? found.manifest.allowed : [];
|
|
304
|
+
const allowed = new Map(listed.filter((entry) => typeof entry?.path === "string" && typeof entry.sha256 === "string").map((entry) => [`${found.folder}/${entry.path}`, entry.sha256]));
|
|
298
305
|
print(`${dryRun ? "Would restore" : "Restoring"} from ${source.device.name}'s backup, ${repo.name} at ${commit.slice(0, 7)}:`);
|
|
299
306
|
for (const task of tasks) {
|
|
300
307
|
const where = task.project ? `in ${task.project.name}` : "globally";
|
|
@@ -320,7 +327,7 @@ export async function runRestore({ plan, claudeDir, config, flags, version, prin
|
|
|
320
327
|
continue;
|
|
321
328
|
}
|
|
322
329
|
done.add(key);
|
|
323
|
-
const result = await restoreItem(item, { mirror, tree, budget, place, scope: task.scope, root: task.folder, claudeDir, force, dryRun });
|
|
330
|
+
const result = await restoreItem(item, { mirror, tree, budget, place, scope: task.scope, root: task.folder, claudeDir, force, dryRun, allowed });
|
|
324
331
|
if (result.outcome === "command") {
|
|
325
332
|
commands.push({ task: where, label, ...result });
|
|
326
333
|
continue;
|
package/src/secrets.mjs
CHANGED
|
@@ -9,6 +9,16 @@ const EDGE = String.raw`(?<![A-Za-z0-9])`;
|
|
|
9
9
|
const END = String.raw`(?![A-Za-z0-9])`;
|
|
10
10
|
// A value that is not a placeholder, a variable, a template, an example or a reference into config or code.
|
|
11
11
|
const NOT_PLACEHOLDER = String.raw`(?!\$\{|\$[A-Za-z_]|<|\{\{|%|your[_-]|xxx|example|placeholder|change[_-]?me|redacted|none\b|null\b|true\b|false\b|\*{3}|process\.|os\.|env\.|config\.|settings\.|this\.|self\.)`;
|
|
12
|
+
// The characters an assigned or bearer value is read with.
|
|
13
|
+
const VALUE = String.raw`[A-Za-z0-9_\-./+=]`;
|
|
14
|
+
// A word in either case, for a pattern that minds case.
|
|
15
|
+
const anyCase = (word) => word.replace(/[a-z]/g, (letter) => `[${letter}${letter.toUpperCase()}]`);
|
|
16
|
+
const EXAMPLE_WORDS = ["dummy", "fake", "sample", "example", "placeholder", "redacted", "change[_-]?me"].map(anyCase).join("|");
|
|
17
|
+
// A value as documentation writes one, not a key: elided (abc123... or abc123…), or holding a word,
|
|
18
|
+
// not part of a longer one, that calls it an example. Real keys are neither.
|
|
19
|
+
const NOT_EXAMPLE = String.raw`(?!${VALUE}{0,255}?(?:\.\.\.|(?<![A-Za-z])(?:${EXAMPLE_WORDS})(?![A-Za-z])))(?!${VALUE}{1,255}…)`;
|
|
20
|
+
// Code such as base64.b64encode( is no value; a quoted value is one, whatever it holds.
|
|
21
|
+
const NOT_CALL = String.raw`(?:(?<=["'\x60])|(?![A-Za-z_]\w{0,60}(?:\.[A-Za-z_]\w{0,60}){1,5}\())`;
|
|
12
22
|
|
|
13
23
|
const PATTERNS = [
|
|
14
24
|
["a private key", /-----BEGIN (?:RSA |EC |DSA |OPENSSH |PGP |ENCRYPTED )?PRIVATE KEY(?: BLOCK)?-----/],
|
|
@@ -25,12 +35,12 @@ const PATTERNS = [
|
|
|
25
35
|
["a JSON Web Token", new RegExp(String.raw`${EDGE}eyJ[A-Za-z0-9_-]{10,2000}\.[A-Za-z0-9_-]{10,2000}\.[A-Za-z0-9_-]{10,2000}(?![A-Za-z0-9_-])`)],
|
|
26
36
|
// user:password@host in an address of any scheme, unless the password is a placeholder.
|
|
27
37
|
["a password in an address", /[a-z][a-z0-9+.-]{0,30}:\/\/[^\s/@:'"]{1,100}:(?!\$\{)[^\s/@'"]{1,200}@/i],
|
|
28
|
-
["a bearer or basic credential", new RegExp(String.raw`\b(?:Bearer|Basic)\s{1,5}(?!\$\{|\$[A-Za-z_]|<)[A-Za-z0-9_\-./+=]{16,2000}(?![A-Za-z0-9_\-./+=])`)],
|
|
38
|
+
["a bearer or basic credential", new RegExp(String.raw`\b(?:Bearer|Basic)\s{1,5}(?!\$\{|\$[A-Za-z_]|<)${NOT_EXAMPLE}[A-Za-z0-9_\-./+=]{16,2000}(?![A-Za-z0-9_\-./+=])`)],
|
|
29
39
|
// A key named as one, assigned a value with a digit in it, as random keys have; the name may be quoted, as in JSON.
|
|
30
40
|
[
|
|
31
41
|
"a key or password in an assignment",
|
|
32
42
|
new RegExp(
|
|
33
|
-
String.raw`${EDGE}(?:api[_-]?key|apikey|secret[_-]?(?:access[_-]?)?key|access[_-]?token|auth[_-]?token|refresh[_-]?token|client[_-]?secret|private[_-]?key|password|passwd|secret|token)${END}["'\x60]?\s{0,5}[:=]\s{0,5}["'\x60]?${NOT_PLACEHOLDER}(?=[A-Za-z0-9_\-./+=]{0,254}[0-9])[A-Za-z0-9_\-./+=]{8,255}`,
|
|
43
|
+
String.raw`${EDGE}(?:api[_-]?key|apikey|secret[_-]?(?:access[_-]?)?key|access[_-]?token|auth[_-]?token|refresh[_-]?token|client[_-]?secret|private[_-]?key|password|passwd|secret|token)${END}["'\x60]?\s{0,5}[:=]\s{0,5}["'\x60]?${NOT_PLACEHOLDER}${NOT_EXAMPLE}${NOT_CALL}(?=[A-Za-z0-9_\-./+=]{0,254}[0-9])[A-Za-z0-9_\-./+=]{8,255}`,
|
|
34
44
|
"i",
|
|
35
45
|
),
|
|
36
46
|
],
|