disk-space-saviour 0.16.0 → 0.17.0

package/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 0.17.0
+
+### Minor Changes
+
+- a518d7b: Reuse recent authenticated scans, stream safe global-cache cleanup before project discovery, and stop emergency scans as soon as the free-space goal is met. Add bounded discovery, persistent directory and size indexes, separate candidate/sizing progress counters, and macOS swap diagnostics while retaining fresh deletion checks.
+
 ## 0.16.0
 
 ### Minor Changes

package/README.md

@@ -76,7 +76,14 @@ Run `dss --help` for every option. The most useful ones:
 - `--scanner NAME` restricts scanning to `projects`, `global`, `versions`,
   `agents` or `system`.
 - `--report FILE` limits cleaning to items in a saved `dss scan --json`
-  report. The current rules rescan those items before deletion.
+  report. `--from FILE` also accepts an audit JSON containing the full report.
+  Recent reports authenticated by this installation are reused; older, unsigned
+  or edited reports require discovery again. Changing discovery or action options
+  also rebuilds the plan. Every deletion still gets fresh checks.
+- `clean` automatically reuses a matching scan from the last 30 minutes.
+  `--fresh` bypasses this plan and the discovery/size index; `--cache-dir DIR`
+  chooses the state location (default `$XDG_STATE_HOME/disk-space-saviour`, or
+  `~/.local/state/disk-space-saviour`).
 
 ## Tiers
 
@@ -179,6 +186,38 @@ print completed findings marked `aborted`; incomplete findings remain blocked.
 The CLI exits 130 or 143 respectively. A second signal exits immediately.
 Library callers can pass an `AbortSignal` and an `onScanProgress` callback.
 
+Global caches run first, prioritizing rules that can use native sizing without
+recursive activity timestamps, and emit findings as each rule finishes. Non-interactive
+`clean` plans or deletes completed safe host caches while the remaining discovery
+continues. `emergency` does the same, checks actual free space after each deletion,
+and skips the remaining scan once the goal is met. Interactive runs show a plan
+before asking for deletion consent. A later phase hitting its budget blocks its
+partial findings while completed independent findings stay usable.
+
+Project discovery uses eight directory workers, prunes matched generated trees,
+Git objects, home media/Photos libraries, macOS Library and mounted volumes.
+Source activity is checked separately without sizing entire project trees again.
+Spotlight (`mdfind`) or `plocate`/`locate` supplies verified early hints when
+available; the walk also runs because an index can miss files. Version-pin discovery
+also prunes dependency/build trees.
+
+The persistent `index/` stores directory listings and candidate measurements.
+Each descendant directory's own mtime/ctime and identity is checked before reusing
+its listing, so a nested addition is visible even when an ancestor did not change.
+Unchanged directories avoid another listing; discovery still checks their metadata.
+Sizing uses bounded workers and `du -skx` when no recursive activity timestamp is
+needed. Sizes cached for up to an hour are labeled **estimated**, because changing
+an existing descendant need not update the candidate's directory mtime. Cleanup
+always bypasses that cache and rechecks processes, open files, Git, path boundaries
+and sizes. Safe cleanup also rechecks source activity. Explicit moderate cleanup
+preserves the ability to remove ignored dependencies when only source changed.
+Library callers opt into state with `cacheDir`.
+
+On macOS, reports include `sysctl vm.swapusage` and the largest resident/compressed
+memory consumers from `top`. These are memory diagnostics, not verified per-process
+swap ownership. When Docker Desktop consumes memory, saving work and restarting it
+or lowering its VM memory limit can help. dss never deletes swap files.
+
 Cache discovery honors tool configuration before default paths, including
 `GOMODCACHE`, `GOPATH`, `GOCACHE`, `CARGO_HOME`, `RUSTUP_HOME`,
 `GRADLE_USER_HOME`, `DENO_DIR`, `BUN_INSTALL_CACHE_DIR` and `XDG_CACHE_HOME`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "disk-space-saviour",
-  "version": "0.16.0",
+  "version": "0.17.0",
   "description": "Safely reclaims disk space: caches and build outputs of 20+ ecosystems, forgotten projects, and Docker (running, stopped and nested containers), with an emergency mode",
   "type": "module",
   "main": "src/index.js",

package/src/clean.js

@@ -44,6 +44,7 @@ import { OTHER_RULES } from './rules/other.js';
 import { compareVersions, VERSION_RULES } from './rules/versions.js';
 import { scanVersions } from './scanners/versions.js';
 import { analyzeRustProfile } from './scanners/rust.js';
+import { currentProjectTier } from './scanners/projects.js';
 import {
   filterItems,
   revalidateReport,
@@ -161,7 +162,7 @@ export class Cleaner {
     this.report = report;
     this.options = options;
     this.contexts = new Map();
-    this.health = new HealthWatch();
+    this.health = new HealthWatch(options.priorHealth);
     this.descriptors = new Map(
       (report.environments ?? []).map((env) => [env.id, env])
     );
@@ -371,6 +372,15 @@ export class Cleaner {
   }
 
   async #staticBlocker(ctx, item) {
+    if (item.project && this.options.tier === 'safe') {
+      const tier = await currentProjectTier(
+        { ...ctx, options: this.options, now: this.options.now() },
+        item
+      );
+      if (tierRank(tier) > tierRank(this.options.tier)) {
+        return `project recently active; requires the ${tier} tier`;
+      }
+    }
     if (item.recheck?.type === 'version') {
       if (!ctx.cacheRootsConfigured) {
         await configureCacheRoots(ctx.env, await ctx.env.homeDirs());
@@ -385,6 +395,7 @@ export class Cleaner {
           options: {
             ...this.options,
             homes: [item.recheck.home],
+            freshSizes: true,
             roots: this.report.options.roots ?? [],
           },
           now: this.options.now(),
@@ -429,9 +440,18 @@ export class Cleaner {
     if (
       ctx.env.kind === 'host' &&
       type === 'remove' &&
-      this.report.docker?.daemons?.some((daemon) => daemon.env === ctx.env.id)
+      (this.report.docker?.daemons?.some(
+        (daemon) => daemon.env === ctx.env.id
+      ) ||
+        this.options.streaming)
     ) {
-      const mounts = await runningBindMounts(new DockerCli(ctx.executor));
+      const docker = new DockerCli(ctx.executor);
+      if (this.options.streaming && !(await docker.info())) {
+        return this.options.docker === true
+          ? 'cannot verify running Docker bind mounts'
+          : null;
+      }
+      const mounts = await runningBindMounts(docker);
       for (const mount of mounts) {
         if (
           item.paths.some(
@@ -907,7 +927,8 @@ export async function clean(report, input = {}) {
   const audit = startAudit(input.command ?? 'clean', {
     dryRun: Boolean(options.dryRun),
     tier: options.tier,
-    report: { createdAt: report.createdAt, totals: report.totals },
+    report,
+    entries: [...(input.priorEntries ?? [])],
   });
   const persistProgress = async (entry) => {
     if (!audit.entries.includes(entry)) {
@@ -927,7 +948,10 @@ export async function clean(report, input = {}) {
     await writeAudit(audit, { ...options, inProgress: true });
   }
   try {
-    for (const item of planItems(report, options)) {
+    const done = new Set(audit.entries.map((entry) => entry.id));
+    for (const item of planItems(report, options).filter(
+      (item) => !done.has(item.id)
+    )) {
       if (options.signal?.aborted) {
         throw new Error('cleanup aborted');
       }

package/src/cli.js

@@ -15,9 +15,11 @@ import { startAudit, writeAudit } from './audit.js';
 import { clean } from './clean.js';
 import { emergency } from './emergency.js';
 import { setTrace } from './exec.js';
-import { resolveGoal } from './options.js';
+import { resolveGoal, resolveOptions } from './options.js';
 import { formatAudit, formatReport } from './report.js';
 import { scan } from './scan.js';
+import { cleanDuringScan } from './fast-clean.js';
+import { defaultStateDirectory, loadScan } from './scan-state.js';
 import { formatBytes } from './units.js';
 
 export const USAGE = `Usage: dss <command> [paths...] [options]
@@ -67,6 +69,9 @@ Consent:
 Output:
   --json                     print the report or audit log as JSON
   --report FILE              clean: use a report saved by scan --json
+  --from FILE                clean/emergency: reuse a scan or audit JSON
+  --fresh                    bypass the recent scan and persistent index
+  --cache-dir DIR            scan state (default: XDG state directory)
   --audit-dir DIR            audit logs ($DSS_AUDIT_DIR)
   --backup-dir DIR           container log backups ($DSS_BACKUP_DIR)
   --verbose                  list everything and trace commands (DSS_DEBUG=1)
@@ -106,6 +111,9 @@ const OPTIONS = {
   'allow-dirty-container': { type: 'string', multiple: true },
   'no-native': { type: 'boolean' },
   report: { type: 'string' },
+  from: { type: 'string' },
+  fresh: { type: 'boolean' },
+  'cache-dir': { type: 'string' },
   'audit-dir': { type: 'string' },
   'backup-dir': { type: 'string' },
   help: { type: 'boolean', short: 'h' },
@@ -182,6 +190,9 @@ export function toOptions(values, paths = []) {
     noNative: values['no-native'],
     auditDir: values['audit-dir'],
     backupDir: values['backup-dir'],
+    cacheDir: values.fresh
+      ? false
+      : (values['cache-dir'] ?? defaultStateDirectory()),
   };
 }
 
@@ -240,14 +251,7 @@ function dockerOnly(options, values) {
 export function scanAudit(report) {
   return startAudit('scan', {
     dryRun: true,
-    report: {
-      createdAt: report.createdAt,
-      durationMs: report.durationMs,
-      totals: report.totals,
-      environments: report.environments.map((env) => env.id),
-      errors: report.errors,
-      items: report.items,
-    },
+    report,
     entries: report.items.map((item) => ({
       id: item.id,
       env: item.env,
@@ -283,7 +287,7 @@ export function processIo() {
   };
 }
 
-const API = { scan, clean, emergency };
+const API = { scan, clean, emergency, cleanDuringScan };
 
 class Cli {
   constructor(io, api) {
@@ -321,10 +325,19 @@ class Cli {
   }
 
   async loadReport(options, values) {
-    if (!values.report) {
+    const file = values.from ?? values.report;
+    if (!file) {
+      const recent = await loadScan(resolveOptions(options));
+      if (recent) {
+        this.io.stderr(
+          `Reusing scan from ${recent.createdAt}; planned items will be checked before deletion.`
+        );
+        return recent;
+      }
       return this.api.scan(options);
     }
-    return JSON.parse(await fsp.readFile(values.report, 'utf8'));
+    const saved = JSON.parse(await fsp.readFile(file, 'utf8'));
+    return saved.report?.schema ? saved.report : saved;
   }
 
   confirmItem() {
@@ -371,6 +384,33 @@ class Cli {
   }
 
   async clean(options, values) {
+    if (
+      this.api.cleanDuringScan &&
+      !values.report &&
+      !values.from &&
+      !this.io.interactive
+    ) {
+      const recent = await loadScan(resolveOptions(options));
+      if (!recent) {
+        const result = await this.consent(values, (extra) =>
+          this.api.cleanDuringScan({
+            ...options,
+            ...extra,
+            streaming: options.docker !== false,
+          })
+        );
+        if (result.report.aborted) {
+          return this.printScan(result.report, options, values);
+        }
+        this.print(values, result.audit, () =>
+          formatAudit(result.audit, { verbose: values.verbose })
+        );
+        return result.audit.aborted ||
+          result.audit.entries.some((entry) => entry.status === 'failed')
+          ? 1
+          : 0;
+      }
+    }
     const report = await this.loadReport(options, values);
     if (report.aborted) {
       return this.printScan(report, options, values);
@@ -393,6 +433,32 @@ class Cli {
     } catch (error) {
       throw new UsageError(error.message);
     }
+    if (
+      this.api === API &&
+      !values.report &&
+      !values.from &&
+      !this.io.interactive
+    ) {
+      const recent = await loadScan(resolveOptions(options));
+      const audit = await this.consent(values, (extra) =>
+        this.api.emergency({
+          ...options,
+          ...(recent ? { report: recent } : {}),
+          ...extra,
+        })
+      );
+      if (audit.scanReport?.aborted) {
+        return this.printScan(audit.scanReport, options, values);
+      }
+      this.print(values, audit, () =>
+        formatAudit(audit, { verbose: values.verbose })
+      );
+      return audit.entries.some((entry) => entry.status === 'failed')
+        ? 1
+        : audit.goalMet
+          ? 0
+          : 3;
+    }
     const report = await this.loadReport(options, values);
     if (report.aborted) {
       return this.printScan(report, options, values);
@@ -461,7 +527,7 @@ export async function runCli(argv, deps = {}) {
     }
     options.onScanProgress = (progress) =>
       io.stderr(
-        `[scan] ${progress.env}: ${progress.scanner}, ${Math.round(progress.elapsedMs / 1000)}s, ${progress.items} items (${progress.status})`
+        `[scan] ${progress.env}: ${progress.scanner}, ${Math.round(progress.elapsedMs / 1000)}s, ${progress.candidates ?? progress.items} candidates, ${progress.filesSized ?? 0} files sized, ${progress.directoriesSized ?? 0} directories sized (${progress.status})`
       );
     return await dispatch(
       new Cli(io, deps.api ?? API),

package/src/emergency.js

@@ -89,12 +89,40 @@ export async function emergency(input = {}) {
   });
   const report = input.report
     ? await revalidateReport(input.report, { ...input, env })
-    : await scan({ ...input, env });
-  audit.report = { createdAt: report.createdAt, totals: report.totals };
+    : goalMet(goal, diskBefore)
+      ? {
+          createdAt: new Date(options.now()).toISOString(),
+          items: [],
+          environments: [{ id: env.id, label: env.label }],
+          totals: {},
+        }
+      : await scan({
+          ...input,
+          env,
+          onScanPhase: async (phase) => {
+            audit.diskAfter = await escalate({
+              report: phase,
+              options: {
+                ...options,
+                tier: 'safe',
+                streaming: options.docker !== false,
+              },
+              env,
+              target,
+              goal,
+              audit,
+            });
+            return !goalMet(goal, audit.diskAfter);
+          },
+        });
+  audit.report = report;
   if (options.audit !== false) {
     await writeAudit(audit, { ...options, inProgress: true });
   }
-  let disk = diskBefore;
+  let disk = audit.diskAfter ?? diskBefore;
+  if (report.aborted) {
+    audit.scanReport = report;
+  }
   try {
     if (!goalMet(goal, disk)) {
       disk = await escalate({ report, options, env, target, goal, audit });
@@ -121,9 +149,10 @@ async function escalate({ report, options, env, target, goal, audit }) {
     ...options,
     env,
     onProgress: persistProgress,
+    priorHealth: audit.health,
   });
   audit.health = cleaner.health.records;
-  const done = new Set();
+  const done = new Set(audit.entries.map((entry) => entry.id));
   const targetDevice = await env.deviceId?.(target);
   const scoped = await Promise.all(
     wantedItems(report, options).map(async (item) => {
@@ -141,8 +170,10 @@ async function escalate({ report, options, env, target, goal, audit }) {
     })
   );
   const wanted = scoped.filter(Boolean);
-  let disk = audit.diskBefore;
-  let freed = 0;
+  let disk = audit.diskAfter ?? audit.diskBefore;
+  let freed = audit.entries
+    .filter((entry) => entry.status === 'planned')
+    .reduce((sum, entry) => sum + entry.plannedBytes, 0);
   for (const tier of TIERS.slice(0, tierRank(options.tier) + 1)) {
     audit.reachedTier = tier;
     const items = selectByTier(wanted, tier)