d4c-pool 0.2.0

package/README.md

@@ -0,0 +1,116 @@
+# d4c
+
+**Dependency Deduplication Done Dirt Cheap.**
+
+Your `node_modules` are scattered across dozens of copies. Pool them.
+
+```bash
+npx d4c-pool dedupe      # make identical files share storage — deletes nothing
+npx d4c-pool gc          # delete trees nobody has touched in a month
+npx d4c-pool history     # what was removed, and how to restore it
+```
+
+Both are dry-run by default.
+
+## Why
+
+On the machine this was built for:
+
+- 976 `node_modules` trees, holding more than the disk had free
+- `typescript/lib/typescript.js` existed in **89 identical copies**
+- one Next.js SWC binary appeared **38 times**, 4.3 GB of the same bytes
+
+Nothing had gone wrong. That is just what happens after a year of
+worktrees, benchmark runs, and abandoned branches.
+
+## `dedupe` — reclaim without losing anything
+
+Finds files that are byte-identical across trees and makes them share
+storage through APFS clones. Every copy stays where it is and keeps
+working; they simply stop occupying separate blocks until one is written
+to, at which point that one diverges on its own.
+
+Matching is by content, not by lockfile. Twenty projects pinning
+`react@18.3.1` have twenty different lockfiles and one identical
+`react/index.js`. Measured on a real tree, content matching found **30.0
+GB** where lockfile grouping found **10.9 GB**.
+
+Because nothing is deleted, `dedupe` does not require a lockfile — which
+makes it the only option for trees `gc` will not touch.
+
+## `gc` — delete what can be rebuilt
+
+Removes trees that have seen no activity past a threshold. Refuses
+anything it cannot restore.
+
+## What neither will touch
+
+- Trees with recent activity, measured from `node_modules` mtime and git
+  index mtime, whichever is more recent
+- Files any process currently has open (checked with `lsof`; if `lsof`
+  cannot run, `dedupe` does nothing rather than guess)
+- Symlinked `node_modules` — the real tree lives elsewhere
+- Hardlinked files, where replacing one link frees nothing
+- Files whose permissions differ, where replacing would drop the exec bit
+- Anything crossing a mount boundary, or matched by `--exclude`
+- For `gc` only: trees with no lockfile able to rebuild them. A parent
+  lockfile counts only if that directory declares the tree as a workspace
+  member
+
+Every check is re-run immediately before the destructive step, never
+trusted from the earlier scan.
+
+## Options
+
+```
+d4c dedupe [options]
+
+  --days N          Idle threshold in days (default 30, minimum 1)
+  --min-file SIZE   Ignore files smaller than this (default 64K)
+  --skip-open-check Proceed without checking for open files
+  --exclude PATTERN Protect matching paths. Repeatable, full match
+  --yes             Apply. Without it nothing changes
+  --json            Machine-readable output on stdout only
+
+d4c gc [options]
+
+  --days N          Idle threshold in days (default 30, minimum 1)
+  --min-size SIZE   Ignore trees smaller than this (e.g. 10M, 1G)
+  --exclude PATTERN Protect matching paths. Repeatable, full match
+  --depth N         How deep to search (default 8)
+  --yes             Actually delete
+  --json            Machine-readable output on stdout only
+
+d4c history [--limit N] [--json]
+```
+
+## Read the numbers as a lower bound
+
+Idle age comes from mtime, which records the last install or git
+operation — not the last read. **A project you still use can look idle,
+and a long-running process leaves no trace at all.** Check the list before
+passing `--yes`.
+
+Sizes come from `st_blocks`, tracking what deletion actually frees, within
+about 3% of measured freed space. Files already sharing blocks through
+APFS clones are over-reported, since that sharing is invisible to
+`st_blocks`.
+
+## Recovery
+
+`gc` appends every deletion to `~/.local/state/d4c/deletions.jsonl` as it
+happens, so an interrupted run is still fully recorded. `d4c history`
+prints each project with the install command that rebuilds it, matched to
+the lockfile that was there.
+
+`dedupe` records its replacements to the same log. It has nothing to
+recover from — the files are still there.
+
+## Requirements
+
+macOS with APFS for `dedupe`, which uses `clonefile(2)` through FFI. `gc`
+runs anywhere. Node 20 or newer.
+
+## License
+
+MIT