@danielsimonjr/repo-tools 2.2.7 → 2.4.0

package/CHANGELOG.md

@@ -6,6 +6,103 @@ All notable changes to this project are recorded in this file. The format follow
 
 ## [Unreleased]
 
+## [2.4.0] - 2026-11-03
+
+### Added
+
+- The five hand-written architecture documents `OVERVIEW.md`, `ARCHITECTURE.md`, `COMPONENTS.md`,
+  `DATAFLOW.md` and `API.md` in `docs/architecture/`. Each ends with a Verification table that
+  `repo-tools check` reads. The root `README.md` has a Documentation section that links all ten
+  architecture documents and the design document.
+- The config key `dispatchCalls` (in section `depgraph` or `map`) lists the calls that register a typed-dispatch
+  constant. The duplicate report tagged only the call `mathTyped` as `DISPATCH_VARIANT`. Repositories with other
+  registration calls received a `TRUE_DUPLICATE` for each registered name. The default list is
+  `["mathTyped"]`, and with the default the reports are byte for byte the reports of 2.3.0. An empty list switches
+  the tag off. `repo-tools map` and `repo-tools check` both read the key, so the measured counts agree. The note in
+  `duplicate-symbols.json` and the text in `duplicate-symbols.md` show the names of the listed calls.
+  `docs/design.md` section 5.1 states the rules.
+- `repo-tools ste --require=<path>` states that the gate must cover a file. The option repeats. The run exits 2 when
+  the file does not exist, or when the run did not examine it. A document that someone deletes inside a folder that
+  still exists gave no finding before.
+- `repo-tools ste --since=<rev>` keeps only the findings on the lines that the working tree adds or changes since the
+  git revision `<rev>`. A long document with old findings can now gate a new edit. A sentence-length finding counts
+  when any line of its paragraph changed. The totals line gives the number of hidden findings. `docs/design.md`
+  section 9.1 states the rules. A run without the two options gives the same output as before.
+- The summary of `repo-tools clones` lists the files with a parse error, one path on each line, in code-unit order.
+  The summary gave only the number before, so a reader could not act on it. The new setting `parseErrorList` (flag
+  `--list-errors`, default 20) sets the most paths in the list. `clones.json` does not hold the setting. A run
+  without parse errors gives the same output as before. `docs/design.md` section 17.7 states the format.
+- `docs/parity-clones.md` has a second repository, deepthinking-mcp (243 files). 1,172 units are the same, and the
+  command finds all 12 exact groups of the prototype. The 29 units that only one side reports lie at the cutoff of
+  50 labels, except two functions that are mostly template strings. Tree-sitter makes a node for each escape
+  sequence of a template string, and the prototype makes one node for each template element.
+
+### Changed
+
+- The five duplicate functions that `repo-tools clones` found in this repository are merged. `realPath` is in
+  `src/io.ts` (it was in `src/chunk/index.ts` and `src/depgraph/extensions.ts`). `rglobParts` is in
+  `src/map/discovery.ts`, and `csprojParts` calls it. `summaryRowsOf` takes the two labels that differ, so the
+  2.0.0 Markdown no longer has a second copy of the row list. The two type-only checks of `src/map/parsing.ts` share
+  one helper. The output of every command is unchanged. The source has 41 lines fewer. `clones` now finds 0
+  exact groups (it found 2) and 1 near pair (it found 8).
+- The repository now has its own `docs/architecture/` folder. `repo-tools map . --out=docs/architecture` writes it:
+  the dependency graph, the file inventory, the duplicate and unused reports, and the test coverage. `repo-tools check`
+  passes against it. Biome skips this folder, because the generator owns the format of its files.
+- The privacy check (`scripts/privacy-check.ts`) keeps the hashes of the candidates of each word, so it hashes a word
+  once. The scan of this repository took about 4.5 s on a Windows machine and takes about 1.5 s now, and the findings
+  are the same. The test "this repository scans clean" timed out at the 5 s default of `bun test` on that machine.
+  A memo of at most 20,000 words keeps the memory bounded.
+- Every report of `repo-tools map` carries the `name` field of `package.json` as the project name. A report uses
+  the folder name only when `package.json` has no name, or when the name is empty or not a string.
+  `dependency-graph.json` and `dependency-graph.yaml` wrote the folder name before, so two checkouts of one repository
+  gave two graphs. A committed `dependency-graph.json` changes once in a repository whose folder name differs from
+  its package name. `tests/unit/map-project-name.test.ts` pins the rule, and `docs/design.md` section 14.4 states it.
+- The two file lists of `TEST_COVERAGE.md` have the headings `Source Files Not Imported By A Test` and
+  `Source Files Imported By A Test`. The headings said `Without Test Coverage` and `With Test Coverage`. The report
+  counts a file when a test file imports it. A file that only an end-to-end spec runs was therefore on the first list
+  under a heading that said it had no coverage. The text for an empty first list changed in the same way.
+  `docs/design.md` section 14.2 states the basis.
+- The README, `docs/design.md` (sections 17 and 17.8) and `docs/parity-clones.md` state the scope of `repo-tools clones`.
+  The command serves the owner of one repository during a refactoring. A score ranks the pairs of one run. A count or
+  a score does not compare between repositories, settings or tool versions. The two parsers of the parity record
+  already give different pair counts on the same input. The parity record is a regression record for the choice
+  of parser. No code changed.
+- `docs/design.md` section 17.8 states that a decorator before a top-level function (the AssemblyScript `@inline`) is a
+  parse error in tree-sitter-typescript. The summary lists the file, and the functions of the file still give units.
+  The AssemblyScript sources of MathTS have one such file in 26. All 16 functions of that file give units with
+  `--min-nodes=1`. No code changed.
+
+### Fixed
+
+- `repo-tools map` writes `typeOnly: true` for the edge of `export type * from 'm'` and `export type * as ns from
+  'm'`. The grammar (`tree-sitter-typescript` 0.23.2) does not know this syntax and puts the word `type` inside an
+  error node, so the edge held `typeOnly: false`. A cycle through such an edge counted as a runtime cycle. A
+  committed `dependency-graph.json` changes in a repository that uses the syntax. `docs/design.md` section 14.4
+  states the rule.
+- `repo-tools ste` does not measure a list of identifiers against the sentence limit. A list item or a sentence
+  whose only words are `and`, `or` and inline code is a list of names, not a sentence. A generated report of
+  `repo-tools map` gave such a bullet of 32 words (limit 25), so `repo-tools ste` failed on its own output. A sentence
+  with one prose word keeps the limit, and so does prose with many code spans. `docs/design.md` section 9 states the
+  difference from the Python checker.
+
+## [2.3.0] - 2026-10-22
+
+### Added
+
+- `repo-tools clones` lists the functions of a TypeScript or JavaScript repository and finds the functions that copy
+  each other. It reads the source and writes files only when `--out` or the `clones.out` config key names a folder.
+  A unit is a named function with at least 50 labels (`--min-nodes`). The command reports exact groups (the same label
+  sequence) and near pairs (shared runs of labels, scored by Jaccard and containment). It prints a summary, and
+  with an output folder it writes `inventory.jsonl` and `clones.json`. Every limit is a flag and a key of the new
+  `clones` section of `repo-tools.config.json`. A flag overrides the file, and the file overrides the default.
+  Exit code 0 means the run finished, 1 means the root holds no source file to read, and 2 means a usage or config
+  error. `docs/design.md` section 17 gives the rules.
+- `docs/parity-clones.md` compares the command with the prototype of the memoryjs duplicate-function audit on 292
+  files. 1,368 units are the same. The command finds all 20 exact groups of the prototype. Every unit that only one
+  side reports lies at the cutoff of 50 labels.
+- The smoke test runs `clones` on a small repository that holds a planted copy, so the shipped bundle proves the
+  command and its tree-sitter loader.
+
 ## [2.2.7] - 2026-10-22
 
 ### Fixed

package/README.md

@@ -11,13 +11,16 @@
 | `chunk` | Splits a large file into chunks, merges the chunks back, and shows which chunks changed. |
 | `compress` | Writes a compact copy of a file for a model context, and restores it. |
 | `query` | Answers structural questions from the graph of `map`, and writes two derived reports. |
+| `clones` | Lists the functions of a TypeScript or JavaScript repository, and finds the functions that copy each other. |
 | `ste` | Checks Markdown or docstring prose against Simplified Technical English. |
 
-Status: version 2.2.7.
+Status: version 2.4.0.
 
 - `map`, `check` and `query` read their settings from `repo-tools.config.json` in the root. The
   section of `map` is `map` (its old name, `depgraph`, is also read). `--config=<file>` makes any
   of the three read another file.
+- `clones` reads its settings from the `clones` section of the same file. A flag overrides the
+  file, and the file overrides the default.
 - `repo-tools depgraph` prints one deprecation line and runs `map`. It accepts the 1.x
   scan-scope flags (`--src`, `--tests`, `--exclude`, `--also-exclude`, `--all`,
   `--reachable-only`, `--include-tests`) with a warning, because they have no effect in 2.0.0.
@@ -41,7 +44,7 @@ repo-tools --version
 
 The executable holds the tool and the Bun runtime. It needs no Bun, Node or `node_modules`.
 Download the file for your platform and `SHA256SUMS` from the
-[GitHub release](https://github.com/danielsimonjr/repo-tools/releases/tag/v2.2.7):
+[GitHub release](https://github.com/danielsimonjr/repo-tools/releases/tag/v2.4.0):
 
 | Platform | File |
 |---|---|
@@ -311,6 +314,43 @@ repo-tools query --emit
 the config key `query.out`, else from `map.out`, else from `docs/architecture`. Run
 `repo-tools query --help` for every option.
 
+## Find the functions that copy each other
+
+`repo-tools clones` lists the functions of a TypeScript or JavaScript repository. It finds the
+functions that copy each other. The command only reads the repository. It writes files only when
+you name an output folder.
+
+```sh
+repo-tools clones .                              # prints the summary and writes no file
+repo-tools clones . --src=src --out=reports/clones
+repo-tools clones . --min-nodes=30 --near=0.7
+```
+
+A function is a unit when it has a name and its body has at least 50 labels. A label is one node
+of the body. Names, literals, types and comments give no detail, so a copy with other names still
+matches. The command finds two kinds of result:
+
+| Result | Meaning |
+|---|---|
+| Exact group | Units with the same label sequence. |
+| Near pair | Two units that share enough runs of 6 labels. |
+
+`--src=<path>...` names the folders to read. The default `auto` reads the files of the `src` area.
+`--out=<dir>` writes `inventory.jsonl` (one line for each unit) and `clones.json` (the groups and the
+pairs) into a folder. Give the folder as a path relative to the root. Run `repo-tools clones --help` for every limit.
+
+Exit codes: 0 when the run finished, even if it found no copy. 1 when the root holds no source file
+to read. 2 on a usage error, a bad config file or a root that is not a folder.
+
+The result is a list of candidates. Read both bodies of a pair before you change anything.
+
+The command serves the owner of one repository during a refactoring. A score ranks the pairs of one run. Do not
+compare counts or scores between repositories, settings or tool versions.
+
+`docs/design.md` (section 17) gives the rules. `docs/parity-clones.md` records the comparison with the
+prototype of the memoryjs audit, on memoryjs and on deepthinking-mcp. The record shows that the two parsers
+agree. The record makes no claim about other repositories.
+
 ## Develop
 
 | Command | Job |
@@ -326,6 +366,24 @@ the config key `query.out`, else from `map.out`, else from `docs/architecture`.
 | `bun run audit:ledger` | Reverts each fix of `docs/fix-ledger-2.0.0.md` in a scratch copy and checks that a test of that fix fails. |
 | `bun run check:design` | Finds each reference to a section of `docs/design.md` that has no heading with that number. |
 
+## Documentation
+
+| Document | Contents |
+|---|---|
+| [docs/architecture/OVERVIEW.md](docs/architecture/OVERVIEW.md) | The project, its metrics and its layout. |
+| [docs/architecture/ARCHITECTURE.md](docs/architecture/ARCHITECTURE.md) | The layers, the key decisions, the build and the test strategy. |
+| [docs/architecture/COMPONENTS.md](docs/architecture/COMPONENTS.md) | Each module, with the real signatures of its exports. |
+| [docs/architecture/DATAFLOW.md](docs/architecture/DATAFLOW.md) | One run of each subcommand, step by step. |
+| [docs/architecture/API.md](docs/architecture/API.md) | The commands, the flags, the config keys, the exit codes and the report files. |
+| [docs/architecture/FILE_INVENTORY.md](docs/architecture/FILE_INVENTORY.md) | Every source file with its area and disposition. Generated. |
+| [docs/architecture/TEST_COVERAGE.md](docs/architecture/TEST_COVERAGE.md) | The source files that a test imports. Generated. |
+| [docs/architecture/DEPENDENCY_GRAPH.md](docs/architecture/DEPENDENCY_GRAPH.md) | Who imports whom. Generated. |
+| [docs/architecture/unused-analysis.md](docs/architecture/unused-analysis.md) | Files and exports with no importer. Generated. |
+| [docs/architecture/duplicate-symbols.md](docs/architecture/duplicate-symbols.md) | Names that more than one file defines. Generated. |
+| [docs/design.md](docs/design.md) | The design of each subcommand, section by section. |
+
+Run `repo-tools check . --docs=docs/architecture` to compare the numbers in these documents with the code.
+
 ## Design
 
 `docs/design.md` describes the modules, the output rules, the config, the extension contract,