@polycode-projects/the-mechanical-code-talker 0.5.0 → 0.6.0

package/README.md

@@ -2,12 +2,11 @@
 
 `@polycode-projects/the-mechanical-code-talker`
 
-A pure-JS, **no-LLM**, offline, **$0** chatbot in the ELIZA/PARRY lineage —
+A pure-JS, **no-LLM**, offline, **$0** chatbot in the ELIZA/PARRY lineage:
 pattern-driven, best-efforts, and obsessed with software the way PARRY was
-obsessed with the mafia. No model calls anywhere in the product: interpretation
-is mechanical (deterministic language libraries, template sets, committed
-corpuses), memory is a graph on disk, and every answer is either grounded or an
-honest miss.
+obsessed with the mafia. No model calls anywhere. Interpretation is mechanical
+(deterministic language libraries, template sets, committed corpuses). Memory is
+a graph on disk. Every answer is either grounded or an honest miss.
 
 ```
 $ tmct
@@ -125,6 +124,41 @@ or a bare user gets a working install in one command.
 > Install-size note: tmct depends on wink-nlp's deterministic English language
 > model (~3.8 MB installed). That model is a lookup table, not an LLM.
 
+### Try it on an example graph
+
+tmct *consumes* a code graph at `<repo>/.tmct/graph.json` — it does not build
+one. Two ready-made example graphs ship in `examples/` so you can see it answer
+real questions with no setup:
+
+```bash
+npm run example:mini       # "Questboard" — a small task-tracker web app (12 modules)
+npm run example:polyglot   # one shared OWL vocabulary across Java / Python / C#
+npm run chat:repo -- ./any/path   # chat over any repo that has a .tmct/graph.json
+```
+
+Questions the **mini-webapp** graph answers:
+
+```
+what classes are there
+describe Task
+how many modules
+which modules import src/core/model.mjs
+what tests cover src/handlers/tasks.mjs
+```
+
+The **polyglot** graph shows the language-neutral idea — Java, Python and C#
+entities all typed to the same `seon:Class` / `seon:Method` / `seon:Module`
+concepts, so one query reasons across every language at once:
+
+```
+how many classes          # 9 — Java + Python + C# counted as one concept
+what classes are there     # Order (Java), Inventory (Python), PaymentService (C#), …
+which modules define PaymentService
+```
+
+See `examples/mini-webapp/README.md` and `examples/polyglot/README.md` for the
+full tours.
+
 ### As a library
 
 ```js
@@ -138,29 +172,31 @@ extension surface.
 ## The repository interface
 
 tmct is not an indexer, so it consumes a graph through a typed contract any
-producer can implement. That contract is now first-class: a **versioned
-(1.0.0), OWL-grounded, machine-readable service definition**
-(`docs/repository-interface.md` + a JSON schema) of every service, its
-arguments, result types, and error contract. A **miss is a value, not a throw** —
-the interface models "no answer" explicitly. tmct ships **reference providers**
-(a fixture graph and the empty/bootstrap graph) that implement every service,
-and a **runnable conformance suite**: tmct's own providers pass it in `npm test`,
-and any external graph producer (seonix first) runs the *same* suite against its
-native implementation to claim conformance. Conformance is the suite, not prose.
-The design deliberately inverts the original relationship — tmct was lifted out
-of seonix; now seonix reorients as a *user* that imports the tmct library and
-exposes its graph to tmct as a service, with the LLM agent staying outside tmct
-exactly as the no-LLM ethos requires.
+producer can implement. That contract is first-class: a **versioned (1.0.0),
+OWL-grounded, machine-readable service definition** (`docs/repository-interface.md`
+plus a JSON schema) of every service, its arguments, result types, and error
+contract. A **miss is a value, not a throw**: the interface models "no answer"
+explicitly. tmct ships **reference providers** (a fixture graph and the
+empty/bootstrap graph) that implement every service, and a **runnable conformance
+suite**. tmct's own providers pass it in `npm test`. Any external graph producer
+(seonix first) runs the same suite against its native implementation to claim
+conformance. Conformance is the suite, not prose. This inverts the original
+relationship: tmct was lifted out of seonix, and seonix now reorients as a *user*
+that imports the tmct library and exposes its graph to tmct as a service. The LLM
+agent stays outside tmct, as the no-LLM ethos requires.
 
 ## Security and supply chain
 
 tmct is $0 to run and meant to be trusted offline, so the supply chain is
-hardened accordingly: CI runs **SAST and secret detection**, a **nightly
-`npm audit` + OSV-Scanner** job watches dependencies, releases are published with
-**npm provenance** (`--provenance`), and there is a coordinated-disclosure
-`SECURITY.md` policy. The content-address hash is single-sourced in
-`src/hash.mjs` so the cross-version-stable fact-id contract has exactly one
-definition.
+hardened:
+
+- CI runs **SAST and secret detection**.
+- A nightly **`npm audit` + OSV-Scanner** job watches dependencies.
+- Releases are published with **npm provenance** (`--provenance`).
+- A coordinated-disclosure `SECURITY.md` policy covers reports.
+
+The content-address hash is single-sourced in `src/hash.mjs`, so the
+cross-version-stable fact-id contract has exactly one definition.
 
 ## Provenance
 

package/bin/tmct.mjs

@@ -40,6 +40,10 @@ Usage:
        [--verbose]             folded blocks (the /memory chat command, from the shell)
   tmct init [--force]          initialize the current directory for tmct: .tmct/,
                                tmct.toml, tier-1 corpus seed, provenance record
+       [--corpus <id>]         also seed a tier-2 corpus (aws|python|java) — opt-in,
+                               offline, $0; init is tier-1-only unless asked
+       [--detect]              suggest a tier-2 corpus from the repo's manifests
+                               (pyproject.toml → python, pom.xml → java); never seeds unasked
   tmct syllogise [--repo <abs>] speculative inference (offline maintenance job): forward-
        [--depth <n>] [--budget <n>]  chain the memory's rdfs:subClassOf closure, materialising
                                bounded, low-trust, retractable entailed facts (never on the chat path)
@@ -295,10 +299,62 @@ async function main() {
     // `tmct init` — the Repository-Interface onboarding surface: scaffold .tmct/,
     // write tmct.toml, seed the tier-1 corpus (offline, opt-out via TMCT_NO_SEED),
     // and record provenance. Idempotent; --force rewrites config + re-records.
+    //
+    // TIERING POLICY: init is OFFLINE, $0 and TIER-1-ONLY by default. A tier-2
+    // domain/language corpus (corpus/tier2/: aws, python, java) is added ONLY when
+    // explicitly asked via `--corpus <id>`. The `--detect` auto-detect is a
+    // documented STUB: it inspects the repo's manifests (pyproject.toml → python,
+    // pom.xml → java) and SUGGESTS the matching corpus, but never seeds it unasked.
     const rest = process.argv.slice(3);
     const { initRepo } = await import("../src/init.mjs");
     const res = await initRepo(process.cwd(), { force: rest.includes("--force") });
     process.stdout.write(res.message + "\n");
+
+    const ci = rest.indexOf("--corpus");
+    const corpusId = ci !== -1 ? rest[ci + 1] : undefined;
+    if (corpusId) {
+      // Seed a tier-2 corpus by id from corpus/tier2/ (same slice shape as tier-1;
+      // provenance-tagged corpus:tier2:<id>). Idempotent (content-hashed fact ids).
+      const { readFile } = await import("node:fs/promises");
+      const { join, dirname } = await import("node:path");
+      const { seedMemory, TIER2_MANIFEST_FILE } = await import("../src/corpus/conceptnet.mjs");
+      let manifest;
+      try { manifest = JSON.parse(await readFile(TIER2_MANIFEST_FILE, "utf8")); }
+      catch (e) { process.stderr.write(`tmct init: cannot read the tier-2 manifest — ${e?.message || e}\n`); process.exit(1); }
+      const entry = (manifest.corpuses || []).find((c) => c.id === corpusId);
+      if (!entry) {
+        const ids = (manifest.corpuses || []).map((c) => c.id).join(", ");
+        process.stderr.write(`tmct init: unknown --corpus "${corpusId}". Available tier-2 corpuses: ${ids}.\n`);
+        process.exit(2);
+      }
+      const slicePath = join(dirname(TIER2_MANIFEST_FILE), entry.file);
+      const seeded = await seedMemory(process.cwd(), { slicePath, provenancePrefix: `corpus:tier2:${entry.id}` });
+      process.stdout.write(
+        `seeded tier-2 corpus "${entry.id}" (${entry.kind}) — ${seeded.appended} fact(s) added`
+        + `${seeded.skipped ? `, ${seeded.skipped} already present` : ""}. Source: corpus/tier2/${entry.file} (${entry.license}).\n`,
+      );
+      return;
+    }
+
+    if (rest.includes("--detect")) {
+      // AUTO-DETECT STUB (documented, non-seeding): map a build manifest to the
+      // tier-2 corpus that fits, and tell the operator how to add it. Kept a stub on
+      // purpose — the $0/offline default never expands the corpus without an ask.
+      const { access } = await import("node:fs/promises");
+      const { join } = await import("node:path");
+      const has = (f) => access(join(process.cwd(), f)).then(() => true, () => false);
+      const DETECT = [["pyproject.toml", "python"], ["pom.xml", "java"]];
+      const found = [];
+      for (const [file, id] of DETECT) if (await has(file)) found.push([file, id]);
+      if (!found.length) {
+        process.stdout.write("no tier-2 corpus auto-detected (looked for pyproject.toml → python, pom.xml → java).\n");
+      } else {
+        for (const [file, id] of found) {
+          process.stdout.write(`detected ${file} — run \`tmct init --corpus ${id}\` to add the ${id} tier-2 corpus (offline, $0).\n`);
+        }
+      }
+      return;
+    }
     return;
   }
 

package/corpus/README.md

@@ -5,14 +5,40 @@ The corpuses tmct ships so that an **empty** tmct still has a vocabulary
 in `src/corpus/`. Related committed data lives in `data/` (response templates
 + the SE phrase book — items 4+7).
 
+## The tiering policy (tier-1 / tier-2 / tier-3)
+
+tmct's knowledge arrives in three tiers, distinguished by **when** it lands and
+**whether it ships in the npm package**:
+
+| Tier | What | Ships in the package? | Lands when | Provenance |
+|---|---|---|---|---|
+| **1 — base** | the general English/tech ConceptNet slice + the response templates + the SE phrasebook — the vocabulary every tmct has out of the box | **yes**, committed here | `tmct init` seeds `.tmct/` from committed data (offline, $0) | `corpus:conceptnet /r/…` |
+| **2 — specialised** | LANGUAGE- or DOMAIN-specific fact sets (`aws`, `python`, `java`, …) so tmct can "expand into a concept for an applicable codebase" | **no** — selected per repo | `tmct init` fetches/generates the chosen tier-2 corpuses into `.tmct/` (Wave-2 wiring) | `corpus:tier2:<id> /r/…` |
+| **3 — learned** | facts tmct writes from the actual conversation / the actual codebase it is pointed at | never committed | at runtime, into `.tmct/memory/` | `chat:…`, `codegraph:…` |
+
+**Offline / $0 is the default at every tier.** Tier-1 is committed. Tier-2's
+sample corpuses are *curated* (generated locally by `tier2/generate.mjs`, no
+network); a tier-2 corpus too large to curate by hand may declare a `fetch`
+source (a URL + a `sha256`), but the network is **opt-in only** — nothing
+reaches out unless the operator asks for it. Tier-3 is whatever the user says.
+
+**Checksums / integrity.** Every tier-2 corpus carries a `sha256` and a `bytes`
+count in `tier2/manifest.json`; a `fetch`-sourced corpus is checksum-verified on
+download (`generate.mjs` `fetchCorpus()`), so a corrupt or tampered fetch fails
+loudly instead of seeding garbage.
+
 ## What's here
 
 | Path | What | Size | Licence |
 |---|---|---|---|
-| `conceptnet/slice.jsonl` | filtered English/tech-domain ConceptNet 5.7 slice (one assertion per line) | ~1.4 MB | **CC-BY-SA 4.0** (see `conceptnet/LICENSE-NOTICE`) |
+| `conceptnet/slice.jsonl` | tier-1: filtered English/tech-domain ConceptNet 5.7 slice, quality-filtered (one assertion per line) | ~1.35 MB | **CC-BY-SA 4.0** (see `conceptnet/LICENSE-NOTICE`) |
 | `conceptnet/fetch-slice.mjs` | regeneration tool — the ConceptNet **API** route (polite, ~1 req/s) | — | MPL-2.0 |
 | `conceptnet/filter-dump.mjs` | regeneration tool — the ConceptNet **dump** route (produced the committed slice; the API was down) | — | MPL-2.0 |
+| `conceptnet/quality-filter.mjs` | second-pass noise filter (drops sentence-fragment/numeric/opinion rows); produced the committed clean slice | — | MPL-2.0 |
 | `conceptnet/README.md` | provenance, retrieval date, seed terms, filter rules, row counts | — | — |
+| `tier2/manifest.json` | tier-2: index of specialised corpuses (id, kind, description, source, sha256, size) | — | MPL-2.0 |
+| `tier2/generate.mjs` | tier-2: the curated-corpus generator + manifest writer (+ opt-in network-fetch path) | — | MPL-2.0 |
+| `tier2/{aws,python,java}.jsonl` | tier-2 SAMPLE corpuses — same fact shape as the tier-1 slice, loadable via the same path | ~4 KB each | MPL-2.0 |
 
 And alongside (same phase, different directory because it is tmct-original
 data, not a derived corpus):
@@ -44,9 +70,55 @@ await seedMemory(repoDir, { limit: 500 }); // capped (fast bootstrap)
 - `ace = "none"` relations (RelatedTo, Synonym, FormOf, …) are deliberately
   NOT seeded — they are kept in the slice for future lexicon/fuzzy-match use.
 
+## Tier-2 specialised corpuses (`tier2/`)
+
+A tier-2 corpus is a small, curated, LANGUAGE- or DOMAIN-specific fact set in
+the **exact tier-1 fact shape** — one JSON object per line,
+`{"start":"/c/en/…","rel":"/r/…","end":"/c/en/…","weight":N,"surfaceText":"…"}`,
+with `rel` drawn only from the mapped relations in
+`src/corpus/conceptnet-map.toml`. Because the shape is identical, a tier-2 file
+loads and seeds through the very same `loadSlice()`/`toFacts()` path as the
+tier-1 slice — `tier2/generate.mjs --verify` proves it (each sample loads and
+all its facts seed cleanly, no `ace=none` dead rows).
+
+The idea: a Python repo pulls the `python` corpus so tmct knows "a dict is a
+kind of hash table"; an AWS project pulls `aws` so it knows "S3 is object
+storage, a bucket is part of S3". Language terms unify onto the **shared CS
+concept vocabulary** (`list → array`, `hashmap → hash table`) so specialised
+knowledge connects to the tier-1 graph instead of floating apart.
+
+**To add a corpus:** add an entry to `CORPUSES` in `tier2/generate.mjs` (a list
+of `[subject, relation, concept]` triples) and run `node tier2/generate.mjs
+--verify`. It rewrites `<id>.jsonl` and `manifest.json` (facts count, byte size,
+sha256) in one deterministic pass. Curated data is authored in that file so it
+stays reviewable; a corpus too big to hand-curate is a `fetch` manifest entry
+(URL + sha256, opt-in network — `fetchCorpus()` is the reference downloader).
+
+**How the coordinator should wire tier-2 into `tmct init` (Wave-2 — NOT done
+here):**
+
+1. Add a tier-2-aware seeder next to `seedMemory()` (e.g. `seedTier2(dir, id)`)
+   that reads `corpus/tier2/manifest.json`, resolves the requested corpus's
+   `file` (curated → already on disk; `fetch` → download + `sha256`-verify
+   first, only when network is explicitly enabled), then runs the SAME
+   `loadSlice → toFacts → appendFact` pipeline — **but stamps provenance
+   `corpus:tier2:<id> <rel>`** instead of the hard-coded `corpus:conceptnet …`
+   string in `toFacts()`. (Simplest: give `toFacts` an optional
+   `provenancePrefix` argument, default `"corpus:conceptnet"`.)
+2. Give `tmct init` a `--corpus <id>[,<id>…]` flag (and/or codebase
+   auto-detection: a `requirements.txt`/`pyproject.toml` → `python`, a
+   `pom.xml`/`build.gradle` → `java`, an AWS SDK dep / `serverless.yml` →
+   `aws`). Default stays tier-1-only, offline, $0.
+3. Idempotency is free — `seedMemory`'s content-hashed fact ids and pre-read
+   skip already handle re-seeds; tier-2 rides the same path.
+
+None of that touches `package.json` or the tier-1 budget; tier-2 files are not
+shipped, so they do not count against the ≤ 1.5 MB slice budget.
+
 ## How to regenerate / extend
 
-See `conceptnet/README.md` — one command per route (API vs dump), plus the
-seed-term list to extend. The test suite (`test/corpus-conceptnet.test.mjs`,
-`test/corpus-templates.test.mjs`) guards the contracts: slice/mapping drift,
-en→en shape, the ≤ 1.5 MB budget, template ids/slots, and end-to-end seeding.
+See `conceptnet/README.md` — one command per route (API vs dump), the
+quality-filter pass, plus the seed-term list to extend. The test suite
+(`test/corpus-conceptnet.test.mjs`, `test/corpus-templates.test.mjs`) guards the
+contracts: slice/mapping drift, en→en shape, the ≤ 1.5 MB budget, template
+ids/slots, and end-to-end seeding.

package/corpus/conceptnet/README.md

@@ -46,9 +46,61 @@ see `LICENSE-NOTICE` in this directory for the full attribution.
    facts.
 6. Deterministic output order: `(rel, start, end)`.
 
-## Row counts (committed slice, 2026-07-04)
+## Quality-filter pass (2026-07-05)
+
+`filter-dump.mjs` keeps the DATA honest (tech-seed match, canonical relations,
+budget) but not the SEMANTICS: ConceptNet's crowd-sourced "Verbosity"/Open-Mind
+rows leave sentence-fragment "concepts" and opinion axioms that read as nonsense
+once seeded ("a computer is a kind of dumb", "a class is a kind of elegance",
+"mouse AtLocation taloned_grip_of_owl", "2 is a kind of software"). A second
+pass, `quality-filter.mjs`, removes those by term/relation shape (never per
+row):
+
+- **numeric endpoint** — bare term all digits (`2`, `1000`, `80386`)
+- **single-char endpoint** — bare term length ≤ 1 (`a`, `r`, `m`)
+- **sentence fragment** — ≥ 4 underscore-words on either endpoint
+  (`taloned_grip_of_owl`, `worlds_largest_interconnected_network_of_networks`)
+- **definitional phrase** — `/r/DefinedAs` with a ≥ 3-word object (real
+  `DefinedAs` is a synonym: `cpu → processor`)
+- **opinion object** — `/r/IsA` / `/r/DefinedAs` whose object is in a small
+  evidence-based set (`elegance, evil, gloom, unreality, universalism, dumb,
+  free, junk`) — never a class
+
+**Result: 14,258 → 13,880 rows (378 cut), 1,399,979 → 1,348,361 bytes.** Cuts by
+reason: sentence-fragment 328, single-char 27, numeric 11, opinion-object 8,
+definitional-phrase 4. Of the 4,170 seedable (mapped, `ace≠none`) facts, 286
+noise facts were removed, leaving **3,884 clean seedable facts**. No relation
+disappeared entirely (the drift guard stays satisfied). Re-run any time with:
 
-**14,258 assertions, 1,399,979 bytes** (34,074,917 dump lines scanned;
+```bash
+node corpus/conceptnet/quality-filter.mjs --in-place corpus/conceptnet/slice.jsonl
+```
+
+## Growing the slice toward ~40k facts — the budget blocker
+
+Growing tier-1 to the operator's ~40k-fact target is **feasible data-wise but
+blocked by the committed budget**: the ConceptNet dump S3 endpoint is reachable,
+but `test/corpus-conceptnet.test.mjs` asserts `size <= 1_500_000`, and the clean
+slice is already 1.35 MB (13,880 rows). 40k facts is ≈ 4 MB — 2.6× over the cap.
+Reaching it needs a **Wave-2 policy change** (raise `MAX_BYTES` in
+`filter-dump.mjs` AND the budget assertion in the test), which is out of scope
+for a data-only pass. When the budget is raised, regrow with a bigger seed list
++ budget, then re-run the quality filter:
+
+```bash
+# 1. widen the domain and the budget (edit SEED_TERMS in fetch-slice.mjs, MAX_BYTES in filter-dump.mjs)
+curl -s https://s3.amazonaws.com/conceptnet/downloads/2019/edges/conceptnet-assertions-5.7.0.csv.gz \
+  | gunzip -c | node corpus/conceptnet/filter-dump.mjs > corpus/conceptnet/slice.jsonl
+# 2. re-apply the semantic quality filter
+node corpus/conceptnet/quality-filter.mjs --in-place corpus/conceptnet/slice.jsonl
+# 3. raise the test's budget assertion to match
+```
+
+## Row counts (pre-filter baseline, 2026-07-04)
+
+**14,258 assertions, 1,399,979 bytes** — the raw filter-dump output, BEFORE the
+2026-07-05 quality-filter pass above trimmed it to 13,880 rows / 1,348,361 bytes
+(34,074,917 dump lines scanned;
 28,802 unique en→en seed assertions matched = 4,170 mappable + 24,632
 `ace="none"`; ALL 4,170 mappable kept, 10,088 none-rows fill the budget).
 29 of the 31 non-filtered canonical relations are present:

package/corpus/conceptnet/quality-filter.mjs

@@ -0,0 +1,95 @@
+#!/usr/bin/env node
+// quality-filter.mjs — a SECOND-PASS noise filter over corpus/conceptnet/slice.jsonl.
+// NOT part of the product path — a maintainer tool, run by hand, result committed.
+//
+// The committed slice is stream-filtered from the ConceptNet dump by
+// filter-dump.mjs (tech-domain seed match + canonical relations + budget). That
+// pass keeps the DATA honest but not the SEMANTICS: ConceptNet's crowd-sourced
+// "Verbosity"/Open-Mind rows leave sentence-fragment "concepts" and opinion
+// axioms in the slice that read as nonsense once they become memory facts —
+// e.g. "a computer is a kind of dumb", "a class is a kind of elegance",
+// "mouse AtLocation taloned_grip_of_owl", "2 is a kind of software".
+//
+// This pass removes exactly those, by term/relation shape only (never by hand
+// per row), so it is reproducible:
+//
+//   node corpus/conceptnet/quality-filter.mjs < corpus/conceptnet/slice.jsonl > slice.clean.jsonl
+//   # or in place (what produced the committed clean slice):
+//   node corpus/conceptnet/quality-filter.mjs --in-place corpus/conceptnet/slice.jsonl
+//
+// Cut rules (a row is DROPPED when ANY applies):
+//   1. numeric endpoint      — start/end bare term is all digits ("2", "1000")
+//   2. single-char endpoint  — bare term length <= 1 ("a", "r", "m")
+//   3. sentence fragment     — bare term is >= 4 underscore-words on EITHER
+//                              endpoint ("taloned_grip_of_owl",
+//                              "worlds_largest_interconnected_network_of_networks")
+//   4. definitional phrase   — /r/DefinedAs whose object is >= 3 words
+//                              (real DefinedAs is a synonym: cpu->processor)
+//   5. opinion object        — /r/IsA or /r/DefinedAs whose object is one of a
+//                              small, evidence-based OPINION set (adjectives /
+//                              value words that never name a class)
+//
+// Rules 1-3 apply to every relation (they only ever remove junk); 4-5 are the
+// definitional band the sims flagged. Stats land on stderr; JSONL on stdout.
+
+import { readFile, writeFile } from "node:fs/promises";
+import { createInterface } from "node:readline";
+
+const bareTerm = (uri) => String(uri || "").replace(/^\/c\/en\//, "");
+const words = (t) => t.split("_").filter(Boolean).length;
+
+// Evidence-based: the only 1-word IsA/DefinedAs objects in the committed slice
+// that are opinions/adjectives rather than classes. Kept explicit (not a POS
+// heuristic) so it never cuts a legitimate abstract class like "abstraction",
+// "cognition" or "relation".
+export const OPINION_OBJECTS = new Set([
+  "elegance", "evil", "gloom", "unreality", "universalism", "dumb", "free", "junk",
+]);
+
+/** Why this row is noise, or null to keep it. Pure function of the row shape. */
+export function cutReason(row) {
+  const s = bareTerm(row.start);
+  const e = bareTerm(row.end);
+  for (const t of [s, e]) {
+    if (/^\d+$/.test(t)) return "numeric-endpoint";
+    if (t.length <= 1) return "single-char-endpoint";
+    if (words(t) >= 4) return "sentence-fragment";
+  }
+  if (row.rel === "/r/DefinedAs" && words(e) >= 3) return "definitional-phrase";
+  if ((row.rel === "/r/IsA" || row.rel === "/r/DefinedAs") && OPINION_OBJECTS.has(e)) return "opinion-object";
+  return null;
+}
+
+async function readLines(stream) {
+  const rl = createInterface({ input: stream, crlfDelay: Infinity });
+  const rows = [];
+  for await (const raw of rl) {
+    const line = raw.trim();
+    if (line) rows.push(JSON.parse(line));
+  }
+  return rows;
+}
+
+const isMain = process.argv[1] && import.meta.url === new URL(`file://${process.argv[1]}`).href;
+if (isMain) {
+  const inPlace = process.argv.includes("--in-place");
+  const fileArg = process.argv.slice(2).find((a) => !a.startsWith("--"));
+  const rows = inPlace
+    ? (await readFile(fileArg, "utf8")).split("\n").filter((l) => l.trim()).map((l) => JSON.parse(l))
+    : await readLines(process.stdin);
+
+  const byReason = new Map();
+  const kept = [];
+  for (const row of rows) {
+    const reason = cutReason(row);
+    if (reason) { byReason.set(reason, (byReason.get(reason) || 0) + 1); continue; }
+    kept.push(row);
+  }
+  const text = kept.map((r) => JSON.stringify(r)).join("\n") + "\n";
+  if (inPlace) await writeFile(fileArg, text);
+  else process.stdout.write(text);
+
+  const cut = rows.length - kept.length;
+  console.error(`quality-filter: ${rows.length} rows in, ${kept.length} kept, ${cut} cut (${text.length} bytes out)`);
+  for (const [reason, n] of [...byReason.entries()].sort((a, b) => b[1] - a[1])) console.error(`  ${reason}: ${n}`);
+}