@333eco/corpus 2.0.0 → 2.1.1

package/README.md

@@ -48,6 +48,22 @@ manufactured on a schedule.
 | Tool | Returns |
 | --- | --- |
 | `search_corpus` | matching documents, provenance envelope, and an excerpt around each match |
+
+⭐ **`search_corpus` matches the phrase first, then all terms.** An exact substring hit
+always wins and ranks above everything else — which keeps `B-Heart` and `Re-Tip` precise,
+since the tokeniser holds hyphenated marks together. If the phrase is absent, a document
+matches when it contains **every** term somewhere (AND, never OR), ranked by how tightly
+those terms cluster. Each hit reports `match: "phrase" | "terms"`, because a term match's
+excerpt need not contain the words you typed.
+
+⚠️ **This replaced a bare `indexOf`, and a real caller paid for it.** The first external
+client to reach the hosted endpoint searched `"gratitude alignment human wellbeing kindness"`
+and got **zero results** — while `gratitude` alone returns 109, `alignment` 50, `kindness` 50.
+Nothing was missing from the corpus; the matcher demanded that exact five-word string appear
+verbatim. **A zero result was recording the matcher's limits while being read as a gap.**
+
+⭐ On a zero result the response now names **which terms appear in no document at all**
+(`absent_terms`), so a dead end says something instead of nothing.
 | `get_document` | one document in full — canonical text, never a summary |
 | `list_documents` | slugs, titles, genres, licences, provenance summaries |
 | `list_predictions` | the research program's pre-registered predictions, with falsifiers and status |
@@ -260,6 +276,104 @@ npm run deploy     # sync + wrangler deploy
 { "mcpServers": { "corpus": { "url": "https://corpus.333.eco/mcp" } } }
 ```
 
+## Telling us what is missing, or broken
+
+```sh
+npx @333eco/corpus --report-gap "what you looked for and did not find"
+npx @333eco/corpus --report-bug "what went wrong, and what you expected instead"
+```
+
+⭐⭐ **A command, not telemetry, and the difference is the whole point.** The most
+useful thing a corpus server can learn is what someone went looking for and did
+not find. The hosted endpoint learns that from its own callers as a property of
+being the server they called. This package runs on *your* machine, so collecting
+it here would be an outbound report about your private reading — and the guard
+against that is not a consent prompt or an opt-out flag. **It is that the serving
+path cannot reach the code that sends.** `server.mjs` loads `report.mjs` with
+a dynamic import inside the argv branch, so a normal session never reads the file
+off disk at all.
+
+⭐ **Both flags share one module and one dynamic import**, so the second kind
+added no second way into the network — which is why generalising was right and
+copying the file would have been wrong.
+
+The command prints the entire payload before sending it, and the payload is the
+text you typed plus the version you have. ⚠️ **A bug report carries exactly the
+same payload as a gap report, deliberately** — attaching a node version and
+platform would be useful to whoever fixes it, but it would give the command two
+different promises about what it sends, and the promise is the valuable part.
+Anything about your environment that matters, put in the text; then you have
+said it on purpose. No machine id, no username, no
+hostname, no path. ⭐ The receiving end deliberately does not record the country
+it could resolve for free: a voluntary note about a missing document has no use
+for where the sender was standing, and collecting a thing because it is available
+is how a narrow purpose widens.
+
+⚠️ **The honest limit, because the claim changed shape when this was added.**
+Before it, *"this package makes no network call"* was verifiable by
+`grep -r fetch src/` returning nothing — the strongest kind of evidence, since it
+needs no reasoning. The claim is now narrower: **there is exactly one `fetch` in
+the package, it is in `src/report.mjs`, and that file is imported from exactly
+one place — a branch requiring an explicit flag.** Still checkable in under a
+minute, but it is a chain of two facts rather than one absence.
+
+## What the remote server records
+
+⛔ **The npm package records nothing and sends nothing.** `npx @333eco/corpus`
+runs on your machine, reads a local file, and makes no outbound request of any
+kind. Everything in this section is about `corpus.333.eco` and only about it.
+
+The asymmetry is deliberate. The hosted endpoint already sees every request it
+answers, so writing down what it was asked adds no reach it did not have. The
+same lines inside the package would be an outbound report about a stranger's
+private reading, which is a different artifact — and not one this is going to
+become.
+
+⭐⭐ **No per-caller identity is computed anywhere.** The client label is the
+*software's* name, taken from the `clientInfo` it volunteers at handshake —
+`claude-code`, `cursor` — never an IP, never a hash of one, never a cookie. Every
+user of a given client is one label. That is not a promise to behave well: there
+is no code path in `worker/src/telemetry.mjs` that derives a per-caller id, so
+there is nothing to leak, sell, subpoena or regret later. The question the server
+wants answered is *which clients reach it*, and that question needs no persons in
+it.
+
+⭐⭐ **The only search text ever stored is a search that found nothing.** The
+reason to log queries at all is to learn what the corpus is missing; a query that
+*succeeded* tells you only what a caller was reading, which is their business.
+So the successful query has no storage path — an absent branch, not a redaction
+step someone has to remember to keep. Remove the enforcer and nothing breaks,
+because there is no enforcer.
+
+| Channel | Carries | Why it exists |
+| --- | --- | --- |
+| Analytics Engine | one row per JSON-RPC call: method, tool, slug, client label, country, protocol, corpus version, result count, error flag, duration — plus the query text **when and only when it matched nothing** | counting; queried by SQL, stays at Cloudflare |
+| `thonly.org/api/track` | `corpus_connect` (a handshake) and `corpus_error` (the corpus asset failed to load) | the two things worth interrupting someone about |
+
+⛔ **Not one beacon per tool call.** An agent working through the corpus fires
+dozens of calls in seconds, and a notification channel that reports each of them
+is a channel nobody reads. The beacon fires on the *handshake*, and the receiving
+function pushes only the **first sighting of a client label**, counting every one
+after it in silence — so a notification means *a client we have never seen
+appeared*. `corpus_error` is exempt and always pushes: it is rare by construction,
+and silence is the wrong default for an outage.
+
+⚠️ **A dead beacon must not look like a quiet one.** If the receiving allowlist
+changes, the POST 403s and the pushes simply stop — indistinguishable from *no new
+clients this week*, which is exactly the reading that would let it stay broken for
+months. So the delivery status is written to Analytics Engine as its own row:
+silence on the phone is then something you can go and check rather than infer.
+
+⚠️ **`ANALYTICS` is unbound under `wrangler dev` without `--remote`.** The module
+degrades to a no-op rather than throwing, so a local session looks entirely normal
+and records nothing — expected, and worth knowing before reading an empty dataset
+as a finding.
+
+The endpoint discloses all of this in its own `GET /` response, under `records`.
+A privacy policy is a page someone has to go and find; this is the endpoint
+describing itself, in the one response a caller gets for free before doing
+anything, so the disclosure travels with the thing it is about.
+
 ## Client configuration
 
 ```json